Revision d5a1a059

View differences:

latex/note.tex
154 154

  
155 155
        \subsubsection{Definition}
156 156
            \textbf{SDK}
157

  
157 158
            \textbf{OpenWrt Build System} = build system for the OpenWrt Linux distribution
158
            \textbf{OpenWrt Linux distribution}
159
            \textbf{Instruction set architecture} = the translator that turns the code a machine is running into instructions a processor can understand and execute. It's an important part of a processor.
160
            \textbf{ARM} = a simpler instruction set architecture comparing to the \texttt{x86}. It has the power efficiency. It uses \texttt{Reduced Instruction Set Computing - RISC}
161
            \textbf{i386} = that chips win over ARM chips on the performance. It uses \texttt{Complex Instruction Set Computing - CISC}.
162
            \textbf{Debian on ARM}
163 159

  
164
        \subsubsection{Compile packages with OpenWrt SDK}
165
            Steps I performed
160
            \textbf{OpenWrt Linux distribution}
166 161

  
167
            \begin{itemize}
168
                \item Test building package with OpenWrt SDK BB on Linux Mint 17 (based on Ubuntu Trusty - 14)
169
                    \begin{lstlisting}
170
svn export svn://svn.openwrt.org/openwrt/packages/utils/sed package/sed
171
grep DEPENDS package/sed/Makefile
172
make package/sed/compile # GOT ERROR HERE
162
            \textbf{Instruction set architecture} = the translator that turns the code a machine is running into instructions a processor can understand and execute. It's an important part of a processor.
173 163

  
174
configure: error: in `/home/quynh/Thesis/openwrt-bb/build_dir/target-mipsel_mips32_uClibc-0.9.33.2/sed-4.2.1':
175
configure: error: C compiler cannot create executables
176
See `config.log' for more details.
177
make[2]: *** [/home/quynh/Thesis/openwrt-bb/build_dir/target-mipsel_mips32_uClibc-0.9.33.2/sed-4.2.1/.configured_] Error 77
178
make[2]: Leaving directory `/home/quynh/Thesis/openwrt-bb/package/sed'
179
make[1]: *** [package/sed/compile] Error 2
180
make[1]: Leaving directory `/home/quynh/Thesis/openwrt-bb'
181
make: *** [package/sed/compile] Error 2
164
            \textbf{ARM} = a simpler instruction set architecture comparing to the \texttt{x86}. It has the power efficiency. It uses \texttt{Reduced Instruction Set Computing - RISC}
182 165

  
183
# SOLUTION
184
sudo apt-get install ccache
185
                    \end{lstlisting}
166
            \textbf{i386} = that chips win over ARM chips on the performance. It uses \texttt{Complex Instruction Set Computing - CISC}.
186 167

  
187
                \item Test building package with Ubuntu 14 in VMWare. I got the same error as when running with Linux Mint 17
168
            \textbf{Debian on ARM}
188 169

  
189
            \end{itemize}
190 170

  
191
        \subsubsection{Compile OpenWRT from OpenWRT CC toolchain}
171
        \subsubsection{Compile OpenWRT CC toolchain to get OpenWrt SDK}
192 172
            Compile the \texttt{OpenWrt Chaos Calmer}. More information can be found in \href{http://wiki.openwrt.org/doc/howto/buildroot.exigence}{OpenWrt build system - Installation}.
193 173

  
194 174
            \href{http://wiki.openwrt.org/doc/howto/build}{This tutorial} shows how to build \texttt{OpenWrt SDK from source}
......
222 202
# 3rd run - NO ERROR
223 203
            \end{lstlisting}
224 204

  
225
            Got \textbf{target-arm\_cortex-a9\_uClibc-0.9.33.2\_eabi} in \texttt{build\_dir}
205
            Inside the directory \textbf{target-arm\_cortex-a9\_uClibc-0.9.33.2\_eabi} in \texttt{build\_dir}, we have \texttt{OpenWrt-SDK-bcm53xx\_gcc-4.8-linaro\_uClibc-0.9.33.2\_eabi.Linux-x86\_64}. This one is the \texttt{compiled toolchain}, or the \texttt{OpenWrt SDK, version Chaos Calmer}
206

  
207
    \subsubsection{Compile packages with OpenWrt SDK}
208
            Steps I performed
209

  
210
            \begin{itemize}
211
                \item Test building package with OpenWrt SDK BB on Linux Mint 17 (based on Ubuntu Trusty - 14)
212
                    \begin{lstlisting}
213
svn export svn://svn.openwrt.org/openwrt/packages/utils/sed package/sed
214
grep DEPENDS package/sed/Makefile
215
make package/sed/compile # GOT ERROR HERE
216

  
217
configure: error: in `/home/quynh/Thesis/openwrt-bb/build_dir/target-mipsel_mips32_uClibc-0.9.33.2/sed-4.2.1':
218
configure: error: C compiler cannot create executables
219
See `config.log' for more details.
220
make[2]: *** [/home/quynh/Thesis/openwrt-bb/build_dir/target-mipsel_mips32_uClibc-0.9.33.2/sed-4.2.1/.configured_] Error 77
221
make[2]: Leaving directory `/home/quynh/Thesis/openwrt-bb/package/sed'
222
make[1]: *** [package/sed/compile] Error 2
223
make[1]: Leaving directory `/home/quynh/Thesis/openwrt-bb'
224
make: *** [package/sed/compile] Error 2
225

  
226
# SOLUTION
227
sudo apt-get install ccache
228
                    \end{lstlisting}
229

  
230
                \item Test building package with Ubuntu 14 in VMWare. I got the same error as when running with Linux Mint 17
231

  
232
                \item Install the library \texttt{ccache} manually would solve the problem
233

  
234
                \item After successfully compile the package, you will find *.ipk in the folder [./bin/bcm53xx/packages/base/]. Where \texttt{bcm53xx} is the code of the target device. It can also be \texttt{ar71xx} or something else.
235
            \end{itemize}
226 236

  
227 237
        \subsubsection{Install QEMU}
228 238
            Check out \href{http://wiki.qemu.rg/Download}{Wiki} for instruction. It will took a while to install, since QEMU required several packages that are not present on my system.
229 239

  
230 240
            After installation, I have a bunch of binary file \texttt{quem-*} in \texttt{/usr/local/bin}
231 241

  
242
        \subsubsection{Generate OpenWrt Image with QEMU}
243
            This \href{http://wiki.openwrt.org/doc/howto/obtain.firmware}{Obtaining Firmware} page summarizes different ways to obtain \texttt{OpenWrt Firmware Image (OFI)} file. After having the OFI, then we can feed it into QEMU.
244

  
245
            This \href{http://wiki.openwrt.org/doc/howto/obtain.firmware.generate}{Image Generator tutorial} has instruction on how to generate Image Builder firmware.
246

  
247
            Since when compiling OpenWrt toolchain, I didn't modify anything in \texttt{menuconfig}, so the target system of the SDK are bcm53xx.
248

  
249
            The profile for bcm53xx can be found \href{https://dev.openwrt.org/browser/trunk/target/linux/bcm53xx/profiles/100-Generic.mk?rev=42143}{here}. I (randomly) pick \texttt{BCM53010}
250

  
251
            \begin{lstlisting}
252
cd openwrt
253
wget https://downloads.openwrt.org/chaos_calmer/15.05/bcm53xx/generic/OpenWrt-ImageBuilder-15.05-bcm53xx.Linux-x86_64.tar.bz2
254
tar -xvjf OpenWrt-ImageBuilder-15.05-bcm53xx.Linux-x86_64.tar
255
cd OpenWrt-ImageBuilder-15.05-bcm53xx.Linux-x86_64
256
mv OpenWrt-ImageBuilder-15.05-bcm53xx.Linux-x86_64 ImageBuilder-15.05-bcm53xx.Linux-x86_64
257

  
258
# If you want to modify 'profiles', then:
259
vim ./ImageBuilder-15.05-bcm53xx.Linux-x86_64/target/linux/bcm53xx/profiles/100-Generic.mk
260
make image PROFILE=Generic
261

  
262
# The image is in ./bin directory
263
# ERROR: Can't find any *.bin file
264
            \end{lstlisting}
265

  
266
            I then tried to generate OpenWrt Firmware Image for \texttt{ar71xx} just like the original tutorial. This time I got all the *.bin file.
267

  
268
        \subsubsection{Run OpenWrt Firmware Image with QEMU}
269
            \begin{lstlisting}
270
qemu-system-x86_64 -kernel openwrt-15.05-ar71xx-generic-vmlinux.elf
271
            \end{lstlisting}
272

  
232 273
    \subsection{Questions}
233 274
        What is \emph{buildroot-compatible package directory}?
234 275

  
......
265 306
    \subsection{Misc}
266 307
        [11:10 - 11:30] I'm doing shitty things now. Removing gcc-base, and as a result most packages are being removed: chrome, mozilla, gedit,  tortoise, skype, python, libqt4, networkmanager. Now I don't have the internet connection anymore.
267 308

  
309
    \include{note_w03}
310

  
268 311
\end{document}
latex/note_w03.tex
1
%%%%%%%%%%%%%%%%%%
2
% WEEK 3
3
%%%%%%%%%%%%%%%%%%
4
\section{Week 3}
5
    Read \href{http://www.ccs.neu.edu/home/noubir/Courses/CS6710/S12/material/OpenWrt_Dev_Tutorial.pdf}{OpenWrt Dev Tutorial}
6

  
7
    Read \href{http://landley.net/aboriginal/presentation.html#cross_advantages}{Developing for non-x86 targets using QEMU}
8

  
9
    \href{http://wiki.prplfoundation.org/wiki/Building_OpenWrt}{Bilding OpenWrt}: screenshots, and detaield instruction on how to build OpenWrt image, and how to run that image with Qemu
10

  
11
    \subsection{QEMU Manual}
12
        \subsubsection{Definitions}
13
            \textbf{QEMU} = QEMU is a generic and open source machine emulator and virtualizer. QEMU has the advantage of being able to run either as a pure emulator or as a native virtual machine. \footnote{http://wiki.qemu.org/Main\_Page}
14
            \begin{itemize}
15
                \item When used as a machine emulator, QEMU can run OSes and programs made for one machine (e.g. an ARM board) on a different machine (e.g. your own PC). By using dynamic translation, it achieves very good performance.
16

  
17
                \item When used as a virtualizer, QEMU achieves near native performances by executing the guest code directly on the host CPU. QEMU supports virtualization when executing under the Xen hypervisor or using the KVM kernel module in Linux. When using KVM, QEMU can virtualize x86, server and embedded PowerPC, and S390 guests.
18
            \end{itemize}
19

  
20
            \textbf{Emulator} = a hardware or software that enables one computer system (called the host) to behave like another computer system (called the guest). An emulator typically enables the host system to run software or use peripheral devices designed for the guest system. \footnote{https://en.wikipedia.org/wiki/Emulator}
21

  
22
            \textbf{Virtualizer (or virtualization machine)} allows an unmodified operating system with all of its installed software to run in a special environment, on top of your existing operating system. This environment, called a "virtual machine", is created by the virtualization software by intercepting access to certain hardware components and certain features. The physical computer is then usually called the "host", while the virtual machine is often called a "guest". Most of the guest code runs unmodified, directly on the host computer, and the guest operating system "thinks" it's running on real machine. \footnote{https://www.virtualbox.org/wiki/Virtualization}
23

  
24
            \textbf{Peripheral devices} are defined as a computer device, such as a keyboard or printer, that is not part of the essential computer (i.e., the memory and microprocessor). These auxiliary devices are intended to be connected to the computer and used.
25

  
26
            \textbf{Cross Compiler} is a compiler capable of creating executable code for a platform other than the one on which the compiler is running. For example, a compiler that runs on a Windows 7 PC but generates code that runs on Android smartphone is a cross compiler
27

  
28
            \textbf{Realview} = this platform is intended to be used with Qemu (realview-eb-mpcore machine).
29

  
30
        \subsubsection{Questions on QEMU}
31
            \textbf{What is the difference between \texttt{qemu-arm} vs \texttt{qemu-system-arm}}
32

  
33
                The first is used to execute ARM binary files, and the second to boot the ARM OS. \footnote{http://opensourceforu.efytimes.com/2011/06/qemu-for-embedded-systems-development-part-1/}
34

  
35

  
36
    \subsection{Load realview target with QEMU}
37
        \begin{lstlisting}
38
FILENAME="OpenWrt-ImageBuilder-15.05-rc3-realview.Linux-x86_64"
39

  
40
wget "https://downloads.openwrt.org/chaos_calmer/15.05-rc3/realview/generic/$FILENAME.tar.bz2"
41
tar -xvjf $FILENAME.tar.bz2
42
rm $FILENAME.tar.bz2
43
cd $FILENAME
44

  
45

  
46
cd ./bin/realview
47
ELFFILE="openwrt-15.05-rc3-realview-vmlinux.elf"
48

  
49
# Boot with this command
50
qemu-system-arm -M realview-eb-mpcore -kernel $ELFFILE -net nic -net user -nographic
51
# Got error: end Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(0,0)
52

  
53
# In the website, there is another *.elf file
54
# With this one, the kernel is boot successfully.
55
wget https://downloads.openwrt.org/chaos_calmer/15.05-rc3/realview/generic/openwrt-15.05-rc3-realview-vmlinux-initramfs.elf
56

  
57

  
58
        \end{lstlisting}
59

  
60
    \subsection{OpenWrt under QEMU}
61
    https://grenville.wordpress.com/2013/01/06/openwrt-under-qemu-on-freebsd/
62

  
63
    \subsection{Transfering files between host and QEMU instance}
64
    \href{http://www.cnx-software.com/2011/10/02/how-to-transfer-files-between-host-and-qemu-via-ssh-and-nfs/}{OpenWRT under Qemu on FreeBSD}: screenshots and commands with an overview on how to build Realview image of OpenWrt and run it under Qemu.
65

  
66
        \begin{lstlisting}
67
# Start OpenWrt instance
68
cd ~/Thesis/imagebuilder/OpenWrt-ImageBuilder-15.05-rc3-realview.Linux-x86_64/bin/realview
69
ELFFILE="openwrt-15.05-rc3-realview-vmlinux-initramfs.elf"
70
qemu-system-arm -M realview-eb-mpcore -kernel $ELFFILE -net nic -net user -nographic
71

  
72
# ELFFILE="openwrt-15.05-rc3-realview-vmlinux.elf" will created error
73
# Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(0,0)
74

  
75
# Try to boot with ext4
76
qemu-system-arm -M realview-eb-mpcore -kernel $ELFFILE -net nic -net user -nographic -sd openwrt-realview-sdcard.img
77

  
78
# From ifconfig - vmnet1 - this one DOES NOT WORK
79
ssh -p 2222 quynh@192.168.52.1
80
# From ifconfig - vmnet8
81
ssh -p 2222 quynh@192.168.166.1
82
# This one allow to access the host file system
83
ssh quynh@192.168.166.1
84
# Copy file from host to QEMU
85
scp quynh@192.168.166.1:~/sed_4.2.1-1_realview.ipk ~/sed.ipk
86
        \end{lstlisting}
87

  
88
    \subsection{Install package on the running OpenWRT instance on QEMU}
89
        Read page 11 on \href{http://www.ccs.neu.edu/home/noubir/Courses/CS6710/S12/material/OpenWrt_Dev_Tutorial.pdf}{OpenWrt Dev Tutorial from NEU}
90

  
91
        \begin{lstlisting}
92
# Download the precompiled SDK
93
FILENAME="OpenWrt-SDK-15.05-realview_gcc-4.8-linaro_uClibc-0.9.33.2_eabi.Linux-x86_64"
94
wget "https://downloads.openwrt.org/chaos_calmer/15.05/realview/generic/$FILENAME.tar.bz2"
95
tar -xvjf $FILENAME.tar.bz2
96
rm $FILENAME.tar.bz2
97
cd $FILENAME
98

  
99
# Download package 'sed' and compile it
100
svn export svn://svn.openwrt.org/openwrt/packages/utils/sed package/sed
101
grep DEPENDS package/sed/Makefile
102
make package/sed/compile
103

  
104
# Transfer the compiled package 'sed' to running instance OpenWrt
105
# From the root@OpenWrt console
106
scp quynh@192.168.166.1:~/sed_4.2.1-1_realview.ipk ~/sed.ipk
107

  
108
# Install the ipk
109
opkg install ~/sed.ipk
110
        \end{lstlisting}
111

  
112
        \textbf{Problem: Only have 0kb available on filesystem /overlay, pkg sed needs 48}
113

  
114
            https://forum.openwrt.org/viewtopic.php?id=58545
115

  
116
            I tried to boot with \emph{image}, but the problem is still there
117
            \begin{lstlisting}
118
EFLFILE="openwrt-15.05-ar71xx-generic-vmlinux.elf"
119
qemu-system-arm -M realview-eb-mpcore -kernel $ELFFILE -net nic -net user -nographic -sd openwrt-realview-sdcard.img
120
            \end{lstlisting}
121

  
122
            Some reasons:
123
            \begin{itemize}
124
                \item However (as far as I understand), the image is mounted as an overlay image which (for what I could gather on the web) means it's readonly, and therefore, I can't install new packages.\footnote{https://forum.openwrt.org/viewtopic.php?id=58545}
125

  
126
                \item You are running a read only image, you need to add an overlay filesystem over that you that you can have full read and write functions. \footnote{https://forum.openwrt.org/viewtopic.php?id=56891}
127

  
128
                \item If I want to store something on my target I do this on some other mounted media (usually flash). Others might write the image into the devices flash and allow changes to be saved into flash. \footnote{http://buildroot.uclibc.narkive.com/pp4A78C6/creating-a-filesystem-for-real-time-linux-using-qemu}
129
            \end{itemize}
130

  
131
        \textbf{Qemu + overlay}
132

  
133
        \textbf{SD card image and Qemu}
134
            \href{https://en.wikibooks.org/wiki/QEMU/Images#Creating_an_image}{Creating an image - Wikibooks}
135

  
136
            \begin{lstlisting}
137
qemu-img create -f qcow2 sdcard.img 500M
138

  
139

  
140
qemu-system-arm -M realview-eb-mpcore -kernel openwrt-15.05-rc3-realview-vmlinux-initramfs.elf \
141
-nographic -m 1024M \
142
-sd sdcard.img \
143
--append "console=ttyAMA0 verbose debug root=/dev/mmcblk0p1"
144

  
145
            \end{lstlisting}
146

  
147
        \textbf{Test: Build an image with additional package}
148
            By this way, we do not need to install additional package with a running instance of OpenWrt
149

  
150
            Where is the Profiles.mk
151

  
152
            \begin{lstlisting}
153
# OpenWrt on Qemu with the default image for Realview Target
154
root@OpenWrt:/# fdisk
155
/bin/ash: fdisk: not found
156

  
157
# OpenWrt on Qemu with the image + fdisk package
158
# Inside the ImageBuilder folder
159

  
160
make image PACKAGES="fdisk"
161

  
162
            \end{lstlisting}
163

  
164
        \textbf{Generate openwrt-15.05-rc3-realview-vmlinux-initramfs.elf}
165
            The current Image Builder only generate openwrt-15.05-rc3-realview-vmlinux.elf, and not openwrt-15.05-rc3-realview-vmlinux-initramfs.elf
166

  
167
            Some similar forum threads:
168
            \begin{itemize}
169
                \item \footnote{https://forum.openwrt.org/viewtopic.php?id=53931}
170
            \end{itemize}
171

  
172
        \textbf{Recompile the OpenWrt toolchain to obtain openwrt-15.05-rc3-realview-vmlinux-initramfs.elf}
173
            By selecting the package that I want to include from $make menuconfig$, then that additional package will be available in the \texttt{initranfs.elf} file.
174

  
175
        \textbf{Add additional package to the compiled toolchain}
176
            Ways to do:
177
            \begin{itemize}
178
                \item Select the package/library from make menuconfig
179
                \item Use the $scripts/diffconfig.sh$ to generate \texttt{config.diff} file. Append the \texttt{config.diff} to the original \texttt{<build dir>/.config}. Then $make defconfig$ will generate the full \texttt{.config} file.
180
                \item Check out guideline on \href{http://wiki.openwrt.org/doc/howto/build}{Updating Feeds section}
181
            \end{itemize}
182

  
183
            Why does not the "Update Feeds" have \texttt{base} as in repo of \href{https://downloads.openwrt.org/chaos_calmer/15.05-rc3/realview/generic/packages/}{realview/generic/packages}
184

  
185
            Where is \texttt{feeds.conf}
186

  
187
            \begin{lstlisting}
188
quynh in ~/Thesis/cc-openwrt
189
$ ./scripts/feeds usage
190
$ ./scripts/feeds list -n
191
packages
192
luci
193
routing
194
telephony
195
management
196
quynh in ~/Thesis/cc-openwrt
197
$ ./scripts/feeds list -s
198
packages src-git  7f3ef93  https://github.com/openwrt/packages.git;for-15.05
199
luci     src-git  af9f093  https://github.com/openwrt/luci.git;for-15.05
200
routing  src-git  5876cd3  https://github.com/openwrt-routing/packages.git;for-15.05
201
telephony src-git  d4ca5e5  https://github.com/openwrt/telephony.git;for-15.05
202
management src-git  ab76d57  https://github.com/openwrt-management/packages.git;for-15.05
203

  
204
            \end{lstlisting}
205

  
206
        \textbf{Save the configuration information}
207
            \href{http://wiki.openwrt.org/doc/howto/build}{Read more in the official document - How to Build}
208

  
209
            \begin{lstlisting}
210
# In folder openwrt-cc
211
# After selecting all the changes in make menuconfig
212

  
213
# Export the diffconfig to file
214
./scripts/diffconfig.sh > diffconfig
215
            \end{lstlisting}
216

  
217
        \textbf{How to 'make' without re-compiling}
218

  
219

  
220

  
221

  
222

  
223

  
224

  
225

  
226

  

Also available in: Unified diff