Thesis » QuynhNguyen-MS

%!TEX root = note.tex

%%%%%%%%%%%%%%%%%%

% WEEK 3

%%%%%%%%%%%%%%%%%%

\section{Week 3}

    Read \href{http://www.ccs.neu.edu/home/noubir/Courses/CS6710/S12/material/OpenWrt_Dev_Tutorial.pdf}{OpenWrt Dev Tutorial}

    Read \href{http://landley.net/aboriginal/presentation.html#cross_advantages}{Developing for non-x86 targets using QEMU}

    \href{http://wiki.prplfoundation.org/wiki/Building_OpenWrt}{Bilding OpenWrt}: screenshots, and detaield instruction on how to build OpenWrt image, and how to run that image with Qemu

    \subsection{QEMU Manual}

        \subsubsection{Definitions}

            \textbf{QEMU} = QEMU is a generic and open source machine emulator and virtualizer. QEMU has the advantage of being able to run either as a pure emulator or as a native virtual machine. \footnote{http://wiki.qemu.org/Main\_Page}

            \begin{itemize}

                \item When used as a machine emulator, QEMU can run OSes and programs made for one machine (e.g. an ARM board) on a different machine (e.g. your own PC). By using dynamic translation, it achieves very good performance.

                \item When used as a virtualizer, QEMU achieves near native performances by executing the guest code directly on the host CPU. QEMU supports virtualization when executing under the Xen hypervisor or using the KVM kernel module in Linux. When using KVM, QEMU can virtualize x86, server and embedded PowerPC, and S390 guests.

            \end{itemize}

            \textbf{Emulator} = a hardware or software that enables one computer system (called the host) to behave like another computer system (called the guest). An emulator typically enables the host system to run software or use peripheral devices designed for the guest system. \footnote{https://en.wikipedia.org/wiki/Emulator}

            \textbf{Virtualizer (or virtualization machine)} allows an unmodified operating system with all of its installed software to run in a special environment, on top of your existing operating system. This environment, called a "virtual machine", is created by the virtualization software by intercepting access to certain hardware components and certain features. The physical computer is then usually called the "host", while the virtual machine is often called a "guest". Most of the guest code runs unmodified, directly on the host computer, and the guest operating system "thinks" it's running on real machine. \footnote{https://www.virtualbox.org/wiki/Virtualization}

            \textbf{Peripheral devices} are defined as a computer device, such as a keyboard or printer, that is not part of the essential computer (i.e., the memory and microprocessor). These auxiliary devices are intended to be connected to the computer and used.

            \textbf{Cross Compiler} is a compiler capable of creating executable code for a platform other than the one on which the compiler is running. For example, a compiler that runs on a Windows 7 PC but generates code that runs on Android smartphone is a cross compiler

            \textbf{Realview} = this platform is intended to be used with Qemu (realview-eb-mpcore machine).

        \subsubsection{Questions on QEMU}

            \textbf{What is the difference between \texttt{qemu-arm} vs \texttt{qemu-system-arm}}

                The first is used to execute ARM binary files, and the second to boot the ARM OS. \footnote{http://opensourceforu.efytimes.com/2011/06/qemu-for-embedded-systems-development-part-1/}

    \subsection{Load realview target with QEMU}

        \begin{lstlisting}

FILENAME="OpenWrt-ImageBuilder-15.05-rc3-realview.Linux-x86_64"

wget "https://downloads.openwrt.org/chaos_calmer/15.05-rc3/realview/generic/$FILENAME.tar.bz2"

tar -xvjf $FILENAME.tar.bz2

rm $FILENAME.tar.bz2

cd $FILENAME

cd ./bin/realview

ELFFILE="openwrt-15.05-rc3-realview-vmlinux.elf"

# Boot with this command

# Got error: end Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(0,0)

# In the website, there is another *.elf file

# With this one, the kernel is boot successfully.

wget https://downloads.openwrt.org/chaos_calmer/15.05-rc3/realview/generic/openwrt-15.05-rc3-realview-vmlinux-initramfs.elf

        \end{lstlisting}

    \subsection{OpenWrt under QEMU}

    https://grenville.wordpress.com/2013/01/06/openwrt-under-qemu-on-freebsd/

    \subsection{Transfering files between host and QEMU instance}

    \href{http://www.cnx-software.com/2011/10/02/how-to-transfer-files-between-host-and-qemu-via-ssh-and-nfs/}{OpenWRT under Qemu on FreeBSD}: screenshots and commands with an overview on how to build Realview image of OpenWrt and run it under Qemu.

        \begin{lstlisting}

# Start OpenWrt instance

cd ~/Thesis/imagebuilder/OpenWrt-ImageBuilder-15.05-rc3-realview.Linux-x86_64/bin/realview

ELFFILE="openwrt-15.05-rc3-realview-vmlinux-initramfs.elf"

ELFFILE="openwrt-realview-vmlinux-initramfs.elf"

qemu-system-arm -M realview-eb-mpcore -kernel $ELFFILE -net nic -net user -nographic

# ELFFILE="openwrt-15.05-rc3-realview-vmlinux.elf" will created error

# Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(0,0)

# Try to boot with ext4

qemu-system-arm -M realview-eb-mpcore -kernel $ELFFILE -net nic -net user -nographic -sd openwrt-realview-sdcard.img

# From ifconfig - vmnet1 - this one DOES NOT WORK

ssh -p 2222 quynh@192.168.52.1

# From ifconfig - vmnet8

ssh -p 2222 quynh@192.168.166.1

# This one allow to access the host file system

ssh quynh@192.168.166.1

# Copy file from host to QEMU

scp quynh@192.168.166.1:~/sed_4.2.1-1_realview.ipk ~/sed.ipk

        \end{lstlisting}

    \subsection{Install package on the running OpenWRT instance on QEMU}

        Read page 11 on \href{http://www.ccs.neu.edu/home/noubir/Courses/CS6710/S12/material/OpenWrt_Dev_Tutorial.pdf}{OpenWrt Dev Tutorial from NEU}

        \begin{lstlisting}

# Download the precompiled SDK

FILENAME="OpenWrt-SDK-15.05-realview_gcc-4.8-linaro_uClibc-0.9.33.2_eabi.Linux-x86_64"

wget "https://downloads.openwrt.org/chaos_calmer/15.05/realview/generic/$FILENAME.tar.bz2"

tar -xvjf $FILENAME.tar.bz2

rm $FILENAME.tar.bz2

cd $FILENAME

# Download package 'sed' and compile it

svn export svn://svn.openwrt.org/openwrt/packages/utils/sed package/sed

grep DEPENDS package/sed/Makefile

make package/sed/compile

# Transfer the compiled package 'sed' to running instance OpenWrt

# From the root@OpenWrt console

scp quynh@192.168.166.1:~/sed_4.2.1-1_realview.ipk ~/sed.ipk

# Install the ipk

opkg install ~/sed.ipk

        \end{lstlisting}

        \textbf{Problem: Only have 0kb available on filesystem /overlay, pkg sed needs 48}

            https://forum.openwrt.org/viewtopic.php?id=58545

            I tried to boot with \emph{image}, but the problem is still there

            \begin{lstlisting}

EFLFILE="openwrt-15.05-ar71xx-generic-vmlinux.elf"

qemu-system-arm -M realview-eb-mpcore -kernel $ELFFILE -net nic -net user -nographic -sd openwrt-realview-sdcard.img

            \end{lstlisting}

            Some reasons:

            \begin{itemize}

                \item However (as far as I understand), the image is mounted as an overlay image which (for what I could gather on the web) means it's readonly, and therefore, I can't install new packages.\footnote{https://forum.openwrt.org/viewtopic.php?id=58545}

                \item You are running a read only image, you need to add an overlay filesystem over that you that you can have full read and write functions. \footnote{https://forum.openwrt.org/viewtopic.php?id=56891}

                \item If I want to store something on my target I do this on some other mounted media (usually flash). Others might write the image into the devices flash and allow changes to be saved into flash. \footnote{http://buildroot.uclibc.narkive.com/pp4A78C6/creating-a-filesystem-for-real-time-linux-using-qemu}

            \end{itemize}

        \textbf{Qemu + overlay}

        \textbf{SD card image and Qemu}

            \href{https://en.wikibooks.org/wiki/QEMU/Images#Creating_an_image}{Creating an image - Wikibooks}

            \begin{lstlisting}

qemu-img create -f qcow2 sdcard.img 500M

qemu-system-arm -M realview-eb-mpcore -kernel openwrt-15.05-rc3-realview-vmlinux-initramfs.elf \

-nographic -m 1024M \

-sd sdcard.img \

--append "console=ttyAMA0 verbose debug root=/dev/mmcblk0p1"

            \end{lstlisting}

        \textbf{Test: Build an image with additional package}

            By this way, we do not need to install additional package with a running instance of OpenWrt

            Where is the Profiles.mk

            \begin{lstlisting}

# OpenWrt on Qemu with the default image for Realview Target

root@OpenWrt:/# fdisk

/bin/ash: fdisk: not found

# OpenWrt on Qemu with the image + fdisk package

# Inside the ImageBuilder folder

make image PACKAGES="fdisk"

            \end{lstlisting}

        \textbf{Generate openwrt-15.05-rc3-realview-vmlinux-initramfs.elf}

            The current Image Builder only generate openwrt-15.05-rc3-realview-vmlinux.elf, and not openwrt-15.05-rc3-realview-vmlinux-initramfs.elf

            Some similar forum threads:

            \begin{itemize}

                \item \footnote{https://forum.openwrt.org/viewtopic.php?id=53931}

            \end{itemize}

        \textbf{Recompile the OpenWrt toolchain to obtain openwrt-15.05-rc3-realview-vmlinux-initramfs.elf}

            By selecting the package that I want to include from $make menuconfig$, then that additional package will be available in the \texttt{initranfs.elf} file.

        \textbf{Add additional package to the compiled toolchain}

            Ways to do:

            \begin{itemize}

                \item Select the package/library from make menuconfig

                \item Use the $scripts/diffconfig.sh$ to generate \texttt{config.diff} file. Append the \texttt{config.diff} to the original \texttt{<build dir>/.config}. Then $make defconfig$ will generate the full \texttt{.config} file.

                \item Check out guideline on \href{http://wiki.openwrt.org/doc/howto/build}{Updating Feeds section}

            \end{itemize}

            Why does not the "Update Feeds" have \texttt{base} as in repo of \href{https://downloads.openwrt.org/chaos_calmer/15.05-rc3/realview/generic/packages/}{realview/generic/packages}

            Where is \texttt{feeds.conf}

            \begin{lstlisting}

quynh in ~/Thesis/cc-openwrt

$ ./scripts/feeds usage

$ ./scripts/feeds list -n

packages

luci

routing

telephony

management

quynh in ~/Thesis/cc-openwrt

$ ./scripts/feeds list -s

packages src-git  7f3ef93  https://github.com/openwrt/packages.git;for-15.05

luci     src-git  af9f093  https://github.com/openwrt/luci.git;for-15.05

routing  src-git  5876cd3  https://github.com/openwrt-routing/packages.git;for-15.05

telephony src-git  d4ca5e5  https://github.com/openwrt/telephony.git;for-15.05

management src-git  ab76d57  https://github.com/openwrt-management/packages.git;for-15.05

            \end{lstlisting}

        \textbf{Save the configuration information}

            \href{http://wiki.openwrt.org/doc/howto/build}{Read more in the official document - How to Build}

            \begin{lstlisting}

# In folder openwrt-cc

# After selecting all the changes in make menuconfig

# Export the diffconfig to file

./scripts/diffconfig.sh > diffconfig

            \end{lstlisting}

        \textbf{How to 'make' without re-compiling}

Prepare the C++ development for Linux Mint