Change font size
It is currently Wed Jan 23, 2019 1:58 am

Forum rules

Keep the training material related in some way to OSGrid and OpenSIM or tools that you would use externally to opensim such as image editors and database tools etc... but ultimately the material presented benefits OpenSIM and OSGrid.

Post a new topicPost a reply Page 1 of 1   [ 1 post ]
Author Message
 Post subject: Using a Raspberry pi-3 with no sd card for sims on osgrid
PostPosted: Tue Oct 04, 2016 10:44 pm 

Joined: Mon Feb 22, 2016 6:37 pm
Posts: 5
* The Newest Branch update is 4.9.5-v7+ as of 1/29/17

MovLab's and Sarah's Revision To Booting MoSa Swarm from a USB Stick, or Hard Drive, or USB Raid with out an sd card

How to boot from a USB Mass Storage Device on a Raspberry Pi 3 only.

This tutorial explains how to boot your Raspberry Pi 3 from a USB mass storage device such as a flash drive or USB hard disk. Be warned that this feature is experimental and may not work with all USB mass storage devices.
Program USB Boot Mode
Before a Pi will boot from usb, or disk , or network boot, (Without an sd card in it,) it needs to be booted with a config option to enable USB boot mode. Enabling this config option requires a special start.elf and bootcode.bin file. These can be installed by using the "next" branch on rpi-update.
Go to the Downloads page and install Raspbian Image or Hypriot Image (For Docker Swarm) onto an SD card using Win32DiskImager if you are on Windows, or dd if you are on Linux/Mac. Boot the Pi.
First, prepare the /boot directory with experimental boot files:
If on raspbian lite or Hypriot you need to install rpi-update before you can use it:
$ sudo apt-get update
$ sudo apt-get install rpi-update
NEW WAY!! USE THIS BELOW NOW!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
$ sudo apt-get update && sudo apt-get -y dist-upgrade && sudo BRANCH=next rpi-update
Then enable USB boot mode with:
$ echo program_usb_boot_mode=1 | sudo tee -a /boot/config.txt
This adds program_usb_boot_mode=1 to the end of /boot/config.txt. Reboot the Pi with sudo reboot, then check that the OTP has been programmed with:
$ sudo vcgencmd otp_dump | grep 17:
Ensure the output 0x3020000a is correct.
( ok to leave next line if you wish)
If you wish, you can remove the program_usb_boot_mode line from config.txt (make sure there is no blank line at the end) so that if you put the SD card in another Pi, it won't program USB boot mode. You can do this with sudo nano /boot/config.txt, for example.
Prepare the USB storage device
Now that your Pi 3 is USB boot-enabled, we can prepare a USB storage device to boot from. Start by inserting the USB storage device (which will be completely erased) into the Pi. Rather than downloading the Raspbian image again, we will copy it from the SD card on the Pi. The source device (sd card) will be /dev/mmcblk0 and the destination device (USB disk) should be /dev/sda assuming you have no other USB devices connected.
We will start by using parted to create a 100MB fat32 partition, followed by a Linux ext4 partition that will take up the rest of the disk.
Please note: Your PROMPT will be "parted' after next line until we quit the program parted.
If you need parted, use this $sudo apt-get install parted
$ sudo parted /dev/sda
(parted) (enter this) mktable msdos
Warning: The existing disk label on /dev/sda will be destroyed and all data on this disk will be lost. Do you want to continue?
Yes/No? (enter this) Yes
(parted) (enter this) mkpart primary fat32 0% 100M
(parted) (enter this) mkpart primary ext4 100M 100%
(parted) (enter this) print
Model: SanDisk Ultra (scsi)
Disk /dev/sda: 30.8GB
Sector size (logical/physical): 512B/512B
Partition Table: msdos
Disk Flags:
Number Start End Size Type File system Flags
1 1049kB 99.6MB 98.6MB primary fat32 lba
2 99.6MB 30.8GB 30.7GB primary ext4 lba
Your parted print output should look similar to the one above.
(if making a raid drive add set 1 raid on) (Explained later in the tutorial)
(parted) (enter this) quit
"Now if using Hypriot as a Filesystem,
$ sudo apt-get install dosfstools
otherwise skip this"
Create the boot and root filesystems:
$ sudo mkfs.vfat -n BOOT -F 32 /dev/sda1
$ sudo mkfs.ext4 /dev/sda2
Mount the target filesystems and copy the running raspbian system to it:
$ sudo mkdir /mnt/target
$ sudo mount /dev/sda2 /mnt/target/
$ sudo mkdir /mnt/target/boot
$ sudo mount /dev/sda1 /mnt/target/boot/
$ sudo apt-get install rsync (on Hypriot filesystem only)
$ sudo rsync -ax --progress / /boot /mnt/target now wait don't touch anything 15-20 min (less with a hard drive)
Regenerate ssh host keys:
$ cd /mnt/target
$ sudo mount --bind /dev dev
$ sudo mount --bind /sys sys
$ sudo mount --bind /proc proc
$ sudo chroot /mnt/target
# rm /etc/ssh/ssh_host*
# dpkg-reconfigure openssh-server
# exit
$ sudo umount dev
$ sudo umount sys
$ sudo umount proc
Edit /boot/cmdline.txt so that it uses the USB storage device as the root filesystem instead of the SD card.
$ sudo sed -i "s,root=/dev/mmcblk0p2,root=/dev/sda2," /mnt/target/boot/cmdline.txt
The same needs to be done for fstab
$ sudo sed -i "s,/dev/mmcblk0p,/dev/sda," /mnt/target/etc/fstab
We need to add a line to cmdline.txt for the drive or usb stick.
$ sudo nano /mnt/target/boot/cmdline.txt
ADD a line to the end of file rootdelay=5
The Line should look something like this
dwc_otg.lpm_enable=0 console=serial0,115200 console=tty1 root=/dev/sda2 rootfstype=ext4 elevator=deadline rootwait rootdelay=5
save it
We need to add a line to config.txt
$ sudo nano /mnt/target/boot/config.txt
add at end of file ADD
(blank line at end of file)
save it
Finally, unmount the target filesystems, and power off the Pi.
$ cd ~
$ sudo umount /mnt/target/boot
$ sudo umount /mnt/target
$ sudo poweroff
Disconnect the power supply from the Pi, remove the SD card and reconnect the power supply. If all has gone well the Pi should begin to boot after a few seconds.
Green light will blink all the time on the pi. (Mine did)
It is ok to do $ sudo apt-get update after this. It will not change the boot Files. You have a boot.back folder in your pi now with original boot code.
Sudo reboot works with out an sd card too...
Now your Raspberry Pi 3 Boots from a USB Stick, or Hard drive
From MovLab SeaRidge and My Partner Sarah Primrose.
Page 1
Above I used a 16G Cruser Fit Usb Stick
The boot drive was sda1 and the system was on sda2
Now we add 3 more 16G Cruser Fit Usb Sticks to the remaining slots
*note - ALL USB Devices need to be the same
You Should have 1 spare as you obtain them
Partition and make the file systems as you did above plus
add a flag as you partition the drives sdb2, sdc2, sdd2
To make the kernel notice the RAID partition and mount it automatically you need to set the raid flag on each Raid Drive
$ sudo parted /dev/sdb
(parted) (enter this) set 1 raid on
(parted) (enter this) quit
$ sudo parted /dev/sdc
(parted) (enter this) set 1 raid on
(parted) (enter this) quit
$ sudo parted /dev/sdd
(parted) (enter this) set 1 raid on
(parted) (enter this) quit
$ sudo mkfs.vfat -n BOOT -F 32 /dev/sdb1 (Backup Boot)
$ sudo mkfs.vfat -n BOOT -F 32 /dev/sdc1 (Backup Boot)
$ sudo mkfs.vfat -n BOOT -F 32 /dev/sdd1 (Backup Boot)
$ sudo mkfs.ext4 /dev/sdb2 (Raid Partition)
$ sudo mkfs.ext4 /dev/sdc2 (Raid Partition)
$ sudo mkfs.ext4 /dev/sdd2 (Raid Partition)
Do not copy the filesystems to these drives as you did above
Now we are ready to get software to make the Raid on the Raspberry Pi 3
pi@raspberrypi:~ $ sudo su
# apt-get install mdadm (answers=tab, ok, then enter, all, tab, ok, then enter again)
Now we will make the raid system (0,1,5, or 10) I used Raid 5 with parity
(*Note- You can use sda later to add it to your raid. Add it in "degraded mode"( Missing) to make the raid and then add Sda2.)
(That is beyond the scope of this Tutorial)
# mdadm --create --verbose /dev/md0 --level=5 --raid-devices=3 /dev/sdb2 /dev/sdc2 /dev/sdd2
if you have an error here, recheck your work and also check # df -h to make sure none of these drives are mounted
If all goes well, we will have a new device /dev/md0
Format RAID
# mkfs.ext4 /dev/md0
We can check it now with this
root@raspberrypi:/home/pi# mdadm --detail /dev/md0
You Should see this now
Rebuild Status : (x)% complete (then) ______________________________________________________________________________
Version : 1.2
Creation Time : Wed Sep 21 00:32:37 2016
Raid Level : raid5
Array Size : 31049728 (29.61 GiB 31.79 GB)
Used Dev Size : 15524864 (14.81 GiB 15.90 GB)
Raid Devices : 3
Total Devices : 3
Persistence : Superblock is persistent
Update Time : Fri Sep 23 05:29:13 2016
State : clean
Active Devices : 3
Working Devices : 3
Failed Devices : 0
Spare Devices : 0
Layout : left-symmetric
Chunk Size : 512K
Name : raspberrypi:0 (local to host raspberrypi)
UUID : 60eccaa3:612300f5:2ebb4ef1:e51752f2
Events : 229
Number Major Minor RaidDevice State
0 8 18 0 active sync /dev/sdb2
1 8 34 1 active sync /dev/sdc2
3 8 50 2 active sync /dev/sdd2
Congratulations! You have a working raid
From MovLab SeaRidge and My Partner Sarah Primrose
Page 2
/etc/mdadm.conf or /etc/mdadm/mdadm.conf (on debian) is the main configuration file for mdadm. After we create our RAID arrays we add them to this file using:
# mdadm --detail --scan >> /etc/mdadm/mdadm.conf for debian or
# mdadm --detail --scan >> /etc/mdadm.conf
It May be a good idea to do this command now
# dpkg-reconfigure mdadm
Respond with the following
Edit this file # nano /etc/default/mdadm
ADD AUTOSTART=true at the bottom of the file and save it
now check # nano /etc/mdadm/mdadm.conf
Make sure we don't have any duplicates at the bottom of the file. We should only have our new raid ARRAY... (1 Line)
It Should look something like this
# mdadm.conf
# Please refer to mdadm.conf(5) for information about this file.

# by default (built-in), scan all partitions (/proc/partitions) and all
# containers for MD superblocks. alternatively, specify devices to scan, using
# wildcards if desired.
#DEVICE partitions containers

# auto-create devices with Debian standard permissions
CREATE owner=root group=disk mode=0660 auto=yes

# automatically tag new arrays as belonging to the local system
HOMEHOST <system>

# instruct the monitoring daemon where to send mail alerts

# definitions of existing MD arrays

# This configuration was auto-generated on Mon, 19 Sep 2016 20:09:41 +0000 by mkconf

ARRAY /dev/md0 metadata=1.2 spares=1 name=raspberrypi:0 UUID=60eccaa3:612300f5:2ebb4ef1:e51752f2
# nano /etc/fstab
ADD /dev/md0 /mnt/target ext4 defaults,noatime 0 1
to the bottom of the file
# mount /dev/md0
we are going to look at df -h to see if our raid drive is here
pi@raspberrypi:~ $ df -h
similar to this
Filesystem Size Used Avail Use% Mounted on
udev 10M 0 10M 0% /dev (THIS IS NOT HERE YET)
tmpfs 186M 4.7M 181M 3% /run
/dev/md0 30G 3.9G 24G 14% / mnt/target <<<<<<<<<<<<<<< IT'S HERE
tmpfs 463M 0 463M 0% /dev/shm
tmpfs 5.0M 4.0K 5.0M 1% /run/lock
tmpfs 463M 0 463M 0% /sys/fs/cgroup
/dev/sda1 93M 26M 67M 28% /boot
tmpfs 93M 0 93M 0% /run/user/1000
we want to see /dev/md0 here
Congratulations if you have this :)
now reboot again and repeat df -h to check if /dev/md0 automounts
if not, double check your steps thus far
From MovLab SeaRidge and My Partner Sarah Primrose
Page 3
At this this point we now have a raid system , but we cannot boot from it. It is created after the filesystem is mounted and running . Thus the file system was mounted and the modules were loaded to allow creation of the raid. We need to fix the kernel to load the raid modules first. In the end the boot process will allow us to have md "(Creation of the raid system)" loaded first then the filesystem can be mounted after on the raid that we just created...
This is a partial copy of my boot process after the fix was implemented...
[ 19.246626] md: raid1 personality registered for level 1
[ 19.438602] raid6: int32x1 gen() 103 MB/s
[ 19.608642] raid6: int32x1 xor() 88 MB/s
[ 19.778818] raid6: int32x2 gen() 140 MB/s
[ 19.948471] raid6: int32x2 xor() 103 MB/s
[ 20.118608] raid6: int32x4 gen() 154 MB/s
[ 20.288425] raid6: int32x4 xor() 113 MB/s
[ 20.458508] raid6: int32x8 gen() 157 MB/s
[ 20.628600] raid6: int32x8 xor() 109 MB/s
[ 20.798478] raid6: neonx1 gen() 354 MB/s
[ 20.968427] raid6: neonx1 xor() 357 MB/s
[ 21.138492] raid6: neonx2 gen() 586 MB/s
[ 21.308430] raid6: neonx2 xor() 503 MB/s
[ 21.478432] raid6: neonx4 gen() 775 MB/s
[ 21.648420] raid6: neonx4 xor() 556 MB/s
[ 21.818499] raid6: neonx8 gen() 628 MB/s
[ 21.988473] raid6: neonx8 xor() 465 MB/s
[ 21.989738] raid6: using algorithm neonx4 gen() 775 MB/s
[ 21.991046] raid6: .... xor() 556 MB/s, rmw enabled
[ 21.992331] raid6: using intx1 recovery algorithm
[ 21.995612] xor: measuring software checksum speed
[ 22.088413] arm4regs : 1012.000 MB/sec
[ 22.188413] 8regs : 679.600 MB/sec
[ 22.288414] 32regs : 632.400 MB/sec
[ 22.388413] neon : 1079.200 MB/sec
[ 22.389556] xor: using function: neon (1079.200 MB/sec)
[ 22.391653] async_tx: api initialized (async)
[ 22.399163] md: raid6 personality registered for level 6
[ 22.400571] md: raid5 personality registered for level 5
[ 22.401923] md: raid4 personality registered for level 4
[ 22.466451] md: md0 stopped.
[ 22.475920] md: bind<sdc2>
[ 22.480168] md: bind<sdd2>
[ 22.483884] md: bind<sdb2>
[ 22.494138] md/raid:md0: device sdb2 operational as raid disk 0
[ 22.496037] md/raid:md0: device sdd2 operational as raid disk 2
[ 22.497818] md/raid:md0: device sdc2 operational as raid disk 1
[ 22.501725] md/raid:md0: allocated 3240kB
[ 22.503711] md/raid:md0: raid level 5 active with 3 out of 3 devices, algorithm 2
[ 22.507218] RAID conf printout:
[ 22.507228] --- level:5 rd:3 wd:3
[ 22.507239] disk 0, o:1, dev:sdb2
[ 22.507249] disk 1, o:1, dev:sdc2
[ 22.507260] disk 2, o:1, dev:sdd2
[ 22.507710] md0: detected capacity change from 0 to 31794921472
[ 22.811436] EXT4-fs (md0): mounted filesystem with ordered data mode. Opts: (null)
Notice the last 2 lines
md0 was created before the EXT4-fs (md0) filesystem
finally the filesystem can now be loaded...
Now we prepare the kernel
Setup the kernel
Configure initramfs
$ sudo su
Add the following lines to # nano /etc/initramfs-tools/modules
This is the contents of mine /etc/initramfs-tools/modules
# List of modules that you want to include in your initramfs.
# They will be loaded at boot time in the order below.
# Syntax: module_name [args ...]
# You must run update-initramfs(8) to effect this change.
# Examples:

Update the initramfs
Use the following command to figure out what kernel version you are running
# uname -r
# update-initramfs -c -k 4.4.17-v7+ (this is mine)
You may see some warnings about symlinks. You can safely ignore these since it is trying to create a symlink on a vfat filesystem which does not support them.

(Substitute your kernel version below)
Edit the boot configuration
Backup the original config.txt
# cp /boot/config.txt /boot/config.txt_orig
# nano /boot/config.txt

Add the following 2 lines to the end of /boot/config.txt:
initramfs initrd.img-4.4.17-v7+ followkernel
This is the contents of mine /boot/config.txt on sda1
# For more options and information see
# ...
# Some settings may impact device functionality. See link above for details

# uncomment if you get no picture on HDMI for a default "safe" mode

# uncomment this if your display has a black border of unused pixels visible
# and your display can output without overscan

# uncomment the following to adjust overscan. Use positive numbers if console
# goes off screen, and negative if there is too much border

# uncomment to force a console size. By default it will be display's size minus
# overscan.

# uncomment if hdmi display is not detected and composite is being output

# uncomment to force a specific HDMI mode (this will force VGA)

# uncomment to force a HDMI mode rather than DVI. This can make audio work in
# DMT (computer monitor) modes

# uncomment to increase signal to HDMI, if you have interference, blanking, or
# no display

# uncomment for composite PAL

#uncomment to overclock the arm. 700 MHz is the default.

# Uncomment some or all of these to enable the optional hardware interfaces

# Uncomment this to enable the lirc-rpi module

# Additional overlays and parameters are documented /boot/overlays/README

# Enable audio (loads snd_bcm2835)
initramfs initrd.img-4.4.17-v7+ followkernel
Reboot and test
# reboot
# uname -r
You should still see the above kernel version

From MovLab SeaRidge and My Partner Sarah Primrose
Page 4

Mount raid to temporary location
We will use the steps we used on Page 1

Mount the target filesystems and copy the running raspbian or Hypriot system to it:
$ sudo mkdir /mnt/target (Still Here)
$ sudo mount /dev/md0 /mnt/target/ (Still Mounted)
$ sudo mkdir /mnt/target/boot
$ sudo mount /dev/sda1 /mnt/target/boot/
(This boot copy will be more of a backup here since were booting from sda1)
$ sudo rsync -ax --progress / /boot /mnt/target now wait don't touch anything 15-20 min (less with a hard drive or usb stick)
(if you get an error here on the copy just redo it)

Make /boot/cmdline.txt look like the following (on sda1)
dwc_otg.lpm_enable=0 console=serial0,115200 console=tty1 root=/dev/md0 rootfstype=ext4 elevator=deadline boot_delay=32 rootdelay=10 rootwait

Make /etc/fstab look like the following (on md0) (/mnt/target/etc/fstab

proc /proc proc defaults 0 0
/dev/sda1 /boot vfat defaults 0 2
#/dev/sda2 / ext4 defaults,noatime 0 1
/dev/md0 / ext4 defaults,noatime 0 1
# a swapfile is not a swap partition, no line here
# use dphys-swapfile swap[on|off] for that


If all went well you are now starting from sda1 (boot) and md0 (filesystem on the raid)

Now the system will boot from the sda1 initramfs, load the raid modules and assemble then mount the raid. Then it will boot into the system running from the RAID.

Great care must be taken if you decide you want to use rpi-update to upgrade your kernel. After doing so, you must use initramfs-update -c -k xxxxxx+ to create a new initramfs for that kernel. Then you have to go into /boot/config.txt and update the last line to point to the new initramfs.
Remember were using
sudo BRANCH=next rpi-update
This is not yet introduced in the normal pi updates. It will be soon...
so don't update the kernel!!!!!!!!!!!! yet :)
this is the final df -h
pi@raspberrypi:/etc $ df -h
Filesystem Size Used Avail Use% Mounted on
udev 10M 0 10M 0% /dev
tmpfs 186M 4.6M 181M 3% /run
/dev/md0 30G 3.9G 24G 14% /
tmpfs 463M 0 463M 0% /dev/shm
tmpfs 5.0M 4.0K 5.0M 1% /run/lock
tmpfs 463M 0 463M 0% /sys/fs/cgroup
/dev/sda1 93M 26M 67M 28% /boot
tmpfs 93M 0 93M 0% /run/user/1000
pi@raspberrypi:/etc $
HypriotOS/armv7: pirate@node-3 in ~
If we are using the Hypriot image for Docker Swarm, we need to do
$ sudo cluster-lab stop
$ sudo apt-get install -y dphys-swapfile
$ sudo poweroff
$ sudo cluster-lab health
you should get this
Internet Connection
[PASS] eth0 exists
[PASS] eth0 has an ip address
[PASS] Internet is reachable
[PASS] DNS works

[PASS] eth0.200 exists
[PASS] eth0.200 has correct IP from vlan network
[PASS] Cluster leader is reachable
[PASS] eth0.200 has exactly one IP
[PASS] eth0.200 has no local link address
[PASS] Avahi process exists
[PASS] Avahi is using eth0.200

[PASS] Docker is running
[PASS] Docker is configured to use Consul as key-value store
[PASS] Docker is configured to listen via tcp at port 2375
[PASS] Docker listens on via tcp at port 2375 (Docker-Engine)

[PASS] Consul Docker image exists
[PASS] Consul Docker container is running
[PASS] Consul is listening on port 8300
[PASS] Consul is listening on port 8301
[PASS] Consul is listening on port 8302
[PASS] Consul is listening on port 8400
[PASS] Consul is listening on port 8500
[PASS] Consul is listening on port 8600
[PASS] Consul API works
[PASS] Cluster-Node is pingable with IP
[PASS] Cluster-Node is pingable with IP
[PASS] Cluster-Node is pingable with IP
[PASS] No Cluster-Node is in status 'failed'
[PASS] Consul is able to talk to Docker-Engine on port 7946 (Serf)

[PASS] Swarm-Join Docker container is running
[PASS] Swarm-Manage Docker container is running
[PASS] Number of Swarm and Consul nodes is equal which means our cluster is healthy

Credits go out to the community who diligently made this possible: ... erry-pi-3/
If you wish, You can use the first partitions on the raid for a backup of the boot sda1....
this will maintain the uuid also.
sudo dd bs=500M if=/dev/sda1 of=/dev/sdb1
sudo dd bs=500M if=/dev/sda1 of=/dev/sdc1
sudo dd bs=500M if=/dev/sda1 of=/dev/sdd1

From MovLab SeaRidge and My Partner Sarah Primrose
Page 5
Speed Results
sudo hdparm -Tt /dev/md0
Timing cached reads: 1302 MB in 2.00 seconds = 651.19 MB/sec
Timing buffered disk reads: 126 MB in 3.01 seconds = 41.90 MB/sec

to be continued
1) $ sudo apt-get install screen
2) $ screen sudo docker exec -i -t d26ee0381ee1 /bin/bash (example)

Display posts from previous:  Sort by  
Post a new topicPost a reply Page 1 of 1   [ 1 post ]

Who is online

Users browsing this forum: No registered users and 2 guests

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  

Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group
610nm Style by Daniel St. Jules of