Here is a short tutorial on how to get started with dist-gem5 and run a simple simulation using aarch64 ISA.
We assume that the users have familiarity with gem5 full-system simulation.
If you use dist-gem5 in your research, please refer to Publication tab for citation information.
You can ask questions in gem5 mailing list if you faced any problem.
Videos: you may watch these screen captures to get started with dist-gem5
Step0- Download and build gem5; Download ARM Full-System stuff
http://www.gem5.org/dist/current/arm/aarch-system-20170421.tar.xz
We assume your gem5 is under $DIST_M5 and you put ARM full-system stuff under “$DIST_M5/disks” and “$DIST_M5/binaries”
Step1- Setup password less SSH
Here we provide simple steps to setup passless ssh for “localhost”. You can lookup the web for more information.
ssh-keygen -t rsa -P "" (press enter for any prompts) ssh-copy-id localhost ssh localhost (this is for testing if it works or not)
Step3- Disable All Listeners
To prevent some transient errors during dist-gem5 simulations, please add the following to the “build_test_system” function of your “fs.py” script:
m5.disableAllListeners()
step4- Run dist-gem5 and take a checkpoint
Following command will start a 4 node distributed simulation on localhost.
If you want to launch simulation on multiple hosts, you need to setup passless ssh for them and then allocate cores to run gem5 processes using “LSB_MCPU_HOSTS” variable. e.g:
LSB_MCPU_HOSTS="hname1 3 hname2 2"
means we have altogether 5 slots allocated to launch the gem5 processes, 3 of them are on host hname1 and 2 of them are on host hname2.
export M5_PATH=$DIST_M5 bash $DIST_M5/util/dist/gem5-dist.sh \ -n 4 \ -r $DIST_M5/rundir \ -c $DIST_M5/ckptdir \ -x $DIST_M5/build/ARM/gem5.opt \ -s $DIST_M5/configs/dist/sw.py \ -f $DIST_M5/configs/example/fs.py \ --fs-args \ --disk-image=$DIST_M5/disks/aarch32-ubuntu-natty-headless.img \ --kernel=$DIST_M5/binaries/vmlinux.aarch32.ll_20131205.0-gem5 \ --dtb-file=$DIST_M5/binaries/vexpress.aarch32.ll_20131205.0-gem5.2cpu.dtb \ --num-cpus=2 \ --machine-type=VExpress_EMM \ --script=$DIST_M5/util/dist/test/simple_bootscript.rcS \ --cf-args \ --dist-sync-start=1000000000000t
For more information regarding the options, please type:
bash $DIST_M5/util/dist/gem5-dist.sh -h
Step5- restore from the checkpoint
Following command will restore from the checkpoint that we took in step-5 and run a detailed simulation:
export M5_PATH=$DIST_M5 bash $DIST_M5/util/dist/gem5-dist.sh \ -n 4 \ -r $DIST_M5/rundir \ -c DIST_M5/ckptdir \ -x $DIST_M5/build/ARM/gem5.opt \ -s $DIST_M5/configs/dist/sw.py \ -f $DIST_M5/configs/example/fs.py \ --fs-args \ --disk-image=$DIST_M5/disks/aarch32-ubuntu-natty-headless.img \ --kernel=$DIST_M5/binaries/vmlinux.aarch32.ll_20131205.0-gem5 \ --dtb-file=$DIST_M5/binaries/vexpress.aarch32.ll_20131205.0-gem5.2cpu.dtb \ --num-cpus=2 \ --cpu-type=arm_detailed \ --machine-type=VExpress_EMM \ --cf-args \ --dist-sync-start=1000000000000t \ --checkpoint-restore=1
Tips for debugging:
When you run a dist-gem5 simulation, it will generate one log file for each simulated node (log.0 to log.n-1) and one log file for the simulated Ethernet switch (log.switch).
The first step for debugging a problematic simulation is looking into these log files. dist-gem5 also enables users to use trace based debugging and/or gdb to dig further into what gem5 is doing under the hood.
You can enable debug flags for your dist-gem5 simulation by passing “–debug-flags” option after the “–m5-args” command line argument. E.g.:
--m5-args --debug-flags=DistEthernet,Ethernet
To debug dist-gem5 with gdb, you need to pass “-debug” command line argument and dist-gem5 will start it’s gem5 processes in gdb terminals.
