Rocket and Chisel - nothing happening in the emulator for assembler tests

Question

I built the riscv-tools and rocket chip emulator. When I run run-asm-tests, the output is an empty file. Nothing seems to happen.

ceez: riscv/rocket-chip/emulator (master *)-> make output/rv64ui-p-add.out
mkdir -p ./output
ln -fs /media/arun/Academics/phd/Bremen/works/Learnings/riscv/rocket-chip/riscv-tools/riscv-tests/isa/rv64ui-p-add.hex output/rv64ui-p-add.hex
./emulator-DefaultCPPConfig +dramsim +max-cycles=100000000 +verbose +loadmem=output/rv64ui-p-add.hex none 3>&1 1>&2 2>&3 | /media/arun/Academics/phd/Bremen/works/Learnings/riscv/riscv/bin/spike-dasm  > output/rv64ui-p-add.out && [ $PIPESTATUS -eq 0 ]

>ceez: riscv/rocket-chip/emulator (master *)-> cat output/rv64ui-p-add.out 
Dramsim2 init successful
Starting store transaction (addr=0 ; tag=19 ; cyc=5831)
Adding store transaction (addr=0; cyc=5835)
[Callback] write complete: id=0 , addr=0x0 , cycle=5862

And if I run it manually and reduce the max-cycles this is what I am getting:

ceez: riscv/rocket-chip/emulator (master *)-> ./emulator-DefaultCPPConfig +dramsim +max-cycles=5000 +verbose +loadmem=output/rv64ui-p-add.hex
Dramsim2 init successful
*** FAILED *** (timeout, seed 1438106295) after 5000 cycles

Note ::

1) I have uncommented the macro #ifdef DEBUG_DRAMSIM2 in mm_dramsim2.cc

2) There are some warnings in the make flow for the emulator from the chisel compiler. Reproduced below, in case if its important.

cd /media/arun/Academics/phd/Bremen/works/Learnings/riscv/rocket-chip && java -Xmx2048M -Xss8M -XX:MaxPermSize=128M -jar sbt-launch.jar "project rocketchip" "run Top --W0W --noIoDebug --backend c --configInstance rocketchip.DefaultCPPConfig --compileInitializationUnoptimized --targetDir emulator/generated-src"
[0m[[0minfo[0m] [0mLoading project definition from /media/arun/Academics/phd/Bremen/works/Learnings/riscv/rocket-chip/project[0m
Using addons: 
[0m[[0minfo[0m] [0mSet current project to rocketchip (in build file:/media/arun/Academics/phd/Bremen/works/Learnings/riscv/rocket-chip/)[0m
[0m[[0minfo[0m] [0mSet current project to rocketchip (in build file:/media/arun/Academics/phd/Bremen/works/Learnings/riscv/rocket-chip/)[0m
[0m[[0minfo[0m] [0mRunning rocketchip.TestGenerator Top --W0W --noIoDebug --backend c --configInstance rocketchip.DefaultCPPConfig --compileInitializationUnoptimized --targetDir emulator/generated-src[0m
CPP elaborate
[[35minfo[0m] [6.096] // COMPILING < (class rocketchip.Top)>(2)
[[35minfo[0m] [6.356] giving names
[[35minfo[0m] [6.580] executing custom transforms
[[35minfo[0m] [6.581] adding clocks and resets
[[35minfo[0m] [6.709] inferring widths
[[35minfo[0m] [7.082] eliminating W0W
[[35minfo[0m] [7.275] checking widths
[[35minfo[0m] [7.377] lowering complex nodes to primitives
[[35minfo[0m] [7.378] removing type nodes
[[35minfo[0m] [7.485] compiling 39307 nodes
[[35minfo[0m] [7.485] computing memory ports
[[35minfo[0m] [7.546] resolving nodes to the components
[[35minfo[0m] [7.914] creating clock domains
[[35minfo[0m] [7.941] pruning unconnected IOs
[[35minfo[0m] [7.962] checking for combinational loops
[[35minfo[0m] [8.072] NO COMBINATIONAL LOOP FOUND
[[35minfo[0m] [8.369] populating clock domains
CppBackend::elaborate: need 501, redundant 425 shadow registers
[[35minfo[0m] [9.322] generating cpp files
CppBackend: createCppFile Top.DefaultCPPConfig-0.cpp
CppBackend: createCppFile Top.DefaultCPPConfig-1.cpp
CppBackend: createCppFile Top.DefaultCPPConfig-2.cpp
CppBackend: createCppFile Top.DefaultCPPConfig-3.cpp
[[33mwarn[0m] tilelink.scala:935: UNABLE TO FIND client_id IN <ManagerTileLinkNetworkPort (class uncore.ManagerTileLinkNetworkPort)> in class uncore.ManagerTileLinkNetworkPort
[[33mwarn[0m] tilelink.scala:937: UNABLE TO FIND client_id IN <ManagerTileLinkNetworkPort (class uncore.ManagerTileLinkNetworkPort)> in class uncore.ManagerTileLinkNetworkPort
[[33mwarn[0m] tilelink.scala:935: UNABLE TO FIND client_id IN <ManagerTileLinkNetworkPort (class uncore.ManagerTileLinkNetworkPort)> in class uncore.ManagerTileLinkNetworkPort
[[33mwarn[0m] tilelink.scala:937: UNABLE TO FIND client_id IN <ManagerTileLinkNetworkPort (class uncore.ManagerTileLinkNetworkPort)> in class uncore.ManagerTileLinkNetworkPort
[[33mwarn[0m] nbdcache.scala:376: UNABLE TO FIND data IN <replay_arb (class Chisel.Arbiter)> in class rocket.MSHRFile
[[33mwarn[0m] cache.scala:153: Mux of Bits instantiated, emits SInt in class uncore.MetadataArray
[[33mwarn[0m] nbdcache.scala:817: UNABLE TO FIND tag IN <metaReadArb (class Chisel.Arbiter)> in class rocket.HellaCache
[[33mwarn[0m] nbdcache.scala:839: UNABLE TO FIND tag IN <metaReadArb (class Chisel.Arbiter)> in class rocket.HellaCache
[[33mwarn[0m] dpath.scala:171: UNABLE TO FIND custom_mrw_csrs IN <dpath (class rocket.Datapath)> in class rocket.Datapath
[[33mwarn[0m] dpath.scala:171: UNABLE TO FIND evec IN <dpath (class rocket.Datapath)> in class rocket.Datapath
[[33mwarn[0m] dpath.scala:171: UNABLE TO FIND fatc IN <dpath (class rocket.Datapath)> in class rocket.Datapath
[[33mwarn[0m] dpath.scala:171: UNABLE TO FIND fcsr_flags IN <dpath (class rocket.Datapath)> in class rocket.Datapath
[[33mwarn[0m] dpath.scala:171: UNABLE TO FIND fcsr_rm IN <dpath (class rocket.Datapath)> in class rocket.Datapath
[[33mwarn[0m] dpath.scala:171: UNABLE TO FIND host IN <dpath (class rocket.Datapath)> in class rocket.Datapath
[[33mwarn[0m] dpath.scala:171: UNABLE TO FIND pc IN <dpath (class rocket.Datapath)> in class rocket.Datapath
[[33mwarn[0m] dpath.scala:171: UNABLE TO FIND ptbr IN <dpath (class rocket.Datapath)> in class rocket.Datapath
[[33mwarn[0m] dpath.scala:171: UNABLE TO FIND rocc IN <dpath (class rocket.Datapath)> in class rocket.Datapath
[[33mwarn[0m] dpath.scala:171: UNABLE TO FIND rw IN <dpath (class rocket.Datapath)> in class rocket.Datapath
[[33mwarn[0m] dpath.scala:171: UNABLE TO FIND time IN <dpath (class rocket.Datapath)> in class rocket.Datapath
[[33mwarn[0m] dpath.scala:171: UNABLE TO FIND uarch_counters IN <dpath (class rocket.Datapath)> in class rocket.Datapath
[[33mwarn[0m] dpath.scala:172: UNABLE TO FIND cause IN <dpath (class rocket.Datapath)> in class rocket.Datapath
[[33mwarn[0m] dpath.scala:172: UNABLE TO FIND csr_replay IN <dpath (class rocket.Datapath)> in class rocket.Datapath
[[33mwarn[0m] dpath.scala:172: UNABLE TO FIND csr_stall IN <dpath (class rocket.Datapath)> in class rocket.Datapath
[[33mwarn[0m] dpath.scala:172: UNABLE TO FIND csr_xcpt IN <dpath (class rocket.Datapath)> in class rocket.Datapath
[[33mwarn[0m] dpath.scala:172: UNABLE TO FIND custom_mrw_csrs IN <dpath (class rocket.Datapath)> in class rocket.Datapath
[[33mwarn[0m] dpath.scala:172: UNABLE TO FIND eret IN <dpath (class rocket.Datapath)> in class rocket.Datapath
[[33mwarn[0m] dpath.scala:172: UNABLE TO FIND evec IN <dpath (class rocket.Datapath)> in class rocket.Datapath
[[33mwarn[0m] dpath.scala:172: UNABLE TO FIND exception IN <dpath (class rocket.Datapath)> in class rocket.Datapath
[[33mwarn[0m] dpath.scala:172: UNABLE TO FIND fatc IN <dpath (class rocket.Datapath)> in class rocket.Datapath
[[33mwarn[0m] dpath.scala:172: UNABLE TO FIND host IN <dpath (class rocket.Datapath)> in class rocket.Datapath
[[33mwarn[0m] dpath.scala:172: UNABLE TO FIND interrupt IN <dpath (class rocket.Datapath)> in class rocket.Datapath
[[33mwarn[0m] dpath.scala:172: UNABLE TO FIND interrupt_cause IN <dpath (class rocket.Datapath)> in class rocket.Datapath
[[33mwarn[0m] dpath.scala:172: UNABLE TO FIND pc IN <dpath (class rocket.Datapath)> in class rocket.Datapath
[[33mwarn[0m] dpath.scala:172: UNABLE TO FIND ptbr IN <dpath (class rocket.Datapath)> in class rocket.Datapath
[[33mwarn[0m] dpath.scala:172: UNABLE TO FIND retire IN <dpath (class rocket.Datapath)> in class rocket.Datapath
[[33mwarn[0m] dpath.scala:172: UNABLE TO FIND rocc IN <dpath (class rocket.Datapath)> in class rocket.Datapath
[[33mwarn[0m] dpath.scala:172: UNABLE TO FIND rw IN <dpath (class rocket.Datapath)> in class rocket.Datapath
[[33mwarn[0m] dpath.scala:172: UNABLE TO FIND status IN <dpath (class rocket.Datapath)> in class rocket.Datapath
[[33mwarn[0m] dpath.scala:172: UNABLE TO FIND time IN <dpath (class rocket.Datapath)> in class rocket.Datapath
[[33mwarn[0m] dpath.scala:172: UNABLE TO FIND uarch_counters IN <dpath (class rocket.Datapath)> in class rocket.Datapath
[[33mwarn[0m] dpath.scala:232: Mux of Bits instantiated, emits SInt in class rocket.Datapath
[[33mwarn[0m] dpath.scala:236: Mux of Bits instantiated, emits SInt in class rocket.Datapath
[[33mwarn[0m] Testing.scala:124: Width.op- setting width to Width(0): 1 < 10 in class rocketchip.TestGenerator$delayedInit$body
[0m[[32msuccess[0m] [0mTotal time: 13 s, completed Jul 28, 2015 4:34:00 PM[0m
g++ -O1 -std=c++11 -I/media/arun/Academics/phd/Bremen/works/Learnings/riscv/rocket-chip/toolchain/include -I/media/arun/Academics/phd/Bremen/works/Learnings/riscv/rocket-chip/csrc -I/media/arun/Academics/phd/Bremen/works/Learnings/riscv/rocket-chip/dramsim2 -include generated-src/Top.DefaultCPPConfig.h -Igenerated-src -c -o emulator.o /media/arun/Academics/phd/Bremen/works/Learnings/riscv/rocket-chip/csrc/emulator.cc
g++ -O1 -std=c++11 -I/media/arun/Academics/phd/Bremen/works/Learnings/riscv/rocket-chip/toolchain/include -I/media/arun/Academics/phd/Bremen/works/Learnings/riscv/rocket-chip/csrc -I/media/arun/Academics/phd/Bremen/works/Learnings/riscv/rocket-chip/dramsim2 -include generated-src/Top.DefaultCPPConfig.h -Igenerated-src -c -o mm.o /media/arun/Academics/phd/Bremen/works/Learnings/riscv/rocket-chip/csrc/mm.cc
g++ -O1 -std=c++11 -I/media/arun/Academics/phd/Bremen/works/Learnings/riscv/rocket-chip/toolchain/include -I/media/arun/Academics/phd/Bremen/works/Learnings/riscv/rocket-chip/csrc -I/media/arun/Academics/phd/Bremen/works/Learnings/riscv/rocket-chip/dramsim2 -include generated-src/Top.DefaultCPPConfig.h -Igenerated-src -c -o mm_dramsim2.o /media/arun/Academics/phd/Bremen/works/Learnings/riscv/rocket-chip/csrc/mm_dramsim2.cc
make -j generated-src/Top.DefaultCPPConfig-0.o generated-src/Top.DefaultCPPConfig-1.o generated-src/Top.DefaultCPPConfig-2.o generated-src/Top.DefaultCPPConfig-3.o
make[1]: Entering directory '/media/arun/Academics/phd/Bremen/works/Learnings/riscv/rocket-chip/emulator'
g++ -O1 -std=c++11 -I/media/arun/Academics/phd/Bremen/works/Learnings/riscv/rocket-chip/toolchain/include -I/media/arun/Academics/phd/Bremen/works/Learnings/riscv/rocket-chip/csrc -I/media/arun/Academics/phd/Bremen/works/Learnings/riscv/rocket-chip/dramsim2 -Igenerated-src -c -o generated-src/Top.DefaultCPPConfig-0.o generated-src/Top.DefaultCPPConfig-0.cpp
g++ -O1 -std=c++11 -I/media/arun/Academics/phd/Bremen/works/Learnings/riscv/rocket-chip/toolchain/include -I/media/arun/Academics/phd/Bremen/works/Learnings/riscv/rocket-chip/csrc -I/media/arun/Academics/phd/Bremen/works/Learnings/riscv/rocket-chip/dramsim2 -Igenerated-src -c -o generated-src/Top.DefaultCPPConfig-1.o generated-src/Top.DefaultCPPConfig-1.cpp
g++ -O1 -std=c++11 -I/media/arun/Academics/phd/Bremen/works/Learnings/riscv/rocket-chip/toolchain/include -I/media/arun/Academics/phd/Bremen/works/Learnings/riscv/rocket-chip/csrc -I/media/arun/Academics/phd/Bremen/works/Learnings/riscv/rocket-chip/dramsim2 -Igenerated-src -c -o generated-src/Top.DefaultCPPConfig-2.o generated-src/Top.DefaultCPPConfig-2.cpp
g++ -O1 -std=c++11 -I/media/arun/Academics/phd/Bremen/works/Learnings/riscv/rocket-chip/toolchain/include -I/media/arun/Academics/phd/Bremen/works/Learnings/riscv/rocket-chip/csrc -I/media/arun/Academics/phd/Bremen/works/Learnings/riscv/rocket-chip/dramsim2 -Igenerated-src -c -o generated-src/Top.DefaultCPPConfig-3.o generated-src/Top.DefaultCPPConfig-3.cpp
make[1]: Leaving directory '/media/arun/Academics/phd/Bremen/works/Learnings/riscv/rocket-chip/emulator'
ld -r generated-src/Top.DefaultCPPConfig-0.o generated-src/Top.DefaultCPPConfig-1.o generated-src/Top.DefaultCPPConfig-2.o generated-src/Top.DefaultCPPConfig-3.o -o Top.DefaultCPPConfig.o
ar rcs libdramsim.a /media/arun/Academics/phd/Bremen/works/Learnings/riscv/rocket-chip/dramsim2/AddressMapping.o /media/arun/Academics/phd/Bremen/works/Learnings/riscv/rocket-chip/dramsim2/Transaction.o /media/arun/Academics/phd/Bremen/works/Learnings/riscv/rocket-chip/dramsim2/BusPacket.o /media/arun/Academics/phd/Bremen/works/Learnings/riscv/rocket-chip/dramsim2/BankState.o /media/arun/Academics/phd/Bremen/works/Learnings/riscv/rocket-chip/dramsim2/MemorySystem.o /media/arun/Academics/phd/Bremen/works/Learnings/riscv/rocket-chip/dramsim2/SimulatorObject.o /media/arun/Academics/phd/Bremen/works/Learnings/riscv/rocket-chip/dramsim2/TraceBasedSim.o /media/arun/Academics/phd/Bremen/works/Learnings/riscv/rocket-chip/dramsim2/Bank.o /media/arun/Academics/phd/Bremen/works/Learnings/riscv/rocket-chip/dramsim2/CommandQueue.o /media/arun/Academics/phd/Bremen/works/Learnings/riscv/rocket-chip/dramsim2/Rank.o /media/arun/Academics/phd/Bremen/works/Learnings/riscv/rocket-chip/dramsim2/MemoryController.o /media/arun/Academics/phd/Bremen/works/Learnings/riscv/rocket-chip/dramsim2/MultiChannelMemorySystem.o /media/arun/Academics/phd/Bremen/works/Learnings/riscv/rocket-chip/dramsim2/IniReader.o
g++ -O1 -std=c++11 -I/media/arun/Academics/phd/Bremen/works/Learnings/riscv/rocket-chip/toolchain/include -I/media/arun/Academics/phd/Bremen/works/Learnings/riscv/rocket-chip/csrc -I/media/arun/Academics/phd/Bremen/works/Learnings/riscv/rocket-chip/dramsim2 -o emulator-DefaultCPPConfig emulator.o mm.o mm_dramsim2.o Top.DefaultCPPConfig.o  -L/media/arun/Academics/phd/Bremen/works/Learnings/riscv/rocket-chip/toolchain/lib -Wl,-rpath,/media/arun/Academics/phd/Bremen/works/Learnings/riscv/rocket-chip/toolchain/lib -L. -ldramsim -lfesvr -lpthread

3) This is my first post in stack-overflow, besides i am very new to chisel/riscv. Please excuse the obvious blunders if any!

Answer 1

"I built the riscv-tools and rocket chip emulator."

The RISC-V tools are still under active development, so you must be very careful that you are compiling the proper version for the Rocket-chip that you checked out. The proper version of riscv-tools is pointed to as a git submodule by the rocket-chip super repo.

cd rocket-chip/riscv-tools; ./build.sh

That should build the proper version of RISC-V tools that is used by Rocket.

---

If you would like to better debug your Rocket processor, you should analyze the generated output files: (rocket-chip/emulator/output/rv64ui-p-add.out). What I assume happened, is your add test was compiled with an incorrect privileged ISA version, which caused an illegal instruction exception to occur. However, because there is no EVEC (exception vector) set by the simple assembly tests, the processor jumps to a random location in code to handle the exception and infinite loops on the illegal instruction that resides at the EVEC location.

Answer 2

@Chris - Thanks for the suggestions.

I could build and run the emulator in Fedora/Bash combination.
I use ArchLinux/Zsh usually and the original error was in this combination.

Following could be what would have happened in Arch:
1) Checked out the rocket-chip and updated the git submodules as given in the tutorial itself and tried to build the riscv-tools first. However there was an error in the make flow at the QEMU stage. (Reproduced below)

...
Makefile:1406: warning: overriding recipe for target 'lib_a-mbtowc_r.o'
Makefile:761: warning: ignoring old recipe for target 'lib_a-mbtowc_r.o'
Makefile:1608: warning: overriding recipe for target 'lib_a-vfwscanf.o'
Makefile:1348: warning: ignoring old recipe for target 'lib_a-vfwscanf.o'
Makefile:1608: warning: overriding recipe for target 'lib_a-vfwscanf.o'
Makefile:1348: warning: ignoring old recipe for target 'lib_a-vfwscanf.o'
install-info: /media/arun/Academics/phd/Bremen/works/Learnings/arch-rocket-chip/rocket-chip/riscv/share/info/dir: empty file
gmake[3]: [/media/arun/Academics/phd/Bremen/works/Learnings/arch-rocket-chip/rocket-chip/riscv/share/info/gcc.info] Error 1 (ignored)
install-info: /media/arun/Academics/phd/Bremen/works/Learnings/arch-rocket-chip/rocket-chip/riscv/share/info/dir: empty file
gmake[3]: [/media/arun/Academics/phd/Bremen/works/Learnings/arch-rocket-chip/rocket-chip/riscv/share/info/cppinternals.info] Error 1 (ignored)
Makefile:1406: warning: overriding recipe for target 'lib_a-mbtowc_r.o'
Makefile:761: warning: ignoring old recipe for target 'lib_a-mbtowc_r.o'
Makefile:1406: warning: overriding recipe for target 'lib_a-mbtowc_r.o'
Makefile:761: warning: ignoring old recipe for target 'lib_a-mbtowc_r.o'
Makefile:1608: warning: overriding recipe for target 'lib_a-vfwscanf.o'
Makefile:1348: warning: ignoring old recipe for target 'lib_a-vfwscanf.o'
Makefile:1608: warning: overriding recipe for target 'lib_a-vfwscanf.o'
Makefile:1348: warning: ignoring old recipe for target 'lib_a-vfwscanf.o'
Makefile:1406: warning: overriding recipe for target 'lib_a-mbtowc_r.o'
Makefile:761: warning: ignoring old recipe for target 'lib_a-mbtowc_r.o'
Makefile:1406: warning: overriding recipe for target 'lib_a-mbtowc_r.o'
Makefile:761: warning: ignoring old recipe for target 'lib_a-mbtowc_r.o'
Makefile:1608: warning: overriding recipe for target 'lib_a-vfwscanf.o'
Makefile:1348: warning: ignoring old recipe for target 'lib_a-vfwscanf.o'
Makefile:1608: warning: overriding recipe for target 'lib_a-vfwscanf.o'
Makefile:1348: warning: ignoring old recipe for target 'lib_a-vfwscanf.o'
Installing project riscv-gnu-toolchain

Configuring project riscv-pk
configure: WARNING: using cross tools not prefixed with host triplet
Building project riscv-pk
Installing project riscv-pk
mkdir //media/arun/Academics/phd/Bremen/works/Learnings/arch-rocket-chip/rocket-chip/riscv/riscv64-unknown-elf/lib/riscv-pk
mkdir //media/arun/Academics/phd/Bremen/works/Learnings/arch-rocket-chip/rocket-chip/riscv/riscv64-unknown-elf/include/riscv-pk

Configuring project riscv-qemu
ceez: arch-rocket-chip/rocket-chip/riscv-tools ((ce9bb22...))-> 
ceez: arch-rocket-chip/rocket-chip/riscv-tools ((ce9bb22...))->      

But when I checked the $RISCV/bin directory, the needed binutils, spike etc were there.
Only missing stuff was the build-tests.
2) Then I checked out a fresh standalone version of the riscv-tools and started again. There also the error was reported in the make flow.
3) At this point, I started to debug further and built some of the hex files manually in the riscv-tests directory (using this standalone version of the riscv-tools).
4) And used these hex files for the rocket-chip emulator run. The result is an empty file for "rocket-chip/emulator/output/rv64ui-p-add.out".

It could be a mismatch between the riscv-tools and the checkedout rocket-chip version, as you mentioned.