How to stop a simulation by timeout?

Question

I have manageg to implement a simulation timeout in VHDL. If processes are running longer the MaxRuntime they get 'killed'.

Unfortunately, this does not work the other way around. If my simulation is finished before MaxRuntime, everything is waiting for the last wait statement on MaxRuntime.

I found that it's possible to combine wait on, wait for and wait until statements into one.

My current code in snippets. A full example is quite to long...

package sim is
  shared variable IsFinalized : BOOLEAN := FALSE;

  procedure initialize(MaxRuntime : TIME := TIME'high);
  procedure finalize;
end package;

package body sim is
  procedure initialize(MaxRuntime : TIME := TIME'high) is
  begin
    -- do init stuff
    if (MaxRuntime = TIME'high) then
      wait on IsFinalized for MaxRuntime;
      finalize;
    end if;
  end procedure;

  procedure finalize;
  begin
    if (not IsFinalized) then
      IsFinalized := TRUE;
      -- do finalize stuff:
      --  -> stop all clocks
      -- write a report
    end if;
  end procedure;
end package body;

entity test is
end entity;

architecture tb of test is
begin
  initialize(200 us);

  process
  begin
    -- simulate work
    wait for 160 us;
    finalize;
  end process;
end architecture;

The wait statement is not exited if IsFinalized changed. My simulation runs for circa 160 us. If I set MaxRuntime to 50 us, the simulation stops at circa 50 us (plus some extra cycles until each process noticed the stop condition). When I set MaxRuntime to 200 us, the simulation exits at 200 us and not after 162 us.

• How can I exit/abort the wait statement?
• Why can't I wait on a variable?

I don't want to use a command line switch for a simulator to set the max execution time.

Answer 1

So it sounds like you want to terminate the simulation, either at time out, or at finished test prior to timeout. It is correct that the simulator will stop when the event queue is empty, but that is pretty high do achieve, as you also experience.

VHDL-2008 has introduced stop and finish in the std.env package for stop or termination of simulation.

In VHDL-2002, a common way is to stop simulation is by an assert with severity FAILURE:

report "OK   ### Sim end: OK :-) ###   (not actual failure)" severity FAILURE;

This method for simulation stop is based on the fact that simulators (e.g. ModelSim) will usually stop simulation when an assert with severity FAILURE occurs.

Answer 2

You cannot wait on a variable for the reasons given by user1155120. So, instead you need to use a signal. (A signal in a package is a global signal).

Unfortunately, even though the global signal is in scope, it still needs to be an output parameter of the procedure, which is ugly. Not only that, in your code, you will then be driving the signal from more than one place, this global signal needs to be a resolved type, eg std_logic. Which is also a bit ugly.

Here is a version of your code with the shared variable replaced by a signal, the boolean type replaced by a std_logic and the global signal added as output parameters:

library IEEE;
use IEEE.std_logic_1164.all;

package sim is
  signal IsFinalized : std_logic := '0';

  procedure initialize(signal f : out std_logic; MaxRuntime : TIME := TIME'high);
  procedure finalize (signal f : out std_logic);
end package;

package body sim is
  procedure initialize(signal f : out std_logic; MaxRuntime : TIME := TIME'high) is
  begin
    -- do init stuff
    if (MaxRuntime = TIME'high) then
      wait on IsFinalized for MaxRuntime;
      finalize(f);
    end if;
  end procedure;

  procedure finalize (signal f : out std_logic) is
  begin
    if (IsFinalized = '0') then
      f <= '1';
      -- do finalize stuff:
      --  -> stop all clocks
      -- write a report
      report "Finished!";
    end if;
  end procedure;
end package body;

use work.sim.all;

entity test is
end entity;

architecture tb of test is
begin
  initialize(IsFinalized, 200 us);

  process
  begin
    -- simulate work
    wait for 160 us;
    finalize(IsFinalized);
    wait;
  end process;
end architecture;

http://www.edaplayground.com/x/VBK

Answer 3

• How can I exit/abort the wait statement?
• Why can't I wait on a variable?

All sequential statements in VHDL are atomic.

A wait statement happens to wait until a resumption condition is met, waiting on signal events, signal expressions or simulation time.

What would you use to select the statement a process resumed at if you could exit/abort it? Would it resume? You'd invalidate the execution of the sequence of statements. VHDL doesn't do exception handling, it's event driven.

And here it's worth remembering that everything that executes in simulation is a process, function calls are expressions and concurrent statements (including procedure calls) are devolved into processes potentially with super-imposed block statements limiting scope, during elaboration.

There's this basic difference between variables and signals (IEEE Std 1076-2008 Appendix I Glossary):

signal: An object with a past history of values. A signal may have multiple drivers, each with a current value and projected future values. The term signal refers to objects declared by signal declarations or port declarations. (6.4.2.3)

variable: An object with a single current value. (6.4.2.4)

You can't wait on something that doesn't have a future value, or a little more simply variables don't signal (from a dictionary - an event or statement that provides the impulse or occasion for something specified to happen).

Simulation is driven by signal events depending on future values. It defines the progression of simulation time. Time is the basis for discrete event simulation.

And about now you could wonder if someone would be telling you the truth if they were to claim VHDL is a general purpose programming language. How can it be while maintaining the ability formally specify the operation of hardware by discrete time events if you have the ability to break and resume a process arbitrarily?

And all this tells you is you might consider using signals instead of share variables.

Morton's stop and finish procedures are found in std.env.

Answer 4

From a test automation point of view it's important to have a time-out that can force a test to a stop using stop/finish/assert failure. Even if your intention is to make everything terminate by not producing more events there is a risk that there is a bug causing everything to hang. If that happens in the first of your tests in a longer nightly test run you're wasting a lot of time.

If you use VUnit it would work like this

library vunit_lib;
context vunit_lib.vunit_context;

entity tb_test is
  generic (runner_cfg : runner_cfg_t);
end tb_test;

architecture tb of tb_test is
begin
    test_runner : process
    begin
      test_runner_setup(runner, runner_cfg);
      while test_suite loop
        if run("Test that hangs") then
          -- Simulate hanging test case
          wait;
        elsif run("Test that goes well") then
          -- Simulate test case running for 160 us
          wait for 160 us;
        end if;
      end loop;

      test_runner_cleanup(runner); -- Normal forced exit point
    end process test_runner;

    test_runner_watchdog(runner, 200 us);
end;

The first test case which hangs is terminated by the watchdog after 200 us such that the second can run.

A problem that you may encounter when forcing the testbench to stop with the test_runner_cleanup procedure is if you have one or more additional test processes that have more things to do when the test_runner process reaches the test_runner_cleanup procedure call. For example, if you have a process like this

another_test_process: process is
begin
  for i in 1 to 4 loop
    wait for 45 us;
    report "Testing something";
  end loop;
  wait;
end process;

it will not be allowed to run all four iterations. The last iteration is supposed to run at 180 us but test_runner_cleanup is called at 160 us.

So there is a need to synchronize the test_runner and another_test_process processes. You can create a package with a global resolved signal to fix this (as discussed before) but VUnit already provide such a signal which you can use to save some time. runner is a record containing, among other things, the current phase of the VUnit testbench. When calling test_runner_cleanup it enters the phase with the same name and then it moves to the test_runner_exit phase in which the simulation is forced to a stop. Any process can prevent VUnit from entering or exiting its phases by placing a temporary lock. The updated another_test_process will prevent VUnit from exiting the test_runner_cleanup phase until it's done with all its iterations.

another_test_process: process is
begin
  lock_exit(runner, test_runner_cleanup);
  for i in 1 to 4 loop
    wait for 45 us;
    report "Testing something";
  end loop;
  unlock_exit(runner, test_runner_cleanup);
  wait;
end process;

Any number of processes can place a lock on a phase and the phase transition is prevented until all locks have been unlocked. You can also get the current phase with the get_phase function which may be useful in some cases.

If you're developing reusable components you should probably not use this technique. It saves you some code but it also makes your components dependent on VUnit and not everyone uses VUnit. Working on that :-)

DISCLAIMER: I'm one of the authors for VUnit.