Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add waveform replay tool #1523

Closed
veripoolbot opened this issue Sep 24, 2019 · 9 comments
Closed

Add waveform replay tool #1523

veripoolbot opened this issue Sep 24, 2019 · 9 comments
Assignees
Labels
area: tests Issue involves the testing system effort: weeks Expect this issue to require weeks or more of invested effort to resolve resolution: abandoned Closed; not enough information or otherwise never finished type: feature-non-IEEE Request to add new feature, outside IEEE 1800

Comments

@veripoolbot
Copy link
Contributor


Author Name: Todd Strader (@toddstrader)
Original Redmine Issue: 1523 from https://www.veripool.org

Original Assignee: Todd Strader (@toddstrader)


When simulating with a --dpi-protect'ed module, the end user cannot see the internals and therefore a waveform capture of an issue may not be immediately useful to the IP vendor. We could create a tool which would extract the inputs from the waveform and play them back to a Verilated instance of the design. The IP vendor would then get the customer's waveform capture of the issue and then be able to create a waveform of what is happening inside of the --dpi-protect'ed module.

We could also check the captured outputs to compare them against the simulated outputs to verify that everything is working correctly with --dpi-protect.

@veripoolbot
Copy link
Contributor Author


Original Redmine Comment
Author Name: Wilson Snyder (@wsnyder)
Original Date: 2019-09-24T14:40:08Z


Would ideally like to see something that works on an arbitrary design without requiring --dpi-protect, and ideally usable on any simulator. This would greatly help debug of suspected Verilator or other simulator's internal bugs.

@veripoolbot
Copy link
Contributor Author


Original Redmine Comment
Author Name: Todd Strader (@toddstrader)
Original Date: 2019-11-07T16:21:21Z


You are correct that this should not be tied to --protect-lib. I've updated the subject accordingly.

I have something kludged together which I need to spend some time cleaning up before I post it. However, I ended up picking up an external VCD parser to build this out:
https://github.com/ben-marshall/verilog-vcd-parser

Before I go to much further down this path, I wanted to discuss if/how we should use this library in Verilator.

One option (which I'm partial to) is to add it as a submodule. It's under the MIT license and my IANAL understanding is that we can build GPL'ed Verilator against it and ship it in the Verilator tarball. Adding a submodule to Verilator may cause unintended consequences in the verilator_ext_tests head-to-head builds. But I can work around that. Or perhaps that's the behavior we want so we can keep an eye on divergence with external libraries?

Another option is to only build the Verilator option (I'm calling it --replay-tool) if this library is installed on the user's system. One issue here is that this library currently doesn't have install targets in its Makefile. This path would also make it less likely for users to have access to the feature because this library definitely isn't available via package managers so it's unlikely people could use --replay-tool unless they build Verilator themselves.

Of course, we could just build our own VCD parser from scratch. But that seems like a waste as this one seems to work pretty well out of the box.

Please let me know what you think.

@veripoolbot
Copy link
Contributor Author


Original Redmine Comment
Author Name: Wilson Snyder (@wsnyder)
Original Date: 2019-11-07T23:43:01Z


https://github.com/ben-marshall/verilog-vcd-parser
Before I go to much further down this path, I wanted to discuss if/how we should use
this library in Verilator.

Interesting. Room for performance enhancements but probably good enough for us. Also could benefit from some friendly coding suggestions to remove all of the this's and other things.

I don't see support for extended VCD. We would probably want that as then have the info to eventually compare outputs. Probably easy thing to add. Maybe the author is willing to help/accept that.

BTW for VCD, my early 1990s Dinotrace parser, which is very fast, would be another but probably inferior starting point. This would need a lot of work to library-ize.

I'd suggest instead to look at using the gtkwave libraries. We already have the GTK writer library. I also think the reader should support FST reading; we want to encourage FST for the future, since VCD can't represent classes etc. I'm suspect Bybell will be happy to take patches if needed, as he has in the past.

Another option is to only build the Verilator option (I'm calling it --replay-tool)

I'm probably missing something, but does this need to be part of the verilator executable at all? Perhaps this flow?

    # Read waveform and make verilog wrapper based on I/O in the VCD
    verilator_replay foo.vcd/foo.fst --wrap foo.v
    # Verilate wrapper, ideally would work on any simulator
    # (might need a foo.cpp also; maybe that Verilator is enhanced to make)
    verilator foo.v top.v
    # Run sim, this needs the file again to playback the data
    ./foo.v +replay+foo.fst

The reason for the extra step is ideally (eventually) the Verilog created should work on any simulator. This would massively help cross-simulator debug. The next step there would be to also have the runtime replay compare output state.

We should compare & discuss these options:

  1. Part of Verilator binary

  2. New binary tool in verilator repo

  3. New tool in GTKwave repo

  4. New tool standalone repo

Note another related project to consider, once the reader libraries are in place, would be to replace the relatively-abandoned vcddiff. (Which entertainly was written by someone who leveraged my Dinotrace VCD parser, which I only found out when poking in to fix a vcddiff bug.) This does seem like a good GTKwave feature (two traces in, diff trace or ascii report out.)

@veripoolbot
Copy link
Contributor Author


Original Redmine Comment
Author Name: Todd Strader (@toddstrader)
Original Date: 2019-11-08T13:13:14Z


I'd suggest instead to look at using the gtkwave libraries.

Good point, I'll check out that codebase too. I'll try to combine gtkwave code with what I've got and then come back to discuss options 1-4.

I'm probably missing something, but does this need to be part of the verilator executable at all?

My starting point was to have Verilator create a C++ shell so it could make the replay executable from the verilated Makefile. But I can see the advantage of being simulator agnostic and starting from the trace file. How are you imagining feeding the trace data to the simulators in a generic fashion? PLI? DPI? Is that why your flow creates a top.v?

Right now I'm just manipulating the verilated class directly. I'd like to start there as I would like to use the replay tool as a way to benchmark native C++ Verilator against things such as cocotb + Verilator. I also plan to try to isolate the reading of the trace data from the simulation itself to aid in this (buffer all the trace data in Verilator-ready form?). Given this, even if we do make this simulator agnostic, I would still like to see bare-bones Verilator C++ support.

Just so you can see more what I'm talking about, here's what I've been messing with:
https://github.com/toddstrader/verilator-dev/tree/replay-tool

It's a complete mess right now, but does replay a sim when I do something like this:

test_regress/t/t_case_reducer.pl --trace
bin/verilator --cc --exe -CFLAGS "-I$PWD/obj_dir -I$PWD/../verilog-vcd-parser/build -I$PWD/include" -LDFLAGS "-L$PWD/../verilog-vcd-parser/build -lverilog-vcd-parser" +define+REPLAY_HACK --trace test_regress/t/t_case_reducer.v replay_main.cpp include/verilated_replay.cpp
make -C obj_dir/ -f Vt_case_reducer.mk
obj_dir/Vt_case_reducer test_regress/obj_vlt/t_case_reducer/simx.vcd top.t.test

The next step there would be to also have the runtime replay compare output state.

Agreed.

@veripoolbot
Copy link
Contributor Author


Original Redmine Comment
Author Name: Todd Strader (@toddstrader)
Original Date: 2019-12-16T13:29:41Z


Read waveform and make verilog wrapper based on I/O in the VCD

The problem with this is that while (I believe) FSTs will tell me which signals at a given scope are input/output ports, VCDs will not. I agree it would be better if we didn't have to involve Verilator in the generation of any of this wrapper code, but if we're going to run off of VCDs I'll need a little more design introspection. Give that, I think we need to learn what the I/O are from Verilator and not from the trace file.

This could be made to run off of Verilator XML. The advantage there is that we don't add another feature to the Verilator swiss army knife. The downside is that it's even more complicated to run this replay tool.

Also, the gtkwave repo does have a pretty straightforward looking API for FST files. But I don't think it has a similar mechanism for VCD files. I'm checking on that now:
https://sourceforge.net/p/gtkwave/mailman/message/36880053/

Given that we need additional introspection for the VCD case, I don't think #3 is a good idea. I can do #2 or #4 if I consume Verilator XML. Personally, I think #2 makes more sense if the tool is that closely coupled to Verilator. However, I think #1 still makes the most sense as it provides the simplest interface for the user.

Also, I'd personally prefer to be on FST traces, but this is holding me back:
https://www.veripool.org/issues/1566-Verilator-FST-dumping-is-slow

I guess we could make that issue a prerequisite to this one and then just forsake VCD files. However, if we want to focus on cross-tool compatibility checking here I worry that an FST-only tool would work against that. Do any of the major commercial simulators support FST dumping? I checked the ModelSim manual and didn't see anything about it.

Given all this, please let me know what you think about options #1-4 and any other thoughts you might have.

@veripoolbot
Copy link
Contributor Author


Original Redmine Comment
Author Name: Wilson Snyder (@wsnyder)
Original Date: 2019-12-17T00:34:55Z


While VCD doesn't have directional information, EVCD does. It should be straightforward to have Verilator create EVCD (optionally) instead. I believe most other simulators also have an option to make this (see IEEE 21.7.3).

We'll see what Tony says about VCD reader support.

As to FST, I recently moved the hot functions relating to slow FST (#�) into GTK wave, which might help, but the bigger issue there is still need FST format itself to change to hold binary.

The FST libraries, in theory, should work with any VPI compliant simulator. Verilator is weird in that for performance it doesn't use the VPI for dumping as others do.

Maybe an approach is to use FST for now, and if it causes pain later we can do EVCD?

@veripoolbot
Copy link
Contributor Author


Original Redmine Comment
Author Name: Todd Strader (@toddstrader)
Original Date: 2019-12-18T21:39:20Z


Tony says there's no VCD API in the gtkwave codebase. We could refactor things to make one or something like that, but I'm down with starting with just FST support. As you say, we can later extend to support EVCD in both this tool and Verilator if needed. I also realized that for cases where I want to replay a VPI-driven sim, it's not going to be as easy as finding all the inputs to a module. So I'm going to have to add a mode to specify signals at runtime. Given this, I can just use vcd2fst to work around slow FST writing at the moment.

At this point I think #2 or #4 makes the most sense. I guess I'm inclined toward #2 as it will probably get exposure to more people that way. However, I'm also fine with #4 if you'd rather not start collecting random EDA tools in the Verilator repo. Please let me know what you think.

The FST libraries, in theory, should work with any VPI compliant simulator. Verilator is weird in that for performance it doesn't use the VPI for dumping as others do.
As in registering value change callbacks via the VPI and writing the FST based on the values you get from that? Is that just theoretical? I don't see any thing mentioning VPI in the gtkwave codebase. I also imagine this could be considerably slower than native tracing depending on how a simulator implements VPI callback handling.

@veripoolbot
Copy link
Contributor Author


Original Redmine Comment
Author Name: Wilson Snyder (@wsnyder)
Original Date: 2019-12-18T22:53:20Z


This tool will help out Verilator, so if you want to go down the new binary in Verilator for now that seems ok. If the code gets too separated/ugly/etc we can split it out if necessary. The verilator_coverage tool is a good small-ish example of how to make another binary.

In GTKWave see contrib/vpi for the VPI interface.

@veripoolbot veripoolbot added area: tests Issue involves the testing system effort: weeks Expect this issue to require weeks or more of invested effort to resolve type: feature-non-IEEE Request to add new feature, outside IEEE 1800 labels Dec 22, 2019
@wsnyder wsnyder added the resolution: abandoned Closed; not enough information or otherwise never finished label Jan 2, 2022
@wsnyder
Copy link
Member

wsnyder commented Jan 2, 2022

Though a good idea, seems unlikely there will be much work towards this, if it gets more attention feel free to reopen.

@wsnyder wsnyder closed this as completed Jan 2, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area: tests Issue involves the testing system effort: weeks Expect this issue to require weeks or more of invested effort to resolve resolution: abandoned Closed; not enough information or otherwise never finished type: feature-non-IEEE Request to add new feature, outside IEEE 1800
Projects
None yet
Development

No branches or pull requests

3 participants