Aaron's Sandbox

A reader pointed me at another HDL generation solution, MyHDL. According to the MyHDL manual,

The goal of the MyHDL project is to empower hardware designers with the elegance and simplicity of the Python language.

Sounds good to me! In the interest of science, I decided to check it out. After reading the manual and tinkering with it for a while, I'm ready to talk about my experience with MyHDL.

MyHDL Features

• Documentation: I was able to install and use MyHDL by following the excellent on-line documentation. Full marks for this!
• Language: MyHDL is a Python package. Python is a decent and usable programming language. It seems about equivalent to Perl, but I get the impression that readable code might flow a little more naturally in Python.
• Built-in verification: this is a big deal. You can code your design in MyHDL and run unit tests against it, all within pure Python. Execution is very fast, and the unit tests have all the power of a modern, "free", high-level language. (Europa has nothing like this.)
• Verilog generation: the toVerilog method converts your design to Verilog. (Right, well, without this feature, MyHDL would be useless.)
• Co-simulation flow (via PLI). I haven't used this yet, but it looks like a well-thought-out story for testing the generated Verilog against the same set of unit tests which were created during the pure-Python design phase. If this works well, it sounds like an extremely useful feature.
• Hierarchy. It should be possible to create a complete, deeply hierarchical design composed of well-defined functional blocks, as I've been demonstrating in Europa.
• Synthesizable subset: it's possible to create a design in MyHDL, and a body of unit tests, only to be informed (by toVerilog) that unsupported language features were used in the design. I don't know that the supported subset will grow as MyHDL is developed - the architecture may prevent this.
• Flat Verilog: toVerilog delivers a single Verilog file with no hierarchy. This is fine for small designs, but I foresee that this flat output structure will be inadequate for large, complex designs. Of course, you could verify an entire system in one step, then generate each system sub-module's Verilog file as a separate step, but then the top-level instance that stitches all the sub-modules together will not have been verified.

Simple example: switchable inverter

I'll start off with the simplest imaginable example: a purely combinational module with one input and one output. The output is either the same as the input, or its logical inverse, according to a generation parameter. Don't close your browser window - even this simple example demonstrates interesting facets of MyHDL.

Here's the implementation:

from myhdl import always_comb
def inv_or_buf(mode, a, x):
  @always_comb
  def buffer():
    x.next = a

  @always_comb
  def inverter():
    x.next = not a

  if (mode == 0):
    logic = buffer
  else:
    logic = inverter
    
  return logic

A few things to note:

• inv_or_buf is a function which returns a locally-defined function (in this case, either buffer or inverter, depending on the generation parameter mode)
• Regular python operators are used to model the logic (either python's not operator, or a simple assignment)
• Suppose mode were used within a single local function to determine whether to buffer or invert the input? Then I'd have a 2-input combinational function, which may seem familiar - in fact, it's a 2-input XOR gate. So, input parameters are either treated as HDL module ports or as generation-time parameters - which it is depends on how the parameters are used.
• Two local functions are created, but only one is returned. This is a MyHDL idiom for creating parameterized logic. (Many thanks to reader Stefaan for his hints on this - It's alien enough to my usual way of thinking that I don't think I would have hit upon it.)
• always_comb is a decorator on the locally-defined functions, which are generator functions. For more info on these topics, you'll want to refer to the MyHDL and python documentation.

Here's something that stands out for me about MyHDL: the low-level routines which define behavior are very simple, with not much more expressive power than the HDL they will eventually be transformed to. If you want to define behavior which depends on a parameter, then for all but the simplest of behaviors, you must declare logic for all possible parameter values, and then conditionally return only the logic which corresponds to the particular parameterization. There is a special case which allows for building ROM-like logic (anything where an output doesn't depend on inputs in an easily-computable way); fortunately, you can make any combinational logic function in ROM-like logic. I rely on this in my implementation of the Avalon-ST error adapter, which I'll get to later.

Generating some Verilog

Here's some code to invoke inv_or_buf, and produce Verilog:

from myhdl import toVerilog, Signal, intbv
from inv_or_buf import inv_or_buf

def make_some_verilog(mode, name):

  a = Signal(intbv(0)[1:])
  x = Signal(intbv(0)[1:])

  toVerilog.name = name
  toVerilog(inv_or_buf, mode, a, x)

make_some_verilog(0, "buffer")
make_some_verilog(1, "inverter")

The resulting Verilog output shows up in two files:

buffer.v:

module buffer (
    a,
    x
);

input [0:0] a;
output [0:0] x;
wire [0:0] x;

assign x = a;

endmodule

inverter.v:

module inverter (
    a,
    x
);

input [0:0] a;
output [0:0] x;
wire [0:0] x;

assign x = (!a);

endmodule

The output is not the most readable code you've ever seen, but it does appear to be correct.

Leaving some for later

That's it for now. I haven't touched on MyHDL unit testing, which is one of its major strengths - I'll leave that for a future article.

Consider a very simple parameterized module, permute, with a single input and output of equal width. Output bits are driven from input bits according to a single generation parameter, mapping, which is a list of integers from 0 to (width - 1) in any order.

For example: mapping = (1, 2, 0) would generate a module containing these assignments:

  assign x[0] = a[1];
  assign x[1] = a[2];
  assign x[2] = a[0];

Making parameterized assignments like this is in europa straightforward (assume @mapping contains the particular permutation to generate):

for my $i (0 .. -1 + @mapping)
{
  $module->add_contents(
    e_assign->new({
      lhs => "x\[$i\]",
      rhs => "a\[$mapping[$i]\]",
    }),
  );
}

(europa generator, derived from europa_module_factory)

(complete Verilog listing, as generated)

A similar implementation in MyHDL turns out to be pretty clean:

  @always_comb
  def logic():
    for i in mapping:
      x.next[i] = a[mapping[i]]

(initial attempt in its entirety)

Using the above, I wrote some simple unit tests which try a variety of permutation mappings of various widths, driving a bunch of input values and verify the resulting output values. This works great! But trouble loomed when I tried to generate some Verilog. My innocent-looking code apparently angers toVerilog, and elicits a giant stack dump. Here's the final part of the stack dump, which I think is the actual error:

(lots of lines of stack trace deleted)
myhdl.ToVerilogError: in file .../permute.py, line 15:
    Requirement violation: Expected (down)range call

(complete stack dump)

So... it is documented that toVerilog does not support everything you can write in python; in fact, you're limited to a pretty small subset of the language in code which will be translated to HDL. This might be what I've run into, here, but sadly I don't know what a "(down)range call" is, nor who was expecting it.

I asked Stefaan, who originally pointed me toward MyHDL, about this, and he mentioned a discussion of an issue in a forum posting. In that thread, someone attempted a different implementation of a permuting module, and ran into trouble. Mr. MyHDL himself, Jan Decaluwe, came to the rescue with a method. The solution is to loop through the mapping not directly by element, but instead by index number, and to use a temp variable to index into the mapping list. (I think this is leveraging off of the same special case which allows inferred RAMs to work.) Here's what the modified generator function looks like:

  @always_comb
  def logic():
    tmp = intbv(0, min=0, max=len(mapping))
    for i in range(len(mapping)):
      index[:] = mapping[i]
      x.next[i] = a[int(index)]

(final MyHDL generator)

I think it's pretty clear that the above version of the generator does the same thing as the initial version (though it's a bit more cluttered), and, since my unit tests pass just fine with this version, I'm pretty happy with it. And, toVerilog runs without error. So what does the generated Verilog look like?

always @(a) begin: _permute_logic
    reg [2-1:0] tmp;
    integer i;
    tmp = 0;
    for (i=0; i<3; i=i+1) begin
        // synthesis parallel_case full_case
        case (i)
            0: tmp = 1;
            1: tmp = 2;
            default: tmp = 0;
        endcase
        x[i] <= a[tmp];
    end
end

(complete Verilog listing, as generated)

This is a lot more complex than the simple list of assignments I was hoping for.

So, the tradeoffs: with MyHDL, it's easy to write unit tests with the full power of python, and the tested behavior can then be written out as Verilog. If the conversion process is successful, the resulting generated Verilog can be regarded (though warily) as tested. But, the generated Verilog may not be so readable, and confirming that it matches the original design intent requires work (via PLI, you can run your unit tests against the generated Verilog in a simulator - haven't tried this yet).

Writing the behavioral definition can be a struggle, since it may not be clear which aspects of the language toVerilog will accept (though that's probably just my lack of experience with the tool showing through).

On the whole I think the tradeoff is worth it: MyHDL should make a strong foundation for building up a library of tested functional building blocks.

As usual, I'm attaching the various files associated with this article. At the top level are the MyHDL generator script, with its test script, verilog generator script and output file. One level down in subdirectory permute, you'll find the Europa generator, associated scripts and output file.

• To run the MyHDL unit tests: python testPermute.py
• To generate Verilog using the MyHDL generarator: python vPermute.py
• To generate Verilog using the europa generator:
  1. in a bash shell, cd to subdirectory permute
  2. run the script run1.sh

Say, do people prefer winzip files over rar files? Let me know.

20080213 20:50: Edit: for no good reason I posted used one parameterization (mapping) for the europa example, and a different one for MyHDL example. That doesn't help to make things clear! I fixed it... sorry if you were confused by the original version.

Another MyHDL experiment: the amazing Avalon-ST error adapter. I've mentioned this component before; here's a quick recap:

• The component has a single Avalon-ST sink and an single Avalon-ST source.
• "ordinary" inputs of the sink (e.g. roles data, valid) wire directly, combinationally, to outputs of the source. (Sink and source data widths must match.)
• "ordinary" inputs of the source (role ready) wire directly, combinationally, to outputs of the sink.
• The sink (and source) have an input (and output) of role error. Each individual sink and source error bit has an associated error type, which is a string. The sink and source error types need not match, nor must they have matching widths, thus some connection rules are required:
  1. matching (string match) error types are wired directly.
  2. if present, an output error bit of type "other" is driven from the logical OR of any otherwise non-matching input error bits
  3. any unmatched output error bits are driven with logical 0

As I showed in a previous article, avalon_st_error_adapter, this component, though simple to describe, is not so easy to generate. The Europa implementation turns out to be acceptable, by dint of some object overloading.

So - how does a MyHDL implementation of this component turn out?

Finding #1: It is possible!... but I certainly struggled with the implementation.

Two aspects of this component were difficult - not because it would be hard to code the logic in Python; rather, because it was hard to code the problem using the limited subset of Python which is convertible to Verilog.

1. the wiring-together of matching error bits, wherever they appeared in the error ports (component permute from the previous article was this problem, in disguise):

   In MyHDL, I created a mapping tuple, outputMapping, whose element of index 'i' gave the index of the input error signal which drove output error bit i. Then I loop over all matching output bits, assign the proper index bit to a variable (which becomes a signal in Verilog), and drive the output from the given input bit, something like this:

   # (j is large enough to hold any index into intermediate.)
   j = intbv(0, min=0, max=2 + len(i_err))
   for i in range(len(outputMapping)):
     j[:] = outputMapping[i]
     o_err.next[i] = intermediate[int(j)]
   

   Notice that I use an intermediate signal, rather than assigning directly from the input error signal. The intermediate signal is 2 bits wider than the input error signal. One "extra" bit is driven with 0 (for otherwise undriven outputs); the other is driven with the logical OR of any "other" input bits. This way, I know the index value from which to drive every output error bit, whether it's a direct-map bit, an "other" bit or an output which must be driven with 0.

2. forming the logical OR of all otherwise unconnected error inputs to drive the 'other' error output. The way I solved the puzzle was to create another tuple, otherList, containing indices of input error bits which are to be OR'ed into the output "other" bit. In another for loop, I loop over all the elements of the tuple, iteratively OR'ing in each bit into the correct bit of the intermediate signal, like this:

   # (k is large enough to hold any index into intermediate.)
   k = intbv(0, min=0, max=2 + len(i_err))
   for i in range(len(otherList)):
     k[:] = otherList[i]
     intermediate[otherIndex] = \
       intermediate[otherIndex] | intermediate[int(k)]
   

The implementation code looks very very unlike the conceptual picture in one's brain - it should be more like straight bit assignments, and an OR gate. The violent disagreement between concept and implementation is the sort of thing that makes my spider-sense tingle.

Finding #2: Unit testing is swell.

My design flow here was to add a feature, add a test for it, add another feature, add another test, ... This worked great: I found lots of bugs quickly as I went, and the need to write tests forced me to carefully consider interface/API issues.

Here's what one of my tests looks like: it tests the function of a the simplest possible error adapter, with a single input and output error bit of matching type:

def testA(self):
  """
    Simple case: data, valid, ready, one-bit matching input and output error.
    (Arguably this shouldn't be a valid error adapter - there's nothing to
    adapt!  Philosophical issue whether or not this should be supported. I'm
    going to allow it, because to make a special case of it to exclude it
    would be more work.)
  """
  # Direct mapping.
  def error_map(i_err):
    return i_err

  self.doATest( \
    "testA", \
    8, \
    { 'o_err': { 'a' : 0, }, 'i_err': { 'a' : 0, }, }, \
    error_map = error_map, \
  )

In this test I define the mapping from input error to output error, the data width, and the hash which defines the error bit mapping. Utility routines do the rest of the work of creating the actual adapter, running the simulation, verifying the outputs vs. the inputs, and running toVerilog on the adapter. Other tests have a similar framework - only the error mapping routine, the data width and the parameterization hash are varied. See test_avalon_st_error_adapter.py for the rest of the tests, and the definition of test utility routine doATest.

Finding #3: That Verilog is nigh-unreadable!

Perhaps as a natural result of the fact that the implementation relies on special MyHDL "tricks", the Verilog output appears to have been written by an evil genius. For example, if the input and output error bits are defined as follows:

{
  'o_err': { 'nomatch' : 0, 'other' : 1, 'a' : 2, 'c' : 3, },
  'i_err': { 'a' : 1, 'b' : 2, 'c' : 0, 'd' : 3, 'e' : 4, 'f' : 5, },
},

you might hope that the output error bit assignment would look something like this:

assign other = i_err[5] | i_err[4] | i_err[3] | i_err[2];
assign o_err = {i_err[0], i_err[1], other, 1'b0};

rather than this:

always @(i_err) begin: _testG_mapOutputErrorBits
    integer i;
    reg [3-1:0] k;
    reg [8-1:0] intermediate;
    reg [3-1:0] j;
    intermediate = {1'h0, 1'h0, i_err};
    k = 0;
    for (i=0; i<4; i=i+1) begin
        // synthesis parallel_case full_case
        case (i)
            0: k = 2;
            1: k = 3;
            2: k = 4;
            default: k = 5;
        endcase
        intermediate[6] = (intermediate[6] | intermediate[k]);
    end
    j = 0;
    for (i=0; i<4; i=i+1) begin
        // synthesis parallel_case full_case
        case (i)
            0: j = 7;
            1: j = 6;
            2: j = 1;
            default: j = 0;
        endcase
        o_err[i] <= intermediate[j];
    end
end

So it goes. The illegibility of the Verilog output may not matter so much in practice, if 1) the unit test facilities lead to well-verified logic, so that the Verilog doesn't need to be studied for bugs (when was the last time you looked for bugs in an assembler listing of your C code?), and 2) the toVerilog function is reliable.

I think that'll be it for now. As usual, I include a full archive of the code:

avalon_st_error_adapter.rar