1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
.. highlight:: none


.. index::
   pair: structure signatures; design
   single: signatures

.. _design-sig:


Signatures in the MPS
=====================

.. mps:prefix:: design.mps.sig
   pair: structure signatures; design
   single: signatures

.. TODO: Use RFC-2119 keywords.


Introduction
------------

Integrity of data structures is absolutely critical to the cost of
deploying the Memory Pool System.  Memory corruption and memory
management bugs are incredibly hard to detect and debug, often
manifesting themselves hours or days after they occur.  One of the key
ways the MPS detects corruption or the passing of illegal data is using
*signatures*.  This simple technique has proved invaluable at catching
defects early.

This document is based on [RB_1995-08-25]_.


Overview
--------

:mps:tag:`overview` Signatures are `magic numbers`_ which are written into
structures when they are created and invalidated (by overwriting with
``SigInvalid``) when they are destroyed. They provide a limited form
of run-time type checking and dynamic scope checking. They are a
simplified form of "Structure Marking", a technique used in the
Multics filesystem [THVV_1995]_.

.. _`magic numbers`: https://en.wikipedia.org/wiki/Magic_number_(programming)


Definitions
-----------

:mps:tag:`field` Nearly every structure should start with a field of type
:c:type:`Sig` with the name ``sig``.  For example::

    typedef struct mps_message_s {
      Sig sig;                      /* design.mps.sig.field */
      Arena arena;                  /* owning arena */
      MessageClass class;           /* Message Class Structure */
      Clock postedClock;            /* mps_clock() at post time, or 0 */
      RingStruct queueRing;         /* Message queue ring */
    } MessageStruct;

:mps:tag:`value` There must also be a definition for the valid value for
that signature::

    #define MessageSig      ((Sig)0x5193e559) /* SIG MESSaGe */

:mps:tag:`value.unique` The hex value should be unique to the structure
type.  (See :mps:ref:`.test.uniq` for a method of ensuring this.)

:mps:tag:`value.hex` This is a 32-bit hex constant, spelled using *hex
transliteration* according to `guide.hex.trans`_::

    ABCDEFGHIJKLMNOPQRSTUVWXYZ
    ABCDEF9811C7340BC6520F3812

.. _guide.hex.trans: guide.hex.trans.rst

:mps:tag:`value.hex.just` Hex transliteration allows the structure to be
recognised when looking at memory in a hex dump or memory window, or
found using memory searches.

:mps:tag:`field.end` In some circumstances the signature should be placed at
the end of the structure.

:mps:tag:`field.end.outer` When a structure extends an *inner structure*
that already has a signature, it is good practice to put the signature
for the outer structure at the end. This gives some extra fencepost
checking.  For example::

  typedef struct MVFFStruct {     /* MVFF pool outer structure */
    PoolStruct poolStruct;        /* generic structure */
    LocusPrefStruct locusPrefStruct; /* the preferences for allocation */
  ...
    Sig sig;                      /* design.mps.sig.field.end.outer */
  } MVFFStruct;


Init and Finish
---------------

:mps:tag:`init` When the structure is initialised, the signature is
initialised as the *last* action, just before validating it.  (Think
of it as putting your signature at the bottom of a document to say
it's done.)  This ensures that the structure will appear invalid until
it is completely initialized and ready to use.  For example::

    void MessageInit(...) {
      ...
      message->arena = arena;
      message->class = class;
      RingInit(&message->queueRing);
      message->postedClock = 0;
      message->sig = MessageSig;
      AVERT(Message, message);
    }

:mps:tag:`finish` When the structure is finished, the signature is
invalidated just after checking the structure, before finishing any of
other fields.  This ensures that the structure appears invalid while
it is being torn down and can't be used after.  For example::

    void MessageFinish(Message message)
    {
      AVERT(Message, message);
      AVER(RingIsSingle(&message->queueRing));

      message->sig = SigInvalid;
      RingFinish(&message->queueRing);
    }

:mps:tag:`ambit` Do not do anything else with signatures.  See
:mps:ref:`.rule.purpose`.


Checking
--------

:mps:tag:`check.arg` Every function that takes a pointer to a signed
structure should check its argument.

:mps:tag:`check.arg.unlocked` A function that does not hold the arena lock
should check the argument using ``AVER(TESTT(type, val))``, which
checks that ``val->sig`` is the correct signature for ``type``.

:mps:tag:`check.arg.locked` A function that holds the arena lock should
check the argument using the :c:macro:`AVERT` macro. This macro has different
definitions depending on how the MPS is compiled (see
`design.mps.config.def.var`_). It may simply check the signature, or
call the full checking function for the structure.

.. _design.mps.config.def.var: config.txt.html#design.mps.config.def.var

:mps:tag:`check.sig` The checking function for the structure should also
validate the signature as its first step using the :c:func:`CHECKS()` macro
(see `design.mps.check.macro.sig <check.txt>`_). For example::

    Bool MessageCheck(Message message)
    {
      CHECKS(Message, message);
      CHECKU(Arena, message->arena);
      CHECKD(MessageClass, message->class);
      ...

This combination makes it extremely difficult to get an object of the
wrong type, an uninitialized object, or a dead object, or a random
pointer into a function.


Rules
-----

:mps:tag:`rule.purpose` **Do not** use signatures for any other purpose.
The code must function in exactly the same way (modulo defects) if
they are removed.  For example, don't use them to make any actual
decisions within the code.  They must not be used to discriminate
between structure variants (or union members). They must not be used
to try to detect *whether* a structure has been initialised or
finished.  They are there to double-check whether these facts are
true. They lose their value as a consistency check if the code uses
them as well.


Tools
-----

:mps:tag:`test.uniq` The Unix command::

    sed -n '/^#define [a-zA-Z]*Sig/s/[^(]*(/(/p' code/*.[ch] | sort | uniq -c

will display all signatures defined in the MPS along with a count of how
many times they are defined.  If any counts are greater than 1, then the
same signature value is being used for different signatures.  This is
undesirable and the problem should be investigated.


References
----------

.. [RB_1995-08-25] Richard Brooksby. Harlequin. 1995-08-25. "`design.mps.sig: The design of the Memory Pool System Signature System <https://info.ravenbrook.com/project/mps/doc/2002-06-18/obsolete-mminfo/mminfo/design/mps/sig/>`__".

.. [THVV_1995] Tom Van Vleck. 1995. "`Structure Marking <https://www.multicians.org/thvv/marking.html>`__".