Introduction

Integrity of data structures is absolutely critical to the cost of deploying the Memory Pool System. Memory corruption and memory management bugs are incredibly hard to detect and debug, often manifesting themselves hours or days after they occur. One of the key ways the MPS detects corruption or the passing of illegal data is using signatures. This simple technique has proved invaluable at catching defects early.

This document is based on [RB_1995-08-25].

Overview

.overview: Signatures are magic numbers which are written into structures when they are created and invalidated (by overwriting with SigInvalid) when they are destroyed. They provide a limited form of run-time type checking and dynamic scope checking. They are a simplified form of "Structure Marking", a technique used in the Multics filesystem [THVV_1995].

Definitions

.field: Nearly every structure should start with a field of type Sig with the name sig. For example:

typedef struct mps_message_s {
  Sig sig;                      /* design.mps.sig.field */
  Arena arena;                  /* owning arena */
  MessageClass class;           /* Message Class Structure */
  Clock postedClock;            /* mps_clock() at post time, or 0 */
  RingStruct queueRing;         /* Message queue ring */
} MessageStruct;

.value: There must also be a definition for the valid value for that signature:

#define MessageSig      ((Sig)0x5193e559) /* SIG MESSaGe */

.value.unique: The hex value should be unique to the structure type. (See .test.uniq for a method of ensuring this.)

.value.hex: This is a 32-bit hex constant, spelled using hex transliteration according to guide.hex.trans:

ABCDEFGHIJKLMNOPQRSTUVWXYZ
ABCDEF9811C7340BC6520F3812

.value.hex.just: Hex transliteration allows the structure to be recognised when looking at memory in a hex dump or memory window, or found using memory searches.

.field.end: In some circumstances the signature should be placed at the end of the structure.

.field.end.outer: When a structure extends an inner structure that already has a signature, it is good practice to put the signature for the outer structure at the end. This gives some extra fencepost checking. For example:

typedef struct MVFFStruct {     /* MVFF pool outer structure */
  PoolStruct poolStruct;        /* generic structure */
  LocusPrefStruct locusPrefStruct; /* the preferences for allocation */
...
  Sig sig;                      /* design.mps.sig.field.end.outer */
} MVFFStruct;

Init and Finish

.init: When the structure is initialised, the signature is initialised as the last action, just before validating it. (Think of it as putting your signature at the bottom of a document to say it's done.) This ensures that the structure will appear invalid until it is completely initialized and ready to use. For example:

void MessageInit(...) {
  ...
  message->arena = arena;
  message->class = class;
  RingInit(&message->queueRing);
  message->postedClock = 0;
  message->sig = MessageSig;
  AVERT(Message, message);
}

.finish: When the structure is finished, the signature is invalidated just after checking the structure, before finishing any of other fields. This ensures that the structure appears invalid while it is being torn down and can't be used after. For example:

void MessageFinish(Message message)
{
  AVERT(Message, message);
  AVER(RingIsSingle(&message->queueRing));

  message->sig = SigInvalid;
  RingFinish(&message->queueRing);
}

.ambit: Do not do anything else with signatures. See .rule.purpose.

Checking

.check.arg: Every function that takes a pointer to a signed structure should check its argument.

.check.arg.unlocked: A function that does not hold the arena lock should check the argument using AVER(TESTT(type, val)), which checks that val->sig is the correct signature for type.

.check.arg.locked: A function that holds the arena lock should check the argument using the AVERT macro. This macro has different definitions depending on how the MPS is compiled (see design.mps.config.def.var). It may simply check the signature, or call the full checking function for the structure.

.check.sig: The checking function for the structure should also validate the signature as its first step using the CHECKS() macro (see design.mps.check.macro.sig). For example:

Bool MessageCheck(Message message)
{
  CHECKS(Message, message);
  CHECKU(Arena, message->arena);
  CHECKD(MessageClass, message->class);
  ...

This combination makes it extremely difficult to get an object of the wrong type, an uninitialized object, or a dead object, or a random pointer into a function.

Rules

.rule.purpose: Do not use signatures for any other purpose. The code must function in exactly the same way (modulo defects) if they are removed. For example, don't use them to make any actual decisions within the code. They must not be used to discriminate between structure variants (or union members). They must not be used to try to detect whether a structure has been initialised or finished. They are there to double-check whether these facts are true. They lose their value as a consistency check if the code uses them as well.

Tools

.test.uniq: The Unix command:

sed -n '/^#define [a-zA-Z]*Sig/s/[^(]*(/(/p' code/*.[ch] | sort | uniq -c

will display all signatures defined in the MPS along with a count of how many times they are defined. If any counts are greater than 1, then the same signature value is being used for different signatures. This is undesirable and the problem should be investigated.

References

Document History

• 2013-05-09 RB Created based on scanty MM document [RB_1995-08-25].
• 2023-03-09 RB Justified the use of signatures at the end of structures (.field.end). Updated markup and improved tagging.

Copyright and License

Copyright © 2013–2023 Ravenbrook Limited.

Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:

1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.