If you want to see just how much of a difference this can make, consider the following simple code:

std::string longestString(
    std::vector<std::vector<std::string>> data) {
  std::string longest = "";
  for (auto subvec : data) {
    for (auto curstring : subvec) {
      if (curstring.size() > longest.size()) {
        longest = curstring;
      }
    }
  }
  return longest;
}

This has a couple of crucial mistakes: in our loop iteration, subvec and curstring are captured by value. This means that the outer loop copies all of the subvectors and their associated strings and then the inner loop copies all of the strings again. Additionally, the data parameter is taken by value, meaning that the caller must also copy the entire nested vector structure. This copying all amounts to a huge amount of unnecessary work!

Fixing this example is, thankfully, simple. All we need to do is make the data parameter and loop range parameters const references:

std::string longestString(
    const std::vector<std::vector<std::string>>&
      data) {
  std::string longest = "";
  for (const auto& subvec : data) {
    for (const auto& curstring : subvec) {
      if (curstring.size() > longest.size()) {
        longest = curstring;
      }
    }
  }
  return longest;
}

In my measurements, with a tiny input, the version with unnecessary copying was 15 times slower than the version without, which shows just how huge this cost can be. In this example, while the performance penalty was high, the unnecessary copy was, thankfully, not too hard to spot as the offending variables are explicitly not references. Unfortunately, however, it is not always quite so clear.

Lifetime Extension

There are often occasions in which we write functions that return a const reference. This might be to allow (logically) read-only access to a protected member variable of an object or a protected global variable (e.g. function local static). Another example would be ‘search’ like functions, including a more optimised version of our longestString function above.

To avoid the copy, the caller of this function must also specify they want a reference:

const std::string& getRefToGlobalVariable();

void foo() {
  // Allowed and safe, but makes an unnecessary copy.
  std::string aVal = getRefToGlobalVariable();

  // Better, no copy!
  const std::string& aRef
    = getRefToGlobalVariable();
}

An interesting thing happens, however, if we try the same thing with a return by value function:

std::string getStringByValue();

void foo() {
  // The obvious way, no problem this time
  std::string aVal = getStringByValue();

  // Interesting...
  const std::string& aRef = getStringByValue();
}

This case of trying to assign a value to a temporary seems like it shouldn’t work. In the first case, getStringByValue will produce a temporary object which is then moved or copied into aVal as appropriate (ignoring copy elision, which you can read about in part 2 of the ‘C++ Lifetime Quiz’). The temporary is then immediately destroyed.

In the second case, we might expect getStringByValue to produce a temporary object whose reference is saved to aRef only for the temporary to be immediately destroyed. If this happened, aRef would be left as a dangling reference making any usage of it undefined behaviour!

However, that isn’t what happens at all. The C++ standard specifies that this is a special case and must be handled differently. Instead, a local unnamed variable is created to store the result of getStringByValue by value and aRef becomes a reference to that unnamed local variable. As a result, aRef isn’t a dangling reference at all and is completely safe to use!

However, we’re sort of getting the worst of both worlds here. We still have to take the result of the function call by value so no copies have been avoided. On top of that, we don’t get any of the benefits of a ‘value’ variable such as named return value optimisation (NRVO), an optimisation that can sometimes avoid copies when returning a named local variable.

The Ternary Operator

The ternary operator is a commonly used alternative to short if statements, as seen is the simple strMax function:

const std::string& strMax(
    const std::string& a,
    const std::string& b) {
  return a > b ? a : b;
}

Thankfully, the ternary operator ‘passes through’ references as we would expect and so strMax doesn’t require any copies!

Now, let’s consider the case of loading a value from a map (by reference, to avoid unnecessary copies) but with a default value in the case that our key isn’t in the map:

const ValueType& makeDefault();

void foo(
    const std::map<std::string, ValueType>& map,
    const std::string& key) {
  auto it = map.find(key);
  const ValueType& myValue =
    it != map.end()
    ? it->second
    : makeDefault();

  // Do something with myValue here
}

As in our strMax example, this works exactly as we would want. Our result in myValue doesn’t require any copies and the (possibly expensive) makeDefault is only called if the key is not in the map.

However, what if makeDefault returns by value instead? This may be the case if it is derived from some configuration we don’t want to cache, for example. We might hope that we get a normal reference in the ‘found in map’ case and lifetime extension of the default in the ‘not in map’ case but this is unfortunately not what happens.

Instead, we get lifetime extension in the ‘not in map’ case but a lifetime extended copy in the ‘found in map’ case, which is not all what we want! The reason this happens is that the ternary operator must have a consistent type across both its branches and, crucially, reference and value types are different.

The operator is forced to choose its type as either const ValueType& or ValueType. If it chose const ValueType& then it would have to cast ValueType to const ValueType&. The lifetime extension we discussed earlier cannot be used here; it is only applicable when the result of an expression is assigned to a local reference variable. This means lifetime extension can’t be used ‘mid-ternary’ and so the cast to const ValueType& would always result in a dangling reference and is therefore forbidden.

This forces the compiler to chose the type to be ValueType which requires a copy in the ‘found in map’ case but avoids dangling references. When assigning the result of this ternary to myValue, lifetime extension is applied. This nasty interaction between ternary operators having uniform return types and lifetime extension means this compiles and work correctly, but with an unnecessary copy subtly and silently eating away at our performance.

Resolving this is not very neat, but a possible alternative implementation would be:

ValueType makeDefault();

void foo(
    const std::map<std::string, ValueType>& map,
    const std::string& key) {
  auto it = map.find(key);

  // Load the default, but only if we need it
  std::optional<ValueType> m_default =
    it != map.end()
    ? std::nullopt
    : makeDefault();

  // Both it->second and *m_default are references,
  // so there is no unnecessary copy
  const ValueType& myValue =
    it != map.end()
    ? it->second
    : *m_default;

  // Do something with myValue here
}

The only question now is how we can identify this case. At the time of writing, none of the tested compilers (Clang, GCC, and MSVC) generate any warnings for this case even with all warning enabled. I’m also not aware of any major static analysis tools that detect this.

Some other alternatives would be to delete the copy constructor of ValueType; all of your unnecessary (and, unfortunately, necessary) copies become compiler errors! This has a rather high false positive rate though and may not be possible at all if ValueType is part of code you do not own. Another alternative is to change the type of myValue to a non-const reference; lifetime extension is only allowed for const references and so this will also result in a compiler error. This only works for lines of code you’re already suspicious of and will also fail to compile in the ‘fixed’ code because the map is const.

If the copy does end up being a significant cost, however, then it is likely to show up in profiling results which is actually how I first discovered this behaviour. At least when it shows up there, you’ll be able to quickly spot the root cause of the problem!

To explore them, we’ll need to add a few methods to our Speaker class:

class Speaker {
  ...

  Speaker(const Speaker& other)
      : word_(other.word_),
        depth_(other.depth_ + 1) {
    std::cout << "Copy constructing: "
      << depth_ << " "
      << word_ << std::endl;
  }

  Speaker(Speaker&& other)
      : word_(other.word_),
        depth_(other.depth_ + 1) {
    other.word_ = "DANGER Moved";
    std::cout << "Move constructing: "
      << depth_ << " "
      << word_ << std::endl;
  }

  Speaker& operator =(const Speaker& other) {
    word_ = other.word_;
    depth_ = other.depth_ + 1;
    std::cout << "Copying: "
      << depth_ << " "
      << word_ << std::endl;
    return *this;
  }

  Speaker& operator =(Speaker&& other) {
    word_ = other.word_;
    other.word_ = "DANGER Moved";
    depth_ = other.depth_ + 1;
    std::cout << "Moving: "
      << depth_ << " "
      << word_ << std::endl;
    return *this;
  }

  ...
};

Let’s start with a simple example to demonstrate a copy:

int main() {
  Speaker s1{"A"};
  Speaker s2{s1};
  return 0;
}

Constructing: 1 A
Copy constructing: 2 A
Destructing: 2 A
Destructing: 1 A

What if we construct s2 using the = symbol instead?

int main() {
  Speaker s1{"A"};
  Speaker s2 = s1;
  return 0;
}

Constructing: 1 A
Copy constructing: 2 A
Destructing: 2 A
Destructing: 1 A

Interestingly, it uses the constructor here and not the assignment operator. The assignment operator can only be used if we have already constructed the object.

int main() {
  Speaker s1{"A"};
  Speaker s2{std::move(s1)};
  return 0;
}

Constructing: 1 A
Move constructing: 2 A
Destructing: 2 A
Destructing: 1 DANGER Moved

In this example, we see one of the surprising behaviours of moving objects; their destructors still get called. Strictly speaking, ‘move’ is a bit of a misnomer. What’s really happening is not a move at all, but a destructive copy. The objects s1 and s2 are always completely separate, we are just asserting that when s2 is constructed by ‘moving’ s1, it is acceptable for s1 to be left in an invalid state. To support this, the standard asserts that any access to a variable after it has been moved—except for its destructor—results in undefined behaviour. The destructor, however, is still run for objects that have been ‘moved’.

Copy Elision

Now, what happens if we try to make the previous example a little more concise? After all, it is invalid to access s1 after s2 is constructed, so why bother giving it a name at all? Instead, let’s construct s2 from a temporary Speaker instead of a named Speaker.

int main() {
  Speaker s2{Speaker{"A"}};
  return 0;
}

Constructing: 1 A
Destructing: 1 A

Oh, that’s a surprise. Only one Speaker gets constructed even though we explicitly try to construct two of them.

It turns out that C++ permits one kind of optimisation that can actually modify the behaviour of a program when used: copy (and move) elision. Essentially, the compiler can see that the temporary Speaker here is a little bit pointless; it would be immediately ‘moved’ into s2 and then be deleted. The copy elision optimisation skips this intermediate temporary object and instead constructs s2 directly. In our example, however, the constructor and destructor of Speaker has some clearly visible side effects, so the behaviour of the program is very obviously modified.

In this particular case, the standard actually demands that copy elision be performed. We will see some more examples of mandatory copy elision later and also examples of optional copy elision where the standard allows this optimisation to be used, but does not require it.

To understand some other cases of copy elision, let’s look at how returning values from functions works without this optimisation. You’ll have to forgive the complexity of this one; the standard allows (but does not require) copy elision when returning a named local variable (plus a few extra conditions, which we don’t need to worry about now). Modern compilers are quite good at doing copy elision of this type, so a little work was necessary to avoid it.

Speaker getSpeaker(bool b) {
  Speaker s1{"A"};
  Speaker s2{"B"};
  if (b) {
    return s1;
  }
  else {
    return s2;
  }
}

int main() {
  getSpeaker(true);
  return 0;
}

Constructing: 1 A
Constructing: 1 B
Move constructing: 2 A
Destructing: 1 B
Destructing: 1 DANGER Moved
Destructing: 2 A

Where did that move constructor come from? It actually comes from the return statement; returning by value may require a copy or move. This copy or move will be performed by the return statement itself, so the fact we don’t use the returned value in main is irrelevant.

Interestingly, as far as the standard is concerned, returning by value may equally not require a copy or move. It is entirely up to the compiler to decide and is not even required to be consistent: one call to getSpeaker may require this move but another may not. In practice, this inconsistency is possible and would typically be a result of inlining.

The destructor order is also a little odd; not the straightforward reverse of construction we are used to. The variables s1 and s2 are local to the scope of getSpeaker so are deleted when that scope exits. The return value, however, is in the scope of main (albeit as a temporary) so isn’t deleted until after control returns to main. This means that s1 and s2 must be deleted before the returned value.

Now, let’s tweak getSpeaker a little and see what happens.

Speaker getSpeaker(bool b) {
  Speaker s1{"A"};
  Speaker s2{"B"};
  Speaker s3{"C"};
  return s2;
}

Constructing: 1 A
Constructing: 1 B
Constructing: 1 C
Destructing: 1 C
Destructing: 1 A
Destructing: 1 B

Suddenly, the move constructor vanishes! There is a rule of thumb for when returning requires a move/copy and when it doesn’t: if a variable (s2) can be guaranteed to be returned when it’s constructed, then a copy or move will not be required. Remember, however, that the standard does not guarantee this rule of thumb will work so do not rely on it!

Now, that we’ve looked into returning values, we can also look at passing them in to a method.

void useSpeaker(Speaker s) {
  s.speak();
}

int main() {
  useSpeaker(Speaker{"A"});
  return 0;
}

Constructing: 1 A
Speaking: 1 A
Destructing: 1 A

This example is pretty straightforward, but it is nice to notice here that we get copy elision saving the temporary from main having to be copied or moved to s.

int main() {
  Speaker s{"A"};
  useSpeaker(s);
  return 0;
}

Constructing: 1 A
Copy constructing: 2 A
Speaking: 2 A
Destructing: 2 A
Destructing: 1 A

However, if we are passing in a named variable, we lose copy elision here. We could try moving s to get it back.

int main() {
  Speaker s{"A"};
  useSpeaker(std::move(s));
  return 0;
}

Constructing: 1 A
Move constructing: 2 A
Speaking: 2 A
Destructing: 2 A
Destructing: 1 DANGER Moved

This hasn’t got us to the copy elision we really wanted, we’re just using the move constructor instead of the copy constructor. The only way around this is to pass in a temporary as in the earlier example or modify the useSpeaker method to take a reference instead.

void useSpeaker(Speaker s1, Speaker s2) {
  s1.speak();
  s2.speak();
}

int main() {
  useSpeaker(Speaker{"A"}, Speaker{"B"});
  return 0;
}

Constructing: 1 B
Constructing: 1 A
Speaking: 1 A
Speaking: 1 B
Destructing: 1 A
Destructing: 1 B

Constructing: 1 A
Constructing: 1 B
Speaking: 1 A
Speaking: 1 B
Destructing: 1 B
Destructing: 1 A

Notice how the second parameter here actually gets constructed before the first parameter. As far as the standard is concerned, the order in which method parameters are evaluated is actually indeterminate and different compilers do choose different orders. While this has far broader implications than just object lifetime, we can see an example of how it impacts lifetime consideration in the following innocent looking example.

void useSpeaker(Speaker s1, Speaker s2) {
  s1.speak();
  s2.speak();
}

int main() {
  Speaker s{"A"};
  useSpeaker(s, std::move(s));
  return 0;
}

Constructing: 1 A
Move constructing: 2 A
Copy constructing: 2 DANGER Moved
Speaking: 2 DANGER Moved
Speaking: 2 A
Destructing: 2 DANGER Moved
Destructing: 2 A
Destructing: 1 DANGER Moved

Constructing: 1 A
Copy constructing: 2 A
Move constructing: 2 A
Speaking: 2 A
Speaking: 2 A
Destructing: 2 A
Destructing: 2 A
Destructing: 1 DANGER Moved

Notice how we might end up copying an object that has already been moved, something that is very definitely undefined behaviour. The problem here is obvious after having seen the last example, but is often very easy to miss in real code.

Temporaries

Speaker passthroughSpeaker(Speaker s) {
  return s;
}

int main() {
  passthroughSpeaker(Speaker{"A"}).speak();
  return 0;
}

Constructing: 1 A
Move constructing: 2 A
Speaking: 2 A
Destructing: 2 A
Destructing: 1 DANGER Moved

This example has actually has a lot going on. Firstly, we’re not getting any copy elision when returning s here. This is because copy elision is actually forbidden when returning a method parameter.

Secondly, what is going on with the destructor order? The lifetime of the temporary used as the argument to passthroughSpeaker is surprisingly long. You might think its lifetime is tied to the s argument of passthroughSpeaker, but this is not the case. Instead, it is tied to the full expression in which it is constructed, i.e. the line of main in which passthroughSpeaker is called. This is also true of the temporary returned by passthroughSpeaker so they are both destroyed at the same time, after the speak method is finished. The common tie-breaking rule is applied, they are destroyed in the reverse order of their construction.

The natural converse of a constructor is a destructor, which runs when the object’s lifetime ends instead of when it starts. This feature, however, is not included in most programming languages; garbage collection in particular is often a barrier to this as it makes it impossible to predict when an object will actually be destroyed. C++ is one of the few languages that do allow destructors to be written and in this series we will explore the wide range of edge cases that can occur.

Even if you’re not writing your own destructors, they are widely used throughout the standard library. Some examples include ‘data-owning’ classes such as std::string, std::vector, and std::map which all use the destructor to free the memory they were using. The std::unique_ptr class similarly uses its destructor to free the memory it points to and std::shared_ptr uses its destructor to decrement a reference counter and, if that counter hits 0, free the memory it points to. Destructors aren’t only used for memory management though; std::unique_lock is used in multi-threaded code, locking a std::mutex in its constructor and unlocking it in its destructor.

In all of the examples below, the output is hidden until you choose to reveal it. I suggest you treat this as a quiz and try to work out the example programs’ output by hand before revealing the correct answer.

To demonstrate clearly when constructors and destructors are being called, we will use the Speaker class defined below.

class Speaker {
 public:
  explicit Speaker(const char* word)
      : word_(word), depth_(1) {
    std::cout << "Constructing: "
      << depth_ << " "
      << word_ << std::endl;
  }

  ~Speaker() {
    std::cout << "Destructing: "
      << depth_ << " "
      << word_ << std::endl;
    word_ = "DANGER Deleted";
  }

  void speak() const {
    std::cout << "Speaking: "
      << depth_ << " "
      << word_ << std::endl;
  }

 protected:
  const char* word_;
  int depth_;
};

Local Storage

Let’s start with a simple example of this in action:

int main() {
  Speaker s{"A"};
  std::cout << "Running" << std::endl;
  return 0;
}

Constructing: 1 A
Running
Destructing: 1 A

The s variable is valid until the end of main and so the destruction happens at the very end. The rule in general is that destruction of local variables happens when they go out of scope. Some examples to demonstrate this:

void foo() {
  Speaker sfoo{"foo"};
}

int main() {
  Speaker s1{"A"};
  if (true) {
    Speaker s2{"B"};
  }
  foo();
  return 0;
}

Constructing: 1 A
Constructing: 1 B
Destructing: 1 B
Constructing: 1 foo
Destructing: 1 foo
Destructing: 1 A

In the previous examples, there has only ever been one variable that needed destructing at any moment. What happens if we have two variables created in the same scope?

int main() {
  Speaker s1{"A"};
  Speaker s2{"B"};
  return 0;
}

Constructing: 1 A
Constructing: 1 B
Destructing: 1 B
Destructing: 1 A

The rule here is that they are always destroyed in the reverse of the creation order so, as s1 was created first, it is destroyed last. This rule also holds in other contexts which we will see later.

Global Storage

We can also have global variables and, as with local variables, their constructor is run at their creation and their destructor when they are destroyed.

Speaker sglobal{"Global"};
Speaker sglobal2{"Global2"};

int main() {
  Speaker slocal{"Local"};
  return 0;
}

Constructing: 1 Global
Constructing: 1 Global2
Constructing: 1 Local
Destructing: 1 Local
Destructing: 1 Global2
Destructing: 1 Global

As you might expect, global variables are constructed before the main method is called and destructed after the main method completes. This, perhaps surprisingly, means that parts of your program can run before main starts and after it finishes! The global variables are also constructed in the natural order (from the top of the file down) and are destructed in the reverse order to their creation.

Now, what happens if we have global variables defined across multiple files?

a.cpp:
Speaker sa{"A"};

main.cpp:
Speaker sb{"B"};

int main() {
  return 0;
}

Constructing: 1 A
Constructing: 1 B
Destructing: 1 B
Destructing: 1 A

Constructing: 1 B
Constructing: 1 A
Destructing: 1 A
Destructing: 1 B

As we saw in the previous example, the order in which global variables are constructed is well-defined within a file but, as this example shows, it is not defined across files. The standard simply gives us no way to decide which of these two global variables is constructed first. In practice, this makes defining singletons as global variables rather risky; we’ll see a better approach (static storage) later but, as with most features in C++, that also has its pitfalls. We are, however, still guaranteed that destruction will happen in the reverse order of creation.

Now, what if we define a global variable in a header instead? In my testing, this actually failed to link due to the symbol having multiple definitions. I’ve seen linkers in the past allow this and, in that case, you would actually get a completely separate global variable for each .cpp file that includes the header. The way around this is to use the extern keyword; this is similar to function declarations in headers in that it doesn’t actually create a global variable and instead only reserves a name and type. This variable must then be initialised in a single .cpp file.

Static Storage

Static storage is typically used to solve the same sorts of problems as global variables, but with a few key differences. The biggest one of these is that they are defined within a function, similarly to local storage, and can only be accessed directly from within the scope in which they are defined. We can clearly demonstrates the global-like behaviour of static variables with a simple example that counts how many times a method is called:

int getInt() {
  static int i = 0;
  i++;
  return i;
}

int main() {
  std::cout << getInt() << std::endl;
  std::cout << getInt() << std::endl;
  return 0;
}

1
2

If you tweak this example and try to access i in main, you will get an error, just as you would if it were a local variable.

Now let’s use an example with our Speaker object to see exactly when static variables are created and destroyed.

void foo() {
  static Speaker s{"static"};
}

int main() {
  std::cout << "Starting main" << std::endl;
  foo();
  foo();
  std::cout << "Ending main" << std::endl;
}

Starting main
Constructing: 1 static
Ending main
Destructing: 1 static

We can see here that static variables are constructed when they are first used and, as with global variables, they are destroyed at the end of the program. Interestingly, static variables still follow the rule that they are destroyed in the reverse order to their creation. This means that the destruction order of this type of global variable can depend on the earlier program flow. Static variables are great if you have singletons that depend on each other in their constructors (as long as there are no circular dependencies) but this destruction order can be a problem if the singletons depend on each other in their destructors.

We may also wonder what happens if a static variable is used within another static variables constructor. Which is destroyed first?

const Speaker& getSpeakerSingleton() {
  static Speaker s{"Singleton"};
  return s;
}

class SpeakerUser {
 public:
  SpeakerUser() {
    getSpeakerSingleton();
  }

  ~SpeakerUser() {
    std::cout << "Destructing SpeakerUser" << std::endl;
  }
};

const SpeakerUser& getSpeakerUserSingleton() {
  static SpeakerUser su;
  return su;
}

int main() {
  getSpeakerUserSingleton();
  return 0;
}

Constructing: 1 Singleton
Destructing SpeakerUser
Destructing: 1 Singleton

This gives us our answer! The practical upshot of this is that if we use a static variable in our constructor, it can also be safely used in our destructor. However, if we don’t use a static variable in the constructor, what happens if we try using it in the destructor? What if it has already been destroyed?

const Speaker& getSpeakerSingleton() {
  static Speaker s{"Singleton"};
  return s;
}

class SpeakerUser {
 public:
  SpeakerUser() {
  }

  ~SpeakerUser() {
    std::cout << "Destructing SpeakerUser" << std::endl;
    getSpeakerSingleton().speak();
  }
};

const SpeakerUser& getSpeakerUserSingleton() {
  static SpeakerUser su;
  return su;
}

int main() {
  getSpeakerUserSingleton();
  return 0;
}

Destructing SpeakerUser
Constructing: 1 Singleton
Speaking: 1 Singleton
Destructing: 1 Singleton

The output here looks perfectly sensible, but we have actually done something vary dangerous; we have constructed a static variable after the main method exits! According to the standard, this is undefined behaviour; the wording is a mouthful but it states that the reverse construction order rule must hold. Therefore, because the Speaker is constructed after the SpeakerUser, the Speaker must be destroyed first but this isn’t possible as it doesn’t exist until after that point! In practice, this sort of program may work, but it is definitely better avoided.

const Speaker& getSpeakerSingleton() {
  static Speaker s{"Singleton"};
  return s;
}

class SpeakerUser {
  public:
  SpeakerUser() {
  }

  ~SpeakerUser() {
    std::cout << "Destructing SpeakerUser" << std::endl;
    getSpeakerSingleton().speak();
  }
};

const SpeakerUser& getSpeakerUserSingleton() {
  static SpeakerUser su;
  return su;
}

int main() {
  getSpeakerUserSingleton();
  getSpeakerSingleton();
  return 0;
}

Constructing: 1 Singleton
Destructing: 1 Singleton
Destructing SpeakerUser
Speaking: 1 DANGER Deleted

In this example, we see another, more dangerous, misuse of static variables. The Speaker is constructed after the SpeakerUser so it must be destroyed first. Because static variables are only ever constructed once, this means that the SpeakerUser tries to access a deleted variable, something that falls very comfortably into the realms of undefined behaviour that should be avoided.

Our final concern here may be about how the destruction order of static and global variables interact.

const Speaker& getSpeakerSingleton() {
  static Speaker s{"Singleton"};
  return s;
}

class SpeakerUser {
  public:
  SpeakerUser() {
    getSpeakerSingleton();
  }

  ~SpeakerUser() {
    std::cout << "Destructing SpeakerUser" << std::endl;
  }
};

Speaker s1{"A"};
SpeakerUser su;
Speaker s2{"B"};

int main() {
  return 0;
}

Constructing: 1 A
Constructing: 1 Singleton
Constructing: 1 B
Destructing: 1 B
Destructing SpeakerUser
Destructing: 1 Singleton
Destructing: 1 A

This example shows that they are interleaved with each other, using the reverse construction order tie-breaking rule.

Dynamic Storage

The last type of storage is dynamic storage. This covers any object created by new.

int main() {
  Speaker* sp = new Speaker{"A"};
  return 0;
}

Constructing: 1 A

This example neatly shows what makes dynamic storage special (and dangerous); the destructor doesn’t get called automatically! These objects will exist in memory and not have their destructors called until we explicit delete them. Forgetting to do this can cause hard to debug memory leaks, so it’s almost always better to use the standard library’s smart pointer classes instead.

int main() {
  Speaker* sp = new Speaker{"A"};
  delete sp;
  return 0;
}

Constructing: 1 A
Destructing: 1 A

We could consider one more example for dynamic storage. Because we call the destructor manually, we could try calling it twice.

int main() {
  Speaker* sp = new Speaker{"A"};
  delete sp;
  delete sp;
  return 0;
}

Constructing: 1 A
Destructing: 1 A
free(): double free detected in tcache 2
** Program crashes **

Ah, better to not do that.

Another thing to note with dynamic storage variables is that the object is not tied to the name at all; sp is a unrecognised command emph to a Speaker, not a Speaker itself. We can see this in the following example.

int main() {
  Speaker* sp = new Speaker{"A"};
  Speaker* sp2 = sp;
  sp->speak();
  sp2->speak();
  delete sp2;
  sp->speak();
  return 0;
}

Constructing: 1 A
Speaking: 1 A
Speaking: 1 A
Destructing: 1 A
Speaking: 6979600

Even though we have two variables, sp and sp2, there is only ever one Speaker. This means that calling speak on sp after deleting sp2 is undefined behaviour, explaining the garbled output we get here.

Member Variables

Let’s add a barebones class that has a Speaker as a member variable:

class SpeakerHolder {
 public:
  SpeakerHolder(const char* word)
      : s(word) {
    s.speak();
  }

  ~SpeakerHolder() {
    s.speak();
  }

 protected:
  Speaker s;
};

And a minimal example using it; the question is whether s is constructed and destructed before or after SpeakerHolder.

int main() {
  SpeakerHolder sh{"A"};
}

Constructing: 1 A
Speaking: 1 A
Speaking: 1 A
Destructing: 1 A

The answer is that we get the behaviour we probably want, the member variable s is totally safe to use in both the constructor and destructor of the SpeakerHolder.

This scheme is remarkably simple and starts by representing the number 0 as \( \lambda fx . x \). We then represent 1 as \( \lambda fx . fx \), 2 as \( \lambda fx . f ( fx ) \), 3 as \( \lambda fx . f ( f ( fx )) \), and so on. The number \( n \) is represented as a lambda function that takes two arguments, \( f \) and \( x \), and repeatedly applies \( f \) to \( x \) a total of \( n \) times! For convenience, I will write \( [n] \) as a shorthand for the Church numeral representing the number \( n \), so \( [0] = \lambda fx . x \) and so on.

You may notice that \( \beta \)-substitution doesn’t allow a lambda function to inspect its arguments at all. As a result, this idea of blurring the lines between functions and data is necessary to represent anything meaningfully within the Lambda Calculus.

Arithmetic With Church Numerals

If you have read the ‘What is Maths’ series, you will know that if we are armed with the ability to add 1 and to describe recursive definitions, we will be able to reconstruct both addition and multiplication. If you haven’t read that series, then fear not; we will go over the details we need here as well.

Before we start, we should think about how we want our algorithms for these arithmetic operations to look. I will claim that, in the example of adding by 1, we want to find some term \( S \) such that \( S [n] =_\beta [n + 1] \) for any natural number \( n \). Similarly, for addition, we want to find some term \( A \) such that \( A [n] [m] =_\beta [n + m] \) for any natural numbers \( n \) and \( m \).

Firstly, let’s think about how we can implement addition by one in the Lambda Calculus. As the only fundamental operation we can perform is substitution, we should think about what sort of substitutions we can make that might achieve this. If we replace \( x \) with \( fx \) in any Church numeral, we will have succeeded in adding one! This naturally results in thinking about some term like \( [n] f ( fx ) \)—this will be the Church numeral \( [n] \), but substituting \( f \) into \( f \) and \( fx \) into \( x \). This is clearly not in the form we want, but it’s a good start.

Next, to make this \( \beta \)-equivalent to a Church number, we have to wrap it in a lambda abstraction, so we need to instead think about \( \lambda fx . [n] f ( fx ) \). If we are concerned about \( f \) and \( x \) being the subject of lambda abstraction both at the top level of this term and hidden away in \( [n] \), which we should be, we can simply appeal to \( \alpha \)-equivalence. The final step is to pull out \( [n] \), allowing us to separate this into two terms, one we shall call \( S \) and \( [n] \). This ends up looking like \( ( \lambda afx . a f (fx) ) [n] \) and so we can write \( S = \lambda afx . a f (fx) \). In order to convince ourselves that this works, let’s look at an example:

\begin{align*} S [3] &= ( \lambda afx . a f (fx) ) [3] \\ &= ( \lambda afx . a f (fx) ) ( \lambda fx . f ( f ( fx ))) \\ &=_\alpha ( \lambda afx . a f (fx) ) ( \lambda g y . g ( g ( g y ))) \\ &\to_\beta \lambda fx . ( \lambda g y . g ( g ( g y ))) f (fx) \\ &\to_\beta \lambda fx . ( \lambda y . f ( f ( f y ))) (fx) \\ &\to_\beta \lambda fx . f ( f ( f ( fx ))) \\ &= [4] \end{align*}

Now that we have addition by one, we are well on our way to general addition. By noticing that \( n + m = n + 1 + 1 + \cdots + 1 \) where there are \( m \) ones, we develop our plan. Recalling our earlier observation that the Church numeral \( [m] \) is also a function that applies some other function \( m \) times, we can see that \( [m] S [n] \) produces a result by applying \( S \) (addition by one) to \( [n] \) a total of \( m \) times. We can again pull out the numbers \( [m] \) and \( [n] \), suggesting the term \( A = \lambda ab . a S b \) is a suitable candidate for addition. For clarity and conciseness, I am reusing \( S \) from above; if we want, we can instead write this out in full, \( A = \lambda ab . a ( \lambda a^\prime fx . a^\prime f ( fx )) b \). Now, let’s check our work with an example:

\begin{align*} A [3] [2] &= ( \lambda ab . a S b ) [3] [2] \\ &\to_\beta ( \lambda b . [3] S b ) [2] \\ &\to_\beta [3] S [2] \\ &= ( \lambda fx . f ( f ( fx ))) S [2] \\ &\to_\beta ( \lambda x . S ( S ( S x ))) [2] \\ &\to_\beta S ( S ( S [2] )) \\ &=_\beta S ( S [3] ) \\ &=_\beta S [4] \\ &=_\beta [5] \end{align*}

We can reuse this same idea again for multiplication. By noting that \( n \times m = n + \cdots + n + 0 \) where \( n \) is repeated \( m \) times, we see that \( [n \times m] \) can be obtained by applying addition by \( n \) to \( [0] \) a total of \( m \) times. This gives us the term \( M = \lambda ab . a X [0] \) where \( X \) is some term that needs to be filled in and should represent addition by \( n \) (assuming that \( b = [n] \)). While we could revisit our work on addition to try and devise some new term that achieves this, it would be easier to simply reuse \( A \)—it turns out that \( X = Ab \) does exactly what we need!

This may be surprising if we think of \( A \) as a two-argument function. However, we can equally think of \( A \) a single-argument function that produces a new single-argument function. With this second interpretation, we can recover the two-argument behaviour, apply \( A \) to the first argument and the resulting function to the second argument. In fact, if you return to the definition of the Lambda Calculus, you will see that there are no ‘true’ two-argument functions; syntax such as \( \lambda ab . \cdots \) is really just a shorthand for \( ( \lambda a . ( \lambda b . \cdots )) \)!

This means that we can neatly say \( M = \lambda ab. a (Ab) [0] \); although it is only neat because of all the shorthands we’re using! I will also leave the example of this multiplication for you to test! It is worth saying that our descriptions of addition and multiplication are not necessarily unique. For example, an alternative (and much more concise) way to implement multiplication in the Lambda Calculus is \( M = \lambda abfx . a (bf) x \). Similarly, we could have implemented addition as \( A = \lambda abfx . a f ( b f x ) \).

While being able to perform basic arithmetic is an essential part of describing algorithms, we will be very limited without some way to ‘inspect’ numbers and make decisions based on our calculations. In the next part, we will explore how we can do just that.

Despite their presence throughout human history, algorithms themselves were never really studied until recently. However, with the development of computers and the increasing formalisation of maths, a rigorous discussion of algorithms became inevitable. What’s more, as with many new developments, several different mathematicians attempted this formalisation at more or less the same time. The two approaches that are most widely used are Turing Machines (developed, but not named, by Turing) and the Lambda Calculus (developed and named by Church). In this series, we will focus on the Lambda Calculus—while it appears much more detached from modern computer hardware than Turing Machines, it is, in my opinion, far more elegant and beautiful.

The Foundation of Computation

One important question that should be answered by a theory of algorithms and computation is what the fundamental operations are. In other words, what is the simplest language you need to be able to describe any algorithm. This may seem like a purely academic question—it is likely to be beyond inconvenient to actually describe an algorithm using these fundamental operations—but it actually has important practical implications. When designing computer hardware and programming languages, you should, at a bare minimum, make sure that it can handle these fundamental operations. If you fail to do so, there will be some algorithms that it will be unable to perform!

In the ‘What is Maths’ series, we used addition by 1 as the fundamental operation in Number Theory. When discussing algorithms, however, there isn’t such a natural feeling fundamental operation—Church and Turing took two totally different approaches.

In Turing Machines, the algorithm and the data it’s working with are kept separately. The data is a list of 0s and 1s and the algorithm is always focused on a single item in that list. At each step, it may update the currently focused item, shift its focus left or right, and decide the next step to perform based on whether the currently focused item is a 0 or a 1.

In the Lambda Calculus, there is no fundamental distinction between algorithm and data. Any data can be interpreted as an algorithm and any algorithm can be treated as data, although it may not always make sense to do so. The fundamental operation of Lambda Calculus is substitution: locating a symbol and replacing it with some other sequence of symbols.

Despite the radical differences between these two approaches, they were proven to be equivalent in the seminal Church-Turing thesis. The idea behind this proof is remarkably simple: provide an algorithm described in the language of Lambda Calculus that can simulate any Turing Machine, and an algorithm described in the language of Turing Machines that can simulate the Lambda Calculus. The details of this proof, however, are anything but simple. In addition, it was argued in this thesis that the Lambda Calculus and Turing Machines are a good definition for computation. While definitions cannot be proven to be true or untrue, this proposal has been, as far as I’m aware, universally accepted. Furthermore, the argument for this definition has only been strengthened in the years since—every subsequent attempt at defining computation is either equivalent to the two above or less powerful.

The Terms in Lambda Calculus

Everything in the Lambda Calculus is made up of ‘terms’ and, as mentioned above, these are used to represent both algorithms and data. There are three rules by which we can construct terms and, furthermore, if something cannot be constructed by these rules, it is not a term of Lambda Calculus.

The first rule is that ‘variables’ are terms. We will represent variables with lower case letter from the English alphabet; for example \( a \), \( x \), and \( z \) are all variables and, therefore, also terms. It should be noted that the variables by themselves have no meaning, they are provided meaning by the terms in which they are used. In addition, we are allowed to use as many variable as we need—if we run out of letters we can get more variables by adding dashes to letters, such as \( a^\prime \).

The second rule is that of application: if \( X \) and \( Y \) are terms, then so is \( (XY) \). We will later see that many terms can be thought of as functions and this rule should be thought of as allowing us to represent a function being applied to some data. In this example, \( X \) represents the function and \( Y \) the data that \( X \) is acting upon. Combining this rule with the first, we can produce a few more terms: \( (xy) \), \( ((xy)z) \), \( (x(yz)) \), and \( (xx) \) for example. To avoid the tedious writing of far too many brackets, I will often cheat a bit and leave them out. When the brackets are left out, they should be added ‘from the left’, so when I write something like \( xyz \), I really mean \( ((xy)z) \) and not \( (x(yz)) \)—the difference is important!

The third and final rule is that of lambda abstraction, and the namesake of the Lambda Calculus itself. It states that if \( X \) is a variable and \( Y \) is a term, then \( ( \lambda X . Y ) \) is a term. This can loosely be thought of as defining functions and we will explore this idea more in the following section. Some example terms, using all three rules are \( ( \lambda x . x ) \), \( ( \lambda x . ( \lambda y . x ) ) \), and \( ( \lambda x . (xx) ) \). I will also introduce another shorthand for this rule: I will write \( \lambda xy . x \) instead of \( ( \lambda x . ( \lambda y . x ) ) \) and include the enclosing brackets only when necessary.

Before we move on, I would like to make a subtle point about these shorthands clear: something like \( \lambda xy . xxy \) is not a true term of the Lambda Calculus, but instead represents the term \( ( \lambda x . ( \lambda y . ((xx)y))) \).

Substitution

The heart of computation in the Lambda Calculus is substitution. There are two kinds of substitution we can perform, which are called \( \alpha \) (alpha) and \( \beta \) (beta) substitution. I think Church must have been really fascinated by the Greek alphabet when he designed this system!

The simpler substitution rule is \( \alpha \)-substitution, and is essentially the codification of the idea that the letters we choose to represent variables are arbitrary. To perform an \( \alpha \)-substitution on a term, we select an original variable that is used in our term and a new variable that is not used in our term. We then replace every occurrence of the original variable with the new one. For example, \( \lambda x . x \to_\alpha \lambda y . y \). The \( \to_\alpha \) symbol is used to indicate that we are applying \( \alpha \)-substitution. Another slightly more complex example is:

\[ \lambda xy . xxy \to_\alpha \lambda xb . xxb \to_\alpha \lambda ab . aab \]

For cases such as the above where several steps are needed to transform one term into another, we introduce the \( =_\alpha \) symbol and the idea of \( \alpha \)-equivalence. For example, we can write \( \lambda xy . xxy =_\alpha \lambda ab . aab \).

Instead of applying \( \alpha \)-substitution to an entire term, we may also apply it to subterms of the form \( \lambda X . Y \) so long as the variable we swap out is \( X \). This is done to encode the idea that when a variable is bound by a lambda abstraction, it is a ‘new’ version of that variable that is separate from any uses of the same symbol outside the abstraction. For example:

\[ ( \lambda x . x ) x \to_\alpha ( \lambda y . y ) x \]

The other substitution rule, \( \beta \)-substitution, is where the magic happens. This substitution rule justifies the meanings I gave for all three of the construction rules. The rule is applied to terms of the form \( ( \lambda X . Y ) Z \); note that \( X \) must be a variable and both \( Y \) and \( Z \) are arbitrary terms. To apply this rule, we will also require that the variable \( X \) is not rebound by any lambda abstraction in \( Y \); if it is then \( \alpha \)-substitution should first be used to resolve this. The result of the application is the term \( Y \), but with every occurrence of the variable \( X \) replaced by the term \( Z \). This is probably best explained by a few examples:

\begin{align*} ( \lambda x . x ) z &\to_\beta z \\ ( \lambda x . xx ) ( \lambda y . y ) &\to_\beta ( \lambda y . y ) ( \lambda y . y ) \\ ( \lambda y . y ) ( \lambda y . y ) &\to_\beta \lambda y . y \\ ( \lambda xy . xxy ) ( \lambda z . z ) a &\to_\beta ( \lambda z . z ) ( \lambda z . z ) a \\ ( \lambda z . z ) ( \lambda z . z ) a &\to_\beta ( \lambda z . z ) a \end{align*}

When using \( \beta \)-substitution, we must be careful about the shorthands we introduced earlier. We have that \( ( \lambda z . z ) a \to_\beta a \), but it is not the case that \( ( \lambda xy . xxy ) ( \lambda z . z ) a \to_\beta ( \lambda xy . xxy ) a \).

Finally, we introduce the \( =_\beta \) symbol, similarly to \( \alpha \)-substitution. However, there are a few subtle differences. Firstly, we allow \( \alpha \)-substitution as steps in a substitution subsequence for the \( =_\beta \) symbol. Secondly, there is a clear directionality to \( \beta \)-substitution but not for \( \alpha \)-substitution. When considering \( \beta \)-equivalence, we may apply \( \beta \)-substitution in either direction.

I imagine that, at this point, this may all seem like pointless symbol shuffling. However, in the next part, we’ll explore how we can represent data in this system and start to develop useful functions.

Furthermore, some of the basic facts (which I won’t prove here) regarding ordering and divisibility are also disturbingly similar. We also have that if \( a \mid b \) and \( b \mid a \), then \( a = b \). In addition, if \( a \mid b \) and \( b \mid c \), then \( a \mid c \).

However, we shall now start to explore the differences between addition and multiplication. It turns out that they are in fact wildly different. One of them can be thought of as tame, while the other is a wholly chaotic beast. As a first peek, we know that for any natural numbers \( a \) and \( b \), we have either \( a \geq b \) or \( b \geq a \) but the corresponding statement about multiplication is not true. For example, neither \( 2 \mid 3 \) nor \( 3 \mid 2 \) is true.

Let us now imagine that we want a way to describe the numbers without referring back to the successor operation. We would like our method to use as few ‘symbols’ as possible; for this discussion, numbers like 10 will be considered a single unique symbol, and not composed of the symbols 1 and 0.

Our first option is simply to replace the successor operation with ‘+’ and ‘1’. As we noted earlier, addition by 1 is exactly the same as taking the successor, so, in combination with 0, these three symbols are all we need. You may be concerned that large numbers are somewhat impractical to describe using this method, but the crucial fact is that it is possible to describe them—the amount of patience needed to do so is irrelevant!

Now, for contrast, what if we wanted a multiplication based scheme instead of an addition based scheme. Because we know that \( a \times b \geq a \) whenever \( b > 0 \), our scheme must still include 0 and 1; they cannot be constructed by multiplying together larger numbers. However, with just ‘\( \times \)’, ‘0’, and ‘1’, we actually cannot construct any numbers larger than 1. Therefore, we must add ‘2’ to this scheme. With this addition, we construct quite a few additional numbers: 4, 8, 16, 32, and so on. But we are still missing some: 3, for example. So again, we add ‘3’ to the scheme. Despite this, we still cannot make the number 5, so this must also be added. On and on we can go, but it will turn out that no finite multiplication based scheme of this form is sufficient (we will prove this later).

In general, this idea of looking for the ‘building blocks’ of some collection, which can be combined with some operation to produce the whole collection, is really quite useful. In some sense, it reduces the problem of understanding the collection into just understanding these building blocks and the way they combine. In this particular case, the radical differences between the building blocks with respect to addition and with respect to multiplication are deeply revealing.

Prime Numbers

The building blocks of the natural numbers with respect to multiplication are (except for 0 and 1) the prime numbers, a concept I’m sure you’re already familiar with. In fact, we have already developed enough knowledge on top of the axioms that we can use the traditional definition.

Because any number is divisible by itself and 1, this is equivalent to the alternative version of the definition: that a prime number is a number divisible only by itself and 1. However, the definition given above makes it a little clearer that we do not wish to consider 1 a prime (this is usually done as it is necessary for prime factorisations to be unique, although we won’t prove that here).

We should now choose some familiar numbers and ask whether or not they are prime. To do so, we will benefit from using a result that follows from a few of the facts given at the end of the previous part: if \( a \mid b \) and \( b \neq 0 \), then \( b \geq a \).

Firstly, 6 is not prime: \( 2 \times 3 = 6 \) so \( 2 \mid 6 \) (this is enough because we already know 1 and 6 divide 6). Now, let’s ask whether 3 is prime. Because of the fact above, we only need to check numbers less than or equal to 3 for divisibility. We already know that 1 and 3 divide 3, so the only other numbers we need to check are 0 and 2. However, 0 divides nothing other than itself, leaving one number left to check. We now consider the sequence \( 2 \times 0 = 0 \); \( 2 \times 1 = 2 \); \( 2 \times 2 = 4 \); \( 2 \times 3 = 6 \); and so on. We can argue that 3 isn’t in this sequence because the sequence is increasing (can you prove this?) and we have already passed 3 without it appearing in the list. Therefore, 3 is not divisible by 2 and so 3 is prime!

While that was a little onerous, a very similar method can be used to test the primality of any other number you are concerned about.

We must now return to our question of building blocks. Certainly, any prime number must be a building block, for it cannot be made by multiplying together two different numbers. However, it is not at all clear that the primes are enough to be the building blocks by themselves; we shall now try to prove that this is the case. To do so, we will make use of a rather fundamental fact: any non-empty collection of natural numbers has a minimum. I suggest you try to prove this using induction—the link between this fact and induction is so deep that they can actually be thought of as equivalent. What’s more, this link transcends the context of natural numbers, although we won’t go any deeper into it for now.

This proof makes use of a technique called ‘proof by contradiction’, a strategy we haven’t seen before that can often seem rather bizarre. Its more historic name is ‘reduction to the absurd’, which makes the strangeness of it clear in the title.

The reason it comes across as rather suspect is that the very first line of the proof states that we are going to assume the statement which we are trying to prove is actually wrong. We then work for a while in a nonsensical wonderland-style system of maths in which numbers can be both prime and not prime. Finally, we proclaim that the system we were working in throughout the proof is nonsense and so our statement is actually true.

To see why this works, we must consider what the consequences of just assuming a statement to be true (or false) are. If we are correct in our assumption, then everything is fine and we continue working in the regular number theory. If, however, our assumption is incorrect, then we have entered wonderland. The goal of a proof by contradiction is to show that, when assuming some statement is false, we enter wonderland. A classic way of doing this is to show that some other statement is both true and false, in our example above this is the claim that \( n \) is prime. The other way this is often done is by ‘disproving’ some result that we know to be true; ‘disproving’ the claim that \( 1 \neq 0 \) is a favourite!

You may also have noticed that my proofs have become a little less rigorous—this is intentional. At this stage of our development of number theory, the proofs become more focused on concepts and less on ‘symbol shunting’. While it is certainly possible to retain the level of rigour from the original proofs, I believe it would only obscure the proof’s spirit.

The Quantity of the Primes

We know a few prime numbers, but we may now wonder just how many there are? How many building blocks do we need to describe any number using multiplication? To answer this, I shall defer to Euclid, who solved this problem over 2000 years ago with what I consider to be one of the most elegant arguments in number theory.

Despite knowing that there are infinitely many primes, we may feel that we haven’t fully answered the question of how many there are. In some imprecise sense, we may feel like there are more multiples of 4 than there are multiples of 2, even though there are infinitely many of both. There are many directions we could approach this question from but for now we shall approach this by asking how big the gaps between two consecutive primes are, where consecutive primes are those that have no other primes between them. In another twist, we shall answer this question by instead thinking about how long a sequence of consecutive non-primes we can find.

It now follows from this result that you can find consecutive primes with as large a gap between them as you want. We can take this result to suggest that the prime numbers are extremely rare, something seemingly at odds with there being infinitely many of them!

Conclusion

It is quite amazing to now go back to the axioms and see just how far we have come from. We started talking about nothing more than 0 and \( S \) and are now discussing a wealth of concepts: addition, multiplication, ordering, prime numbers, and more. It is also startling to look back at those early proofs. We started with proofs that involved a lot of pushing symbols around and these final proofs forego the heavy use of symbols and instead connect concepts and ideas.

While I doubt that many of the facts we’ve proven here are new to you, I hope that you will have gained an appreciation for the connections between them and the logic underpinning them. On top of that, though, you are now equipped with the mathematical tools needed to develop your own proofs and further explore number theory. In fact, I encourage you to do just that and will finish this series by presenting a few problems for you to experiment with!

A Selection of Number Theory Problems

1. Let us call a number \( n \) ‘square’ if there is a number \( r \) such that \( r^2 = n \). Which numbers can be written as the sum of two squares?
2. A prime triple is a set of three numbers that can be written as \( p \), \( p + 2 \), and \( p + 4 \) and are all prime. How many prime triples are there?
3. Can you prove that if \( a \mid b \) and \( a \mid c \), then \( a \) divides the difference between \( b \) and \( c \)? You may notice that we relied on this fact in the proof that there are infinitely many primes.
4. Show that, if \( p \) is a prime larger than 3, then \( p^2 \) is 1 larger than a multiple of 24.

Notice that this relies on the definition of addition; trying to define this without reusing addition would be rather tricky. This also means that it will take a rather long time to fully unwind this definition for all but the smallest numbers!

As this definition is very similar in spirit to the definition of addition, I won’t discuss this too much further. It turns out, somewhat unsurprisingly, that multiplication is also associative and commutative. I encourage you to think about how this might be proven, but as it’s rather similar to addition, I won’t discuss it here.

I will, however, show a proof of an important fact about how multiplication and addition interact, often called ‘multiplying out brackets’, or ‘distributivity’. The proof consists mostly of the tedious shuffling around of symbols, but it’s worth paying attention to how it relies heavily on the commutativity and associativity of addition. Just try to imagine how unpleasant the proof would be if we hadn’t already proved these facts!

In the same way we defined multiplication in terms of addition, we can also define exponentiation (also called powers) in terms of multiplication.

\begin{align*} a^0 &= 1 \\ a^{Sb} &= a \times a^b \end{align*}

We won’t discuss exponentiation too much here, and we won’t go any further in this hierarchy. However, there is no fundamental reason why you should stop at exponentiation; in fact, I would encourage you to think about what might come next!

Quite surprisingly, exponentiation is neither commutative nor associative. What’s more, it isn’t too hard to find a counterexample: \( 2^3 = 8 \), but \( 3^2 = 9 \). However, this counterexample does very little to explain anything in a satisfying way. I believe you will gain the most understanding by trying to prove commutativity yourself, even if it is an impossible task. A small hint though: it has a lot to do with the topics we’ll discuss in the next part.

Ordering and Divisibility

At the end of the previous part, I posed the question of what might make a good definition for the statement ‘\( a \) is greater than \( b \)’. Shortly, I will reveal my preferred definition. I say preferred here because there is not necessarily a correct answer to this question—there are in fact multiple definitions that may all look rather different at first, but actually turn out to be equivalent. Despite this, there are some ‘bad’ definitions. For example, I could define ‘\( a \) is greater than \( b \)’ to mean \( a = 0 \) and \( b \neq 0 \). While this is, in the abstract philosophical sense discussed earlier, a legitimate thing to do, it is also rather perverse because it so rudely contradicts our preconceived ideas of what ‘greater than’ should mean.

If this definition is different to yours, can you prove that they’re equivalent? If not, can you find an example where the two definitions disagree?

We will discuss the concept of divisibility more later, but I want to bring it up here to highlight the similarity it has to the definition of ordering. It is quite remarkable that these two concepts, which intuitively feel so distinct, have near identical definitions! Our symbolic form of this definition is \( b \mid a \), which can be read as ‘\( b \) divides \( a \)’.

Armed with this new definition, we should try out some examples. Firstly, because \( 0 + 1 = 1 \), we can say \( 1 \geq 0 \). We also know that \( 1 \neq 0 \) so we can go further and say \( 1 > 0 \). As another example, we know \( 3 + 2 = 5 \) and \( 3 \neq 5 \), so we can say \( 5 > 3 \).

Now let’s try something the other way around, is \( 0 \geq 1 \)? Obviously, we are tempted to say no, but can we prove it? This actually turns out to be quite a bit harder than the examples above. Before, our examples consisted of plucking the difference out of thin air and saying ‘look, it works’! It isn’t always easy to pluck the right thing out of thin air, but, in this case, we have the advantage of a strong intuition. However, proving that 0 is not greater than or equal to 1 actually requires us to show that there is no choice of difference that would work. In other words, we need to show that for all natural numbers \( d \), we have \( 1 + d \neq 0 \).

Thankfully, this proof is quite manageable. By using commutativity, we can say \( 1 + d = d + 1 \). Then recalling the definition of addition, we have that \( d + 1 = S d \). Finally, axiom 4 tells us that \( S d \) is not 0 and so we have proved that 0 is not greater than 1. By extending this logic, we can prove that 0 is not greater than any natural number, although I shall leave the details for you to sort out.

When I suggested proving that 0 is not greater than 1, I expect your intuition immediately jumped in to claim that this must be because \( 1 > 0 \). However, this line of proof relies on a fact that we haven’t yet proven: if \( a \geq b \) and \( b \geq a \), then \( a = b \).

To prove this statement, we will first need an intermediate result. The intermediate result we need is actually so important that is has a name—the Cancellation Lemma. If you haven’t seen the word ‘lemma’ before, it is often used to describe results that are very useful, but not terribly interesting by themselves.

This lemma is, I believe, the most obscure use of induction we have seen so far. The first line of the proof makes the observation that the claim is equivalent to saying that if \( b + a = c + a \), then \( b = c \). The latter is much neater to prove as it removes the need to keep using commutativity.

Now, the induction here is of a rather different style than what we have seen up until now. Essentially, all the statements we’ve used induction on previously have just been an equation containing the number on which we are using induction. For example, when we wanted to prove that \( S ( n + a ) = S n + a \), we just used induction over \( a \) on this equation.

In the cancellation lemma however, this is not enough. The statement we use induction on is ‘for any natural numbers \( b \) and \( c \) such that \( b + a = c + a \), it is the case that \( b = c \)’. It goes without saying, but this is a much more complex statement than just an equation. Despite this though, the actual structure of the induction is the same as everything we have seen before. You may have to read the proof a few times before you fully understand it.

A natural result of the cancellation lemma is that if \( a + b = a \), then \( b = 0 \); it is actually this result that we use in the following proof.

This proof is really quite exciting—we were able to prove something very substantial without using induction at all! This is really a testament to the usefulness of the results we have already proven. It is also worth noting that we didn’t refer to a single axiom or the fundamental \( S \) operation. In some sense, we have achieved a ‘higher level’ view of the natural numbers, not just in our results, but in the proofs as well.

I will now leave you with some facts that will be important to us in the following part, but which I will let you prove:

• If \( a \geq b \) and \( b \geq c \), then \( a \geq c \).
• For all numbers, either \( a \geq b \) or \( b \geq a \) (or both).
• For all numbers, \( a + b \geq a \).
• For all numbers, \( a \times b \geq a \); this statement actually needs an extra condition to be true, can you spot what it is? The previous statement will also serve as a very useful intermediate result here.

In contrast to the definition of \( D \) we discussed previously, addition is an operation that requires two numbers as input.

It is worth noting that we must, at this stage, be far more careful about how we treat addition then we may be used to. Firstly, it is not at all obvious, from our definition, that \( a + b \) is the same as \( b + a \). Secondly, something like \( a + b + c \) is not covered at all by our definition, so we shouldn’t write this. We can instead write \( ( a + b ) + c \) or \( a + ( b + c ) \) but it is not clear that these are the same either, so it is important to make the brackets explicit. Despite these warnings, if our definition is ‘good’, we will eventually be able to prove that these familiar facts hold.

Now for an example:

\begin{align*} 3 + 2 &= S S S 0 + S S 0 & \text{by the definitions of 2 and 3} \\ &= S ( S S S 0 + S 0 ) & \text{by the definition of +} \\ &= S S ( S S S 0 + 0 ) & \text{by the definition of +} \\ &= S S ( S S S 0 ) & \text{by the definition of +} \\ &= S S S S S 0 & \text{just moving brackets for clarity} \\ &= 5 & \text{by the definition of 5} \end{align*}

This example has worked out exactly as we would have wanted! Another example worth looking at is \( n + 1 \), where \( n \) is an arbitrary natural number. It turns out that \( n + 1 = S n \), which justifies that intuitive description of \( S \) as being addition by 1. In fact, we can go one step further and provide a whole list of similar examples, all of which can be easily obtained by unwrapping the definition a few times.

\begin{align*} n + 0 &= n \\ n + 1 &= S n \\ n + 2 &= S S n \\ n + 3 &= S S S n \\ &\cdots \end{align*}

However, if we tried a ‘reversed’ list of examples, starting at \( 0 + n \), we would run into trouble right away. We cannot unwrap the definition at all here without first knowing whether \( n \) is either 0 or the successor of another natural number. Crucially, this is not a problem with the definition—this problem arises because we don’t know enough about \( n \).

You may have noticed another approach; if we substitute 0 into \( n \) in our first list of examples, you will see that the resulting infinite list of results corresponds exactly to saying \( 0 + n = n \). This idea of collapsing an infinite list of similar statements into a single claim is exactly what induction allows us to do. We can now start to write a formal proof.

In the proof above, I have left out the tedious details of proving everything that needs to be a natural number is, in fact, a natural number. This makes the proof much clearer and easier to read. Furthermore, it is, in my opinion, clear enough in every case except for one which I shall leave for you to prove—is \( a + b \) a natural number whenever \( a \) and \( b \) are?

Leaving out these details also makes it clear just how intimately induction and recursion are linked; our inductive step involves little more than simply unwrapping a single level of the recursive definition.

Flipping Addition

We will now try to prove another key fact about addition: \( a + b = b + a \). In case I haven’t already introduced enough obscure maths words, this property is called commutativity. You may find it interesting to attempt to prove this yourself before reading ahead. However, it is harder than any of the proofs we have seen up until now!

The first thing we must be careful of is that induction only lets us ‘decompose’ one number at a time. In situations like this though, there are two numbers we might want to perform induction on and so we must take extra care not to misuse the inductive axiom. My preferred method for making sure I keep things in order is to only use induction on a single number in any given proof. By instead referring to intermediate results that are proved separately, we can more clearly see that everything is in order.

If you tried proving this claim yourself using induction, you will have found that the base case is a very similar statement to what we just proved. This is our first example of using an intermediate result. Instead of trying to copy the proof that \( 0 + a = a \) into our proof of \( a + b = b + a \), we can simply say that it’s already been proved. In addition to referencing our above proof, all that is needed is to reference the definition of addition: \( a + 0 = a \).

The inductive step, however, poses a little more difficulty; this is not at all unusual with induction. In the proof I provide below, we attempt to prove \( a + S n = S n + a \) using the inductive hypothesis \( a + n = n + a \). The only thing I can see that we’re allowed to do here is to unwrap the definition of addition in \( a + S n \), transforming it to \( S ( a + n ) \). This now leaves us with the option to apply the inductive hypothesis, transforming our formula further to \( S ( n + a ) \).

Unfortunately, however, this is where we get stuck. It is not at all obvious that \( S ( n + a ) = S n + a \). This is where we return to the idea of intermediate results, and the proof of this statement is below.

You will notice that we are now using induction on the other number. By using induction in this way, we achieve a radical transformation of the problem we’re trying to solve. Instead of the awkward \( S ( n + a ) = S n + a \) which leaves us little room to move, we get to look at \( S ( n + S a ) = S n + S a \) and have an inductive hypothesis to boot. All these extra \( S \)s open up our ability to unwind the addition and the inductive hypothesis is, as usual, an incredibly powerful tool.

In fact, in this particular proof, the inductive hypothesis continues to do all the work. The only other steps we need are unwrapping and rewrapping addition, which are really doing nothing more than massaging the expressions into the shape that allows the inductive hypothesis to be used.

Before we continue on to finish our proof, it is worth noticing something interesting about the last two theorems we have proved: \( 0 + a = a \) is the same as the first line in the definition of addition, but flipped, and \( S n + a = S ( n + a ) \) is the same as the second line, but flipped. After noticing this, commutativity feels inevitable.

Now that we are armed with this intermediate result, the proof of addition’s commutativity falls into place.

In case that wasn’t enough for you, try proving what is called the associative property: \( ( a + b ) + c = a + ( b + c ) \). As we have already proved the commutative property, you may now use that freely, in the same way that we used \( S ( n + a ) = S n + a \) and \( 0 + a = a \) in our proof of the commutative property. Be warned though: this is a much more challenging proof and you are likely to need several intermediate results!

Another exercise to consider is how we might define the statement ‘\( a \) is greater than \( b \)’ in the language of number theory. There are multiple equivalent ways to do this, but my favourite makes uses of addition in a rather elegant way.

As discussed in the previous section, we know that \( \{ 0, 1, 2, 3, \ldots \} \) are all natural numbers, but are they all the natural numbers or are we missing some?

Imagine, for a moment, that we have an extra object \( \infty \) such that \( S \infty = \infty \) and we want to know whether it is a natural number. The first axiom obviously does not apply immediately. The second axiom can be applied, but, rather unhelpfully, only tells us that if \( \infty \) is a natural number, then \( \infty \) is a natural number. The third axiom tells us only that there is no natural number \( n \) other than \( \infty \) such that \( S n = \infty \). The fourth axiom is equally unhelpful.

Crucially, it turns out that the first four axioms are not capable of proving whether or not \( \infty \) is a natural number! In order to resolve this problem, we require the inductive axiom. To apply the inductive axiom, we first need to select a statement we wish to prove for all natural numbers \( n \). In this example, we shall use the statement \( n \neq \infty \). We then need to prove two things: that our statement is true about 0 (we call this the base case), and that we can do a ‘logical step’ (we call this the inductive case).

It is clear that \( 0 \neq \infty \) so our statement is true about 0. Now, our logical step starts by assuming we have some natural number \( n \) about which our statement is true—in this case, \( n \neq \infty \). To complete our logical step, we must prove that the statement is also true about \( S n \)—in this particular case, that is \( S n \neq \infty \). We actually already noticed that this step follows immediately from specialising the third axiom: \( n = \infty \) if and only if \( S n = S \infty \). Or, because \( S \infty = \infty \), we can write this equivalently as \( n = \infty \) if and only if \( S n = \infty \). As we are supposing \( n \neq \infty \), it follows that \( S n \neq \infty \) also. We can now apply the inductive axiom, which tells us that \( n \neq \infty \) for all natural numbers \( n \); in particular, \( \infty \) is not a natural number!

We can think of this induction as doing the following:

1. We start by knowing \( 0 \neq \infty \).
2. Our ‘logical step’ allows us to go from 0 to 1, so we now know \( 1 \neq \infty \).
3. Our ‘logical step’ allows us to go from 1 to 2, so we now know \( 2 \neq \infty \).
4. …

The inductive axiom tells us that this infinite process will eventually capture all natural numbers, and so this proof method is legitimate.

Now, let's look at another example of using induction.

As a side note, I would like to point out that we have not proven \( \infty \) is not something that can reasonably be discussed; only that it is not within the framework of number theory. Considering strange objects such as this and trying to extend the current system to include them has turned out to be a very useful method of discovering new maths!

Recursion

Recursion and induction are rather similar concepts, but where induction applies to proofs, recursion applies to definitions. Let's look at an example: defining a function \( D ( n ) \) that, informally, we will think of as doubling a number.

The first line of this definition is totally straightforward and we can consider it as being similar to the base case of induction. The second line, at first glance, appears more suspect. Apparently we need to know what \( D \) is before we can know what \( D \) is; a mathematical chicken and egg!

However, from another perspective, the second line can be interpreted as saying we need to know what \( D ( n ) \) is before we can know what \( D ( S n ) \) is. This line of thinking is much less troubling, especially as \( n \) is ‘smaller’ than \( S n \)—even if, strictly speaking, we haven't actually defined what ‘smaller’ means in the context of natural numbers. In fact, if we try to compute \( D ( 2 ) \), we will find that this second perspective is far more helpful.

\begin{align*} D ( 2 ) &= D ( S S 0 ) & \text{by the definition of 2} \\ &= S S D ( S 0 ) & \text{by the definition of \( D \)} \\ &= S S S S D ( 0 ) & \text{the definition of \( D \) again} \\ &= S S S S 0 & \text{the definition of \( D \), yet again} \\ &= 4 & \text{by the definition of 4} \end{align*}

Whilst expanding out this recursive definition is painfully slow, we do eventually get ourselves an answer, and it's exactly what we might have hoped for! Our definition works because of two key facts about the natural numbers; I won’t provide proofs here so feel free to test your knowledge of induction by attempting to prove them yourself. Firstly, every natural number is either 0 or the successor of another natural number, but not both. This fact means that there is exactly one line of our definition that applies—having no lines or multiple lines applying to a number would be a problem. Secondly, every natural number can be written as a finite number of \( S \)s followed by a 0. This second fact essentially guarantees that our process of repeatedly applying the definition will eventually stop.

Another question you may wish to consider is whether \( D ( n ) \) is always a natural number. If you think you know, try and prove it!

Peano claims that there is a collection of ‘things’ we can refer to as natural numbers, and an operation, called \( S \), that can be applied to these numbers. If \( n \) is a natural number, then we would write \( S ( n ) \) to refer to the result of applying \( S \) to \( n \). Formally, we call \( S \) the ‘successor’ operation, but informally we can think of it as adding 1. I will typically omit the parentheses, writing \( S n \) instead of \( S ( n ) \), for the sake of clarity. We now suppose the natural numbers and \( S \) together satisfy the following five axioms:

1. 0 is a natural number.
2. For every natural number \( n \), we have that \( S n \) is also a natural number.
3. For all natural numbers \( n \) and \( m \), we have that \( n = m \) if and only if \( S n = S m \).
4. For every natural number \( n \), we have \( S n \neq 0 \).
5. Suppose \( P ( n ) \) is a statement about natural numbers. If \( P ( 0 ) \) is true and, for every natural number \( n \), \( P ( n ) \) implies \( P ( S n ) \), then \( P ( n ) \) is true for every natural number \( n \).

I hope you can at least accept the first four of those axioms, we’ll discuss the fifth in a bit more detail later. Hopefully I can convince you to accept that as well if you haven’t already!

As a word of warning, I may drop the word ‘natural’ when talking about the natural numbers, instead just referring to them as numbers. As I’m sure you’ll agree, writing out the word natural over and over again is simply too much effort.

A Definition Diversion

While it is very nice to think that these facts are enough to derive all of number theory, there may, at first glance, seem to be a few glaring omissions. Namely, does the number 1 exist? What is addition? What is multiplication? What other concepts are we missing?

To kick this philosophical discussion off, I’ll throw a definition at you. We will define 1 to be \( S 0 \).

Firstly, does the definition even make sense? Well, axiom 1 tells us that 0 is a natural number, which means it makes sense to talk about \( S 0 \). Secondly, axiom 2 now tells us that \( S 0 \) is a natural number itself. So, to no one’s surprise, we have proven that 1 is a natural number!

At first glance, it might look like we’ve cheated and introduced a sixth axiom, namely \( 1 = S 0 \). This raises the rather subtle point of what mathematicians mean when they provide a ‘definition’.

Imagine that instead of defining 1 to be \( S 0 \), I had defined Peano’s number to be \( S 0 \). I expect that in this case, you would have no problems with accepting this definition—there would be no concern about whether it constitutes an extra axiom. The difference here is that we have a preconception of ‘1’ is, but we don’t have any opinions about what Peano’s number should be.

When provided a definition in formal mathematics, we should throw aside our preconceptions and treat this as something totally new, even if it has a familiar name. In our particular case, when we define \( 1 = S 0 \), we should think of this as a new ‘1’, and not the ‘1’ that we are familiar with.

However, there is a slight hiccup in this line of thinking: how can our axioms of number theory tell us anything about numbers if they require us talk about a ‘new 1’? As we learn more about number theory using the axioms, if our definition of ‘1’ is ‘good’, then we shall find that it behaves exactly how we would expect our ‘preconceived 1’ to behave. We may then claim that anything we learn about our ‘new 1’ actually tells us about the real world, although that is an argument that must be made outside of the framework of mathematics. It is entirely possible that it turns out our definition is ‘bad’ and doesn’t correspond at all with our ‘preconceived 1’; if this turns out to be the case then drawing the distinction between our ‘new 1’ and our ‘preconceived 1’ will have been essential.

With this point cleared, I would like to provide a few more definitions:

\begin{align*} 2 &= SS0 \\ 3 &= SSS0 \\ 4 &= SSSS0 \\ &\cdots \end{align*}

Similarly to how we proved that 1 is a natural number, we can prove that all these new definitions are also natural numbers.

In the next part, we’ll continue our look at the axioms, focusing on the fifth and most subtle of them.

To make my meaning clearer, let me ask another question. What is a car? If you asked this to the average person, they would, after giving you some funny looks, tell you about how it's a machine that can be used to get from one place to another quickly. After further interrogation, they may tell you about how it's controlled by turning a wheel and pushing a series of pedals. If you asked a mechanic what a car is, you would still get a rather funny look, but afterwards you may well get a very different response, and learn about internal combustion engines, suspension geometry, and fuel mixes.

While neither of these people are wrong, they describe wholly different aspects of what it means to be a car. To be clear, I don't intend to suggest that either point of view is better or more correct; it takes many brilliant engineers to build a Formula 1 car, but they need Lewis Hamilton to get it across the finish line. Equally, Lewis Hamilton is unlikely to be able to build a world class race car.

To start to see this new perspective on maths, let's use the decimal system as an example. We all use this system to write numbers, but how do we know it works?

The Decimal System

For simplicity, we shall only think about how to write non-negative, whole numbers (also called natural numbers) for now. To recap, what do we mean when we write a number such as 123? When first learning this system, we often would have labelled the digits, from right to left, as ones, tens, and hundreds. This would mean that we can read 123 as 3 ones, 2 tens, and 1 hundred (or, slightly more symbolically \( ( 3 \times 1 ) + ( 2 \times 10 ) + ( 1 \times 100 ) \)).

This system begs the question, can every non-negative whole number be written this way, or do we miss some out? If we start with a number such as 123, we can first divide by 10 to get 12 with remainder 3, so our ones column is 3. Then, 12 divided by 10 is 1 with remainder 2, so our tens column is 2. Next, 1 divided by 10 is 0 with remainder 1, so our hundreds column is 1. We have now reached 0 so we can stop, and see that our answer is, somewhat unsurprisingly, 123. This example seems somewhat contrived as we started with a number already in decimal form. However, we can see that the method does not rely on this; we could still use it if our starting number were instead written in a different form such as a tally.

To make this argument slightly more formal, we should provide it in a general form. This means that rather than starting with a specific number such as 123, we should start with an unknown number and give it a name. In this argument, we shall call our number \( n \). This style of argument makes it easier to ensure that we don't accidentally use some property that is unique to a specific number and only use universal properties of numbers. It also has the added bonus of making us sound a bit more fancy.

We can now start by dividing \( n \) by ten—calling the result \( n^\prime \) and putting the remainder into the ones column. Now, let's divide \( n^\prime \) by ten—calling the result \( n^{\prime\prime} \) and putting the remainder into the tens column. Repeat until you reach 0 and you've successfully converted \( n \) into a decimal number!

The Flaw in all Arguments

I would now like to pose another question: is this proof correct? While I certainly believe it is, what might a deeply pedantic cynic say? They may say one of several things:

• how do you know that you can divide any number by 10;
• how do you know that the remainder is always a single digit (i.e. between 0 and 9);
• how do you know that the division process will always reach 0?

Upon hearing these complaints, there are a few things we could do. We could provide an argument for each of those criticisms. Alternatively, we could simply claim that it is obvious any number can be divided by 10, so no elaboration is needed. It turns out that each of these options are fraught with danger!

If we choose to provide further arguments, they will rely on various other facts being true. Each of those other facts must also have arguments justifying their use, or we claim those facts are obvious. However, whenever we claim something to be obvious, we risk the possibility of being wrong.

To demonstrate that this is a real concern, we can consider a real example. I won't go too deeply into it, but during the development of set theory, it was believed that any collection of sets could themselves be contained within a set. For example, there is a set containing all sets. This seems like a totally reasonable belief until you consider Russell's paradox. He noticed that some sets, such as the set of all sets, contain themselves. Equally, some sets, such as the empty set, do not. We can now consider a set, \( R \), that is the set of all sets not containing themselves. If you ask whether \( R \) contains itself, you will find a troubling contradiction. The only way to resolve this is to reconsider the very fundamentals of set theory!

The consequence of these concerns is that we can never finish our argument; whatever we produce lacks the detail needed to be truly watertight. Descartes was a mathematician and philosopher and, in order to come up with his most well known quote, he took these concerns to their logical extreme. He asked whether we can know anything at all, even questioning whether we can trust our own senses. After all, what if everything we experience is just an illusion, as in ‘The Matrix’ perhaps? Eventually, Descartes realised there is something we can know with absolute confidence: “I think, therefore I am”. In other words, even if we are not willing to trust our own senses, we at least know that we exist.

Resolving Descartes’ Dilemma

I shall not discuss the philosophical resolutions to this problem here, but I shall discuss the mathematical resolution. It is surprisingly simple: take a small number of facts (which we shall call axioms) and believe them on faith alone! From these axioms, we can use logical arguments to prove whatever other facts we need, so long as those other facts really do follow from the axioms.

It should be immediately clear, however, that the choice of axioms is rather important. First of all, if one of our axioms is actually wrong then everything we claim to have proven is likely also nonsense. This means that all of our axioms should be simple enough that they are not in any doubt. Meeting this requirement is also rather useful if you want to convince someone else that you're argument is sound. Another requirement of our axioms, is that they should be ‘powerful’ enough that we can prove whatever we need; it would be frustrating to be unable to complete a proof because we were missing some crucial axiom.

In the following part, we will begin to explore a set of axioms that aim to describe Number Theory, the study of non-negative whole numbers (i.e. 0, 1, 2, 3, …).