Does adding enumerators into enum break ABI?

Question

In particular, i got following code in library interface:

typedef enum
{
    state1,
    state2,
    state3,
    state4,
    state5,
    state_error = -1,
} State;

I strictly forbidden to break ABI. However, I want to add state6 and state7. Will it break ABI?

I found here some tip, but i somewhat doubt if it`s my case?

You can...

append new enumerators to an existing enum.

Exeption: if that leads to the compiler choosing a larger underlying type for the enum,that makes the change binary-incompatible. Unfortunately, compilers have some leeway to choose the underlying type, so from an API-design perspective it's recommended to add a Max.... enumerator with an explicit large value (=255, =1<<15, etc) to create an interval of numeric enumerator values that is guaranteed to fit into the chosen underlying type, whatever that may be.

Adding a new enum while leaving existing ones alone shouldn't break ABI compatibility as long as the size of the underlying type doesn't change as a result of the additions. Which is what your quoted material says. — Jonathan Leffler

mity mity · Accepted Answer · 2014-12-04T18:01:59

Your question is a nice example why long-term maintaining of ABI compatibility is a difficult task. The core of the problem here is that the compatibility depends not just on the given type, but also on how it is used in function/method prototypes or complex types (e.g. structures, unions etc.).

(1) If the enumeration is used strictly as an input into the library (e.g. as a parameter of a function which just changes of behavior the function/library), then it keeps the compatibility: You changed the contract in a way which can never hurt the customer i.e. the calling application. Old applications shall never use the new value and will get the old behavior, new applications just get more options.

(2) If the enumeration is used anywhere as an output from the library (e.g. return value or function filling some address provided by caller aka an output parameter), the change would break the ABI. Consider the enumeration to be a contract saying "the application never sees values other then those listed". Adding new enum member would break this contract because old applications could now see values they never counted with.

That is at least, if there are no measures to protect old applications from falling into these troubles. Generally speaking, the library still can output the new value, but never for any valid input potentially provided by the old applications.

There are some design patterns allowing such enum expansions:

E.g. the library can provide an initialization function which allows to specify version of ABI the application is ready for. Old application ask for version 1.0 and never get the new value on input; newer application specify 1.1. or 2.0 or if the new enum value as added in the version 1.1, and then it may get the new value.)

Or, if a function DoSomething() is getting some flags on input, you may add a new flag where application can specify it's ready to see the new output value.

Or, if that's not possible, new version of the library may add a new function DoSomethingEx() which provides the more complex behavior than the original DoSomething(). DoSomethingEx() now can return the new enum value, the DoSomething() cannot.

As a side note if you ever need to add such DoSomethingEx(), do it in a way that allows similar expansions in the future. For consistency, it's usually a good idea to design it so that DoSomethingEx() with default flags (usually zero) behaves the same way DoSomething() and only with some new flag(s) it offers a different and more complex behavior.

Drawback of course is that the library implementation has to check what the application is ready for and provide a behavior compatible for expectations of old applications. It does not seem as much but over time and many versions of the library, there may be dozens of such checks accumulated in the library implementation, making it more complex and harder to maintain.

Does adding enumerators into enum break ABI?

3 Answers