In theory, there's nothing difficult here. Storing selectionIndexes is no different from storing any other NSIndexSet. You just need to make an entity with a Transformable property called, maybe selectedRows, and then put the value of arrayController.selectionIndexes in there right before you save the file, and read it out right after reading the document in. You don't need to worry about anything else, CoreData will just handle it. If you don't already have a logical "root" element to store this on, just make a new entity, perhaps called UIState with such a property on it and store it there.
The way I see it, you probably don't actually want to bind this value to that entity, for a couple reasons. The first is that you probably don't want changing the selection alone to dirty the document, right? I mean, maybe you do, but that sounds like more of an annoyance than a feature, and saving the selection, in general, sounds more like a convenience than a charter feature. But I don't know your application, so it's hard to say.
The other reason is it looks like if you bind arrayController.selectionIndexes and arrayController.contentArray both in IB, when it binds the contentArray binding, it looks like it will write out an initial value for selectionIndexes right after contentArray is bound when the doc opens, clobbering your real value. (I can imagine it being something like, 'You just set the content, so any previous value for the selection is invalid now, let's overwrite it with nothing!') So, at the very least, if you bound the value, you'd have to figure out some clever way to avoid having your real value clobbered by this initial write.
It seems to me like you'd be better off just not binding this directly to the model, but rather binding it to a transient property on the document which you can read from at save time, and write to at doc open time. That's where I'd go with this.