You are right that this is probably not a document based application, as they open documents in new windows by default.
To layout the window like that, there’d be an NSSplitView that contains the 3 panes. Each pane may optionally contain a view loaded from an NSViewController, which can help keep the code modularised, but it depends on what you’re trying to do if this is appropriate.
The left pane would be an NSOutlineView (a NSTableView subclass), the middle an NSTableView, but I’m not sure exactly how the right-hand side view would be created (lots of custom NSViews and other things, possibly WebView)
That popover options window is possibly a NSPopover (which contains an NSViewController), but that’s only compatible with OS X 10.7, so may also be totally custom for backwards compatibility and easier customisation.
Also note this is a fairly complicated example you’ve given, with lots of custom controls that are probably harder to create than they look:
To get the outline views on the left to have unread counts and icons (from memory) is not built into AppKit, so was all custom created. To do things like that, you’ll need a solid understanding of NSCell vs NSView, and ideally also know about Core Animation layer backed views, and what to use for different aspects.
The window has a taller-than usual title bar. This means the developer probably had to do some crazy stuff to get it to work, if not create the whole window from scratch.
That’s just the start. There’s lots of really nice design in there that’s custom and done from scratch.
Designing Mac apps can be hard sometimes. AppKit is pretty old (back from the NEXT days), and has lots of legacy stuck in it. UIKit on iOS on the other hand is quite nice – Apple clearly learned from their past and made things much better.
I’ve hardly touched on the controllers and model behind all that. There’s lots of different ways you could do it. For persistence, you could use CoreData, sqlite, NSKeyedArchived, just to name a few. Brent Simmons (past developer of another RSS reader, NetNewsWire) wrote some interesting blog posts about that:
http://inessential.com/2010/02/26/on_switching_away_from_core_data
http://inessential.com/2011/09/22/core_data_revisited
The way you design your model & controllers really depends on the specific problem. Cocoa really forces you to stick to MVC though – if you don’t, things are guaranteed to end up messy.
I hope that all helps! I’m really only just learning myself too.