Firewalls in general can be stateful or stateless.
Stateful firewalls keep track of connections. If you establish a connection from local port 12345 to remote port 80, your stateful firewall will allow return traffic through port 12345 for that connection.
Stateless firewalls treat each and every packet independently. That's why they're called stateless - they maintain no state. You'd have to specifically open port 12345 to inbound traffic for the above example to work. The challenge this introduces is that the ports your apps use for outbound connections are arbitrary (they're not known in advance).
Stateless firewalls are generally more efficient because they are very simple, don't have to maintain state, and only look at the packet header.
In the specific case of AWS, they offer both types of firewall:
- Security Groups which are stateful
- Network Access Control Lists which are stateless