The easiest way to do this is to create multiple ECS services. You can create these in the same cluster or in different clusters (it really depends on how you want to set up your infrastructure). Put the service for container B behind an application load balancer and then let the load balancer handle connecting to individual containers during requests from container A. To do this in CloudFormation, you'll need to create the following CloudFormation resource types:
AWS::ECS::Service
AWS::ElasticLoadBalancingV2::LoadBalancer
AWS::ElasticLoadBalancingV2::TargetGroup
AWS::ElasticLoadBalancingV2::Listener
Basically, you'll add each of those resource types to your CloudFormation template. The resources will reference each other in these ways:
- AWS::ECS::Service -> ARN of TargetGroup (in LoadBalancers property)
- AWS::ElasticLoadBalancingV2::Listener -> ARN of LoadBalancer (LoadBalancerArn property), ARN of TargetGroup (in DefaultActions property)
Note that you can make the AWS::ElasticLoadBalancingV2::LoadBalancer resource internal or internet-facing using the "Scheme" property. If you want to keep service B private, you just set this to "internal".
Of course, you'll also need to add various resource types to build out the rest of your ECS cluster/service but I'm assuming you already have this part in your template.
Once you deploy the CloudFormation stack changes, the load balancer will have a URL (you can add this to the outputs of the stack by using "Fn::GetAtt": ["LogicalIDOfLBResource", "DNSName"] - see the "DNSName" return value in the docs here). When you make requests to this URL, the load balancer will automatically connect to any of the B containers and return the response from the container, so you can use this URL in the A containers to connect from A to B.