Localstack with MassTransit not getting messages

Question

I am having issues testing MassTransit with LocalStack, but with the real SNS/SQS in AWS everything works fine, so I suspect it's an issue with LocalStack unless MassTransit requires something else than configuring ServiceURL. See https://github.com/MassTransit/MassTransit/issues/1476

I run LocalStack as following, just with SNS and SQS

docker run -it -e SERVICES=sns,sqs -e TEST_AWS_ACCOUNT_ID="000000000000" -e DEFAULT_REGION="us-east-1" -e LOCALSTACK_HOSTNAME="localhost" -e -rm --privileged --name localstack_main -p 4566:4566 -p 4571:4571 -p 8080-8081:8080-8081  -v "/tmp/localstack:/tmp/localstack" -v "/var/run/docker.sock:/var/run/docker.sock" -e DOCKER_HOST="unix:///var/run/docker.sock" -e HOST_TMP_FOLDER="/tmp/localstack" "localstack/localstack"

Now with MassTransit I create the bus and start it. The only change I make for MassTransit to work with LocalStack is setting the ServiceURL in SNS and SQS. The rest should work in the same way (I think)

var region = "localhost:4566";
var accessKey = "test";
var secretKey = "test";
var busControl = Bus.Factory.CreateUsingAmazonSqs(c =>
{
    var hostAddress = new Uri($"amazonsqs://{region}")
    var hostConfigurator = new AmazonSqsHostConfigurator(hostAddress);
    hostConfigurator.AccessKey(accessKey);
    hostConfigurator.SecretKey(secretKey);
    hostConfigurator.Config(new AmazonSimpleNotificationServiceConfig {ServiceURL = $"http://{region}"});
    hostConfigurator.Config(new AmazonSQSConfig {ServiceURL = $"http://{region}"});
    c.Host(hostConfigurator.Settings);
});

When running my project I can connect and publish events, no errors. I subscribe to events, no errors. I can see the topics, subscriptions and queue are properly created in LocalStack.

I can also see with Commandeer that there are "invisible" messages in the queue (not sure what that is) so it seems the problem is in the receiving part.

Is there anything additional requirement to configure in MassTransit to consume published events?

UPDATE 1: One interesting thing is that I can keep the subscriber listening for long time and during this time Commandeer shows there are invisible messages in the queue. As soon as I stop the subscriber (and my application) I can see that Commandeer moves messages from "invisible" to "messages". Cannot peek messages though.

Just to chime in, I run all the tests and samples locally using the (older) docker image: localstack/localstack latest 86200da6e9fa 10 months ago 880MB — Chris Patterson
I'll try to run an old image from that time then, thanks. If I'm not missing any additional config other than service url in consumer then it must be some thing with localstack. Will probably switch to in memory transport for automated tests. Ta. — diegosasw
Confirmed, Thanks for your help Chris, you are right. I've tried with an older image and it works well. Not the one you mentioned though, couldn't find that, but this is good enough. I'll answer my own question then and report the bug to localstack — diegosasw

diegosasw diegosasw · Accepted Answer · 2021-03-23T20:03:25

I've confirmed the problem is with localstack latest image as I've tried with an older one, as per Chris' suggestion, and it works well.

With localstack/localstack:0.11.2 it works well

docker run -it -e SERVICES=sns,sqs -e TEST_AWS_ACCOUNT_ID="000000000000" -e DEFAULT_REGION="us-east-1" -e LOCALSTACK_HOSTNAME="localhost" -e -rm --privileged --name localstack_main -p 4566:4566 -p 4571:4571 -p 8080-8081:8080-8081  -v "/tmp/localstack:/tmp/localstack" -v "/var/run/docker.sock:/var/run/docker.sock" -e DOCKER_HOST="unix:///var/run/docker.sock" -e HOST_TMP_FOLDER="/tmp/localstack" "localstack/localstack:0.11.2"

With latest (I think it was bdfbe53666a4dd13a09dd9e4b155e2fb750b8041daf7efc69783cb4208b6cacc but not a 100% sure) it doesn't work.

The following image versions don't work either:

localstack/localstack:0.12.8
localstack/localstack:0.11.6

UPDATE 1: I've created a simple repo with instructions to reproduce the issue https://gitlab.com/sunnyatticsoftware/sandbox/issue-localstack-masstransit

Notice the repo has a wrapper abstraction over MassTransit to use with a clean architecture. This doesn't affect the issue, but it was easier to copy-paste the needed parts rather than building up a sample from scratch.

UPDATE 2: Verified that the latest version localstack/localstack:0.12.9.1 works well for above scenario (see repo)

UPDATE 3 (2021-01-12): I attempted the version localstack/localstack:0.12.9.1 again and it does not work. Not sure if the previous time it really worked or the docker image was overwritten. In any case I'm back at using the version localstack/localstack:0.11.2 again, because the latest is also broken, unfortunately.

I can see the messages in the queue as hidden.

awslocal sqs get-queue-attributes --queue-url http://localhost:4566/000000000000/sample-queue --attribute-names All
{
    "Attributes": {
        "ApproximateNumberOfMessages": "0",
        "ApproximateNumberOfMessagesDelayed": "0",
        "ApproximateNumberOfMessagesNotVisible": "4",
        "CreatedTimestamp": "1626087449.988218",
        "DelaySeconds": "0",
        "LastModifiedTimestamp": "1626087450.113652",
        "MaximumMessageSize": "262144",
        "MessageRetentionPeriod": "345600",
        "QueueArn": "arn:aws:sqs:us-east-1:000000000000:sample-queue",
        "Policy": "{\"Version\": \"2012-10-17\", \"Statement\": [{\"Sid\": \"0d948ac2a9ea4ed7b2c0609642107f0f\", \"Effect\": \"Allow\", \"Principal\": {\"AWS\": \"*\"}, \"Action\": \"sqs:SendMessage\", \"Resource\": \"arn:aws:sqs:us-east-1:000000000000:sample-queue\", \"Condition\": {\"ArnLike\": {\"aws:SourceArn\": \"arn:aws:sns:us-east-1:000000000000:*\"}}}]}",
        "ReceiveMessageWaitTimeSeconds": "0",
        "VisibilityTimeout": "30"
    }
}

Localstack with MassTransit not getting messages

1 Answers