Factory & Repository Concepts
To answer your question, I think it's important to focus on responsibilities of the concepts defined by DDD.
In the blue book, there is a section that deals with the problem that you describe:
A FACTORY handles the beginning of an object’s life; a REPOSITORY helps manage the middle and the end.
and specifically for your question:
Because the REPOSITORY is, in this case, creating objects based on data, many people consider the REPOSITORY to be a FACTORY—indeed it is, from a technical point of view.
(both quotes from Evans, chapter 6, section "The relationship with factories")
To keep the concepts pure, it is important that the interface of your factories and repositories are clean. So don't allow creating new business objects through the repository interface, and don't allow querying existing ones through the factory interface.
Keeping the interfaces clean does however not mean that you should not use a factory from the repository implementation, because, after all, the repository creates an instance at some point, and if that instance creation is complex, a factory is the appropriate solution.
To quote Evans again:
Whenever there is exposed complexity in reconstituting an object from another medium, the FACTORY is a good option.
Note, however, that the repository will most likely call a different method on the factory than the clients that really want to create a new domain object (as opposed to reconstitution).
There is even an example in Evans' book that illustrates approach:
Answer to your question
Now that it is clear that this is allowed, lets focus on your question of where to put the factory:
The DDD factory interface belongs in the domain, because your domain logic uses this to create domain objects.
The DDD reconstitution factory interface does not belong in the domain, since this is only relevant for your repository. It does not exist in the real world of your domain.
Now if you're using an architecture that prohibits dependencies from the domain to the infrastructure (which you probably should when applying DDD), it's clear that the factory implementation belongs in the infrastructure. Note that it does not matter whether you call your layers layers, rings, realms or whatever, the dependencies are the important part.