I have a problem that consumed long hours from me without a viable solution. I have no experience on Sharepoint and it is possible that I am overdoing it, I would not be surprised if the solution is much simpler than I expected.
The scenario is as the following:
1) We have a SharePoint installation at Client (A). Client (A) environment consists of testing and production environments. The production environment a separate server for each WFE and for BEDS. The testing environment consists of WFE & BEDS in a single server. Both environments are on the same domain.
2) Sharepoint version is 2013.
3) Client (A) established two-way trust relation with Client (B) domain.
The problem: The production Sharepoint PeoplePicker can successfully grab users from Domain (B) with no configuration done at PeoplePicker at all. However, the testing Sharepoint PeoplePicker fails to grab domain (B) trusted users. And fails with "Cannot find exact match" error.
I tried the following solutions on the testing environment (where WFE & BEDS are colocated) to locate the issue:
1- Checking all PeoplePicker relevant properties (Peoplepicker-searchadcustomquery, Peoplepicker-onlysearchwithinsitecollection, Peoplepicker-searchadforests, setsiteuseraccountdirectorypath,etc.) and nothing at all. Also, I ran people picker port tester on both environments (production & testing) to compare any firewall configuration differences, even though some ports are complaining and differences between UDP ports (445,135) exists, still, I dont think it is the problem as Wireshark step (3) later will tell why it is not a possibility. From what I read on the internet, I have to configure nothing for people picker in two way trust scenario, still tried to do one way trust configuration, nothing works & I reverted the changes.
PeoplePicker port tester run results
2- Configured Sharepoint to verbose logging (all components since I do not know which is responsible for PeoplePicker) collected logs and searched the logs for the queried user. No useful information, a screenshot is attached from ULS Viewer, the user is masked.
3- Collected Wireshark traffic dump and filtered by LDAP. I can clearly the that the LDAP response contains the queried user from the trusted along with all user attributes and domain name. That is why I excluded any people-picker filtration reasons ,domain search limitations or network ports issues. Screenshot is attached.
4- After excluding the network issue (as LDAP query returns to WFE successfully), I decided to see how the flow inside Sharepoint goes before showing the result in PeoplePicker. If found this article from Microsoft, describing PeoplePicker workflow. From the diagram below, I can conclude that the LDAP request successfully passed steps 1,2 and 3. I need to check steps from 4 to 11 which are MS-WSSFO protocol handshakes. I read about the protocol, and found it consists of group of stored procedure calls from WFE to BEDS. I tried to debug the protocol by using SQL Server Profile and searching for (proc_GetTpWebMetadataAndListMetadata) & (proc_GetListMetadataAndEventReceivers) stored procedures but nothing found.
5- A friend of mine suggested configuring UPSA (User Profile Service Application) connection to the trusted domain (Domain B) active directory, I granted Sharepoint service users the required permissions on (Domain B) active directory, configured the connections and tested the synchronization and it does load users as I can see from Synchronization Service Manager, however, PeoplePicker still fails. However, I think this step was unnecessary as PeoplePicker and UPS are not related as far as I know, and since Production Sharepoint PeoplePicker is working fine without configuring UPSA. Any help please?
