SharePoint Knowledge Sharing.
InfoPath Web Service Connectivity Issue.
This post covers everything I’ve experienced about InfoPath Web Service connectivity issues.
After a recent load balancer update to distribute a building’s user requests evenly among two WFEs, we received reports from users that they receive intermittent error messages when they open InfoPath forms that are connected to web services. After gathering information regarding the error, we noticed two type of errors: 5566 and 7893. Going through the ULS logs, I noticed a 401 error message in the logs after each time the form load request was generated.
After reading and reading on the web and collecting as much evidence available, I started checking on certain things (Steps for each provided in bullet points):
1- I made sure the DisableLoopBackCheck is set to 1 in registry. Though this is for environments using NTLM, and my environment is using Kerberos, but I set it up anyways (per Microsoft’s recommendation):
Click Start , click Run , type regedit, and then click OK . In Registry Editor, locate and then click the following registry key:
2- I made sure the BackConnectionHostNames value in the registry is set with the correct URLs.
Click Start , click Run , type regedit, and then click OK . In Registry Editor, locate and then click the following registry key:
3- I made sure all the correct Service Principle Names are set on my service account for my Kerberos authentication.
You can check for duplicates by running SetSPN -X You can check for the list of SPNs set on your service account by running SetSPN -l <DOMAIN>\<ServiceAccount> Keep in mind that for each Web Application URL you’d need to have an SPN setup on the service account. For more information try here which is a great source for Kerberos implementation for SharePoint.
After doing all this, I was still seeing the issue. I started looking more into the network aspect of the issue. After tracing the packets with my network monitoring tool, I realized that the user request goes to the HLB (Hardware Load Balancer) and picks one WFE, the WFE makes the Web Service call to the load balancer to retrieve the information requested on the form.
Bingo! That was the wake up call. When the WFE makes the web service call to the load balancer, whenever the packet is distributed to the other WFE, I receive a 401. So all I needed to do was to keep the packet within the same WFE. Simply adding a host file entry on the WFE to point all the Web Applications to the IP address of the server should work. But in my case, all my sites are hosted on SSL through the load balancer. Meaning that the WFEs do not host any 443 sites. So here is what I did:
1- Grabbed the certificates from the HLB and imported them into each WFE’s certificate store.
After copying the certs into the server, click on Start> Run Type in mmc and from File>Add Snap in select Certificates Click the Computer Account radio button and click Next and then Finish Expand the Personal container and right click on Certificate. Click on All Tasks>Import and point to the certificate Repeat this for each cert.
2- Added 443 binding to each WFE’s each web application on IIS with the host header matching the URL of the web site.
appcmd set site /site.name:”<IIS Site Name>” /+bindings.[protocol=’https’,bindingInformation=’*:443:<Site URL>’]
Note: <IIS Site Name> should match the Web Application name on IIS, <Site URL> should match the URL of the site. e.g. www.contoso.com.
3- Adding the certificate to the site.
Ran the following script to obtain the hash of the certificate.
Certutil –store my.
Note: this will give you a list of certs in your cert store. Find the one for each web app accordingly and copy the hash. Make sure to clear out all spaces.
Ran the following script to apply the cert into IIS sites that have the 443 binding.
netsh http add sslcert ipport=0.0.0.0:443 certhash=<CertHash> appid=
Note: <CertHash> should be replaced with the cert hash that we collected before. appid section should only have the format, so you can use the same statement above for this section.
4- Last but not least, open the host file on each WFE (c:\Windows\System32\driver\etc\hosts) and add a host entry to point each web app to the IP of the WFE accordingly.
If WFE 1 has the IP address of 11.22.33.44 and the SharePoint web app address is contoso.com add the following entry on WFE 1:
If WFE 2 has the IP address of 55.66.77.88 add the following entry on WFE 2:
Boom! the problem was solved. No more authentication issues. Still for the 7893 which is essentially a time out issue, i had to go into SharePoint Central Admin> General Application Settings> Configure InfoPath Forms Service. Play with the timeout to adjust it according to your environment.
Hope this is helpful to you all.
Update: Please be aware that the best way to capture these types of certificate related issues is to enable CAPI2 logs and capture related information. You can find more information regarding that at this post from Benjamin Perkins.