Crawl Sharepoint 2013 site as a regular web site

Question

I've created a "branding" site with Sharepoint 2013. I'm not being able to configure the search engine so that it finds the content from the custom web parts. Most "static" content (institutional content) from the web site are displayed through custom web parts.

So my approach now is to try to configure the Content Search with "Web site" type, in order to have the crawl processing all pages as a regular website. But it's not working, it seems to keep crawling as a Sharepoint site, because the content from webparts are not being "searchable", even if I remove the "MicrosoftSharePointTeamServices" http header.

So, how could I configure the search in order to have the contents from web parts "searchable"?

Answer 1

Finally found the answer!

All webparts are rendered with class="noindex" by default, which causes crawl process to ignore it.

To solve the issue: go to "Site Settings" -> "Search and offline availability" and for "Indexing ASPX Page Content" configuration select "Always index all Web Parts on this site".

This will make sharepoint do not include "noindex" class anymore.

If there are any subsites, the configuration must be done at subsite leve as well.