2
votes

I am using a Filter do generate dynamicly content to be visible for webcrawlers (https://developers.google.com/webmasters/ajax-crawling/docs/specification). This filter is working fine if the incoming url contains a path (http://www.unclestock.com/app.jsp#!s=GOOG). If the incoming url contains just my domain (and a fragment), say http://www.unclestock.com#!s=GOOG, the welcome file (app.jsp) is returned, but the filter is not honnoured.

My web.xml contains the following filter map:

<filter-mapping>
 <filter-name>crawler</filter-name>
 <url-pattern>/app.jsp</url-pattern>
</filter-mapping>

<welcome-file-list>
 <welcome-file>app.jsp</welcome-file>
</welcome-file-list>

I have tried to use an index.html welcome file instead, which redirects to app.jsp. The filter is then executed. However, this does not solve my problem: A client side redirect is not followed by the crawlers (which is the idea), and with server side redirect, I would loose my url fragment (which I also need).

Do you see any alternative solution?

I'm using Google Appengine.

1

1 Answers

1
votes

I solved it by using a welcome servlet which does a RequestDispatcher forward. Note that the dispatcher FORWARD must be added to the filter-mapping in order to have the filter working during the foward.

web.xml:
<filter-mapping>
  <filter-name>crawler</filter-name>
  <url-pattern>*.jsp</url-pattern>
  <dispatcher>REQUEST</dispatcher> 
  <dispatcher>FORWARD</dispatcher>
</filter-mapping>

<welcome-file-list>
  <welcome-file>welcome</welcome-file>
</welcome-file-list>

<servlet>
  <servlet-name>welcome</servlet-name>
  <servlet-class>Welcome</servlet-class>
</servlet>
<servlet-mapping>
  <servlet-name>welcome</servlet-name>
  <url-pattern>/welcome</url-pattern>
</servlet-mapping>

Welcome.java:
public class Welcome extends RemoteServiceServlet {
  public void doGet(HttpServletRequest req, HttpServletResponse resp) throws ServletException, IOException {
    RequestDispatcher rd = req.getRequestDispatcher("app.jsp");
    rd.forward(req, resp);
  }
}