1
votes

I deployed spring boot Rest application in azure tomcat app service. Here I copied application from my local tomcat webapps folder to azure tomcat webapps fodler using FTP( copied app folder structure, not war file).

Its working fine.. but after some time server is not responding and returning HTTP Error 502.3 - Bad Gateway.

What could be the reason for this error. Is there any issue with my spring configuration ? appreciate any help.

we have two kind of URI's, one health check URI and another one is to fetch the data from database.

When app service is down - only myapp/healthcheck is returning status code 200.

And remaining all requests which having uri - myapp/ssp/*** returning status code 502.3

When I restart the app service, every thing working, but after some time issue reoccurring.

I am suspecting problem in my AuthenticationFilter.java & RequestValidator. java & ControllerAdvice files

Here is spring boot configuration

here is my configuration. `@SpringBootApplication( exclude = HibernateJpaAutoConfiguration.class ) @ComponentScan( { "com.xx.ssp" , "com.microsoft.applicationinsights" } ) public class SSPApplication{

/**
 * @param args
 */
public static void main( String[ ] args ) {

    SpringApplication sspSpringBootApplciation = new SpringApplication( SSPApplication.class );
    sspSpringBootApplciation.addListeners( new ApplicationPidFileWriter( ) );
    sspSpringBootApplciation.run( args );
}

/**
 * @return
 */
@Bean
public WebSecurityConfigurerAdapter webSecurityConfigurerAdapter( ) {

    return new ApplicationSecurity( );
}

@Bean
public String telemetryConfig( ) {

    String telemetryKey = "zzzz";
    if( telemetryKey != null ){
        TelemetryConfiguration.getActive( ).setInstrumentationKey( telemetryKey );
    }
    return telemetryKey;
}

@Bean
public FilterRegistrationBean aiFilterRegistration( ) {

    FilterRegistrationBean registration = new FilterRegistrationBean( );
    registration.setFilter( webRequestTrackingFilter( ) );
    registration.addUrlPatterns( "/*" );
    registration.setOrder( 1 );
    return registration;
}

@Bean( name = "WebRequestTrackingFilter" )
public Filter webRequestTrackingFilter( ) {

    return new WebRequestTrackingFilter( );
}

@Bean
public ErrorPageFilter errorPageFilter( ) {

    return new ErrorPageFilter( );
}

@Bean
public FilterRegistrationBean disableSpringBootErrorFilter( ErrorPageFilter filter ) {

    FilterRegistrationBean filterRegistrationBean = new FilterRegistrationBean( );
    filterRegistrationBean.setFilter( filter );
    filterRegistrationBean.setEnabled( false );
    return filterRegistrationBean;
}

} `

`@ControllerAdvice public class RestEntityExceptionHandler extends ResponseEntityExceptionHandler{

@Autowired
private Environment environment;

private SSPLogger logger = new SSPLogger( getClass( ) );

/**
 * @return
 */
private String getEnvironment( ) {

    String[ ] activeProfiles = environment.getActiveProfiles( );
    if( activeProfiles != null && activeProfiles.length > 1 ){
        return SSPEnvironment.getEnvironment( activeProfiles[ 0 ] ).name( );
    }
    return SSPEnvironment.UNKNOWN.name( );

}

/**
 * @param errorCode
 * @return
 */
private HttpStatus getHttpstatusCode( String errorCode ) {

    try{
        return HttpStatus.valueOf( Integer.parseInt( errorCode ) );
    } catch ( Exception e ){

        return HttpStatus.INTERNAL_SERVER_ERROR;
    }
}

/**
 * @param ex
 * @return
 */
@ExceptionHandler( {    .class , IllegalStateException.class , SSPRestException.class ,
        Exception.class } )
@ResponseBody
ResponseEntity< Object > handleControllerException( Exception ex ) {

    HttpHeaders headers = new HttpHeaders( );
    headers.setContentType( MediaType.APPLICATION_JSON );
    ex.printStackTrace( );
    if( ex instanceof SSPRestException ){
        SSPRestException exp = ( SSPRestException ) ex;
        CustomErrorResponse customErrorResponse = new CustomErrorResponse( );
        customErrorResponse.setStatusCode( exp.getErrorCode( ) );
        customErrorResponse.setErrorMessage( exp.getErrorMessage( ) );
        customErrorResponse.setErrorDescription( exp.getErrorDescription( ) );
        customErrorResponse.setErrorKey( exp.getErrorKey( ) );
        customErrorResponse.setEnvironment( getEnvironment( ) );
        logger.logException( LogLevel.ERROR , customErrorResponse );

        HttpStatus httpstatusCode = getHttpstatusCode( exp.getErrorCode( ) );

        return new ResponseEntity<>( customErrorResponse , headers , httpstatusCode );
    }
    if( ex instanceof HttpRequestMethodNotSupportedException ){
        String statusCode = "405";
        headers.setContentType( MediaType.APPLICATION_JSON );
        CustomErrorResponse customErrorResponse = new CustomErrorResponse( );
        customErrorResponse.setStatusCode( statusCode );
        customErrorResponse.setErrorMessage( ex.getMessage( ) );
        customErrorResponse.setErrorKey( "" );
        customErrorResponse.setEnvironment( getEnvironment( ) );
        HttpStatus httpstatusCode = getHttpstatusCode( statusCode );
        return new ResponseEntity<>( customErrorResponse , headers , httpstatusCode );
    } else{
        String statusCode = "UNKNOWN";
        CustomErrorResponse customErrorResponse = new CustomErrorResponse( );
        customErrorResponse.setStatusCode( statusCode );
        customErrorResponse.setErrorMessage( ex.getMessage( ) );
        if( ex.getCause( ) != null ){
            customErrorResponse.setErrorDescription( ex.getCause( ).getMessage( ) );
        }
        customErrorResponse.setErrorKey( "UNAHNDLED_EXCEPTION" );
        customErrorResponse.setEnvironment( getEnvironment( ) );
        HttpStatus httpstatusCode = getHttpstatusCode( statusCode );
        logger.logException( LogLevel.ERROR , customErrorResponse );
        return new ResponseEntity<>( customErrorResponse , headers , httpstatusCode );
    }
}

}` ``` @Order( SecurityProperties.ACCESS_OVERRIDE_ORDER ) public class ApplicationSecurity extends WebSecurityConfigurerAdapter{

    @Override
    protected void configure( HttpSecurity http ) throws Exception {

    http.csrf( ).disable( ).sessionManagement( ).sessionCreationPolicy( SessionCreationPolicy.STATELESS );
    http.authorizeRequests( ).antMatchers( "/spp/**" ).addFilterBefore( new AuthenticationFilter( ) , SecurityContextPersistenceFilter.class );
http.authorizeRequests( ).antMatchers( "/healthcheck/**" , "/unoauth/**" ).permitAll( ).antMatchers( HttpMethod.OPTIONS , "/**" ).permitAll( );

```

`@Component public class RequestValidator extends HandlerInterceptorAdapter{

@Autowired
private AppConstants appConstants;

@Autowired
private CacheTemplate cacheTemplate;

@Autowired
private RestClient restClient;

/*
 * (non-Javadoc)
 * @see org.springframework.web.servlet.handler.HandlerInterceptorAdapter#
 * afterCompletion(javax.servlet.http.HttpServletRequest,
 * javax.servlet.http.HttpServletResponse, java.lang.Object,
 * java.lang.Exception)
 */
@Override
public void afterCompletion( HttpServletRequest request , HttpServletResponse response , Object object ,
                Exception arg3 ) throws Exception {

}

/**
 * @param cookies
 * @return
 */
private String getGsidFromRequest( Cookie[ ] cookies ) {

    String gsid = null;
    if( cookies != null ){
        Cookie gsidCookie = Arrays.stream( cookies ).filter( x-> x.getName( ).equals( "gsid" ) ).findFirst( )
                        .orElse( null );
        if( gsidCookie != null ){
            gsid = gsidCookie.getValue( );
        }
        if( Utils.isEmpty( gsid ) ){
            throw new SSPRestException( ErrorCode.InvalidRequest , "gsid is missing" , getClass( ) );
        }
    }
    return gsid;
}

/**
 * @return
 */
private UserCashedInfo getTempUserInfo( ) {

    UserCashedInfo cashedInfo = new UserCashedInfo( );
    cashedInfo.setEmailId( "sssssss" );
    cashedInfo.setUserId( 1 );
    cashedInfo.setUserRole( 1 );
    cashedInfo.setRefreshToken( "" );
    cashedInfo.setCaseInstanceId( 1 );
    cashedInfo.setUserName( "Test User" );
    return cashedInfo;
}

/**
 * @param accessToken
 * @return
 */
private boolean isRequestHadAccessToken( String accessToken ) {

    if( Utils.isEmpty( accessToken ) ){
        throw new SSPRestException( ErrorCode.InvalidRequest , "Access Token is Mandatory" , getClass( ) );
    }
    return true;
}

/*
 * (non-Javadoc)
 * @see org.springframework.web.servlet.handler.HandlerInterceptorAdapter#
 * postHandle(javax.servlet.http.HttpServletRequest,
 * javax.servlet.http.HttpServletResponse, java.lang.Object,
 * org.springframework.web.servlet.ModelAndView)
 */
@Override
public void postHandle( HttpServletRequest request , HttpServletResponse response , Object object ,
                ModelAndView model ) throws Exception {

}

/*
 * (non-Javadoc)
 * @see org.springframework.web.servlet.handler.HandlerInterceptorAdapter#
 * preHandle(javax.servlet.http.HttpServletRequest,
 * javax.servlet.http.HttpServletResponse, java.lang.Object)
 */
@Override
public boolean preHandle( HttpServletRequest request , HttpServletResponse response , Object object )
                throws Exception {

    boolean isValidRequest = false;

    String isOauthRequiredParam = request.getParameter( "isOauth" );

    String methodName = request.getMethod( );
    if( methodName != null && methodName.equalsIgnoreCase( "OPTIONS" ) ){
        isValidRequest = true;
    }

    if( !isValidRequest ){
        if( !Utils.isEmpty( isOauthRequiredParam ) && isOauthRequiredParam.equals( "false" ) ){
            isValidRequest = true;
            UserCashedInfo cashedInfo = getTempUserInfo( );
            Utils.updateUserLocalThread( cashedInfo );
        } else{
            String accessToken = request.getHeader( "oauth" );
            boolean isRequestHadAccessToken = isRequestHadAccessToken( accessToken );
            String gsid = getGsidFromRequest( request.getCookies( ) );
            if( isRequestHadAccessToken && !Utils.isEmpty( gsid ) ){
                UserCashedInfo cashedInfo = cacheTemplate.getItem( gsid );
                if( cashedInfo != null ){
                    String userEmail = validateAccessToken( accessToken );
                    if( userEmail != null && userEmail.equals( cashedInfo.getEmailId( ) ) ){
                        Utils.updateUserLocalThread( cashedInfo );
                        isValidRequest = true;
                    } else{
                        throw new SSPRestException( ErrorCode.UnAuthenticateUser , "" , getClass( ) );
                    }

                } else{
                    throw new SSPRestException( ErrorCode.UnAuthenticateUser , "" , getClass( ) );
                }
            } else{
                throw new SSPRestException( ErrorCode.GsidMissing , "" , getClass( ) );
            }
        }
    }
    return isValidRequest;
}

/**
 * @param accessToken
 * @return
 */
private String validateAccessToken( String accessToken ) {

    CustomeJsonResponse validateResponse = restClient.ValidateAcessToken( accessToken );
    if( validateResponse.getStatusCode( ) == 200 ){
        return ( ( ValidateAcessTokenResponse ) validateResponse.getData( ) ).getAccess_token( )
                        .get( appConstants.getUserEmail( ) );
    } else{
        throw new SSPRestException( ErrorCode.ApValidateAccessTokenError , "" , getClass( ) );
    }
}

} `

` public class AuthenticationFilter extends GenericFilter{

private SSPLogger logger = new SSPLogger( getClass( ) );

@Override
public void destroy( ) {

}

/*
 * (non-Javadoc)
 * @see com.xxx.ssp.security.GenericFilter#doFilter(javax.servlet.
 * ServletRequest, javax.servlet.ServletResponse, javax.servlet.FilterChain)
 */
@Override
public void doFilter( ServletRequest req , ServletResponse res , FilterChain chain )
                throws IOException , ServletException {

    HttpServletRequest request = ( HttpServletRequest ) req;

    String accessToken = request.getHeader( "oauth" );
    boolean isValidRequest = false;
    String isOauthRequiredParam = request.getParameter( "isOauth" );
    String methodName = request.getMethod( );
    if( methodName != null && methodName.equalsIgnoreCase( "OPTIONS" ) ){
        isValidRequest = true;
    } else{
        if( !Utils.isEmpty( isOauthRequiredParam ) && isOauthRequiredParam.equals( "false" ) ){
            isValidRequest = true;
        } else{
            logger.debug( "Requeste validation required : true" );

            boolean isRequestHadAccessToken = isRequestHadAccessToken( accessToken );
            String gsid = getGsidFromRequest( request.getCookies( ) );

            if( isRequestHadAccessToken && !Utils.isEmpty( gsid ) ){
                isValidRequest = true;
            } else{
                throw new SSPRestException( ErrorCode.InternalServerError ,
                                " Access Token or GSID is mssing in the request" , getClass( ) );
            }
        }
    }
    if( isValidRequest ){
        chain.doFilter( req , res );
    } else{
        throw new SSPRestException( ErrorCode.UnauthorisedRequest , "Invalid User" , getClass( ) );
    }
}

/**
 * @param cookies
 * @return
 */
private String getGsidFromRequest( Cookie[ ] cookies ) {

    String gsid = null;
    if( cookies != null ){
        Cookie gsidCookie = Arrays.stream( cookies ).filter( x-> x.getName( ).equals( "gsid" ) ).findFirst( )
                        .orElse( null );
        if( gsidCookie != null ){
            gsid = gsidCookie.getValue( );
        }
        if( Utils.isEmpty( gsid ) ){
            throw new SSPRestException( ErrorCode.UnAuthenticateUser , "gsid is missing" , getClass( ) );
        }
    }
    return gsid;
}

/*
 * (non-Javadoc)
 * @see
 * com.xxx.ssp.security.GenericFilter#init(javax.servlet.FilterConfig)
 */
@Override
public void init( FilterConfig arg0 ) throws ServletException {

}

/**
 * @param accessToken
 * @return
 */
private boolean isRequestHadAccessToken( String accessToken ) {

    if( Utils.isEmpty( accessToken ) ){
        throw new SSPRestException( ErrorCode.UnAuthenticateUser , "Access Token is Mandatory" , getClass( ) );
    }
    return true;
}

}`

`@Component public class SSPCORSFilter implements Filter{

private SSPLogger sSPLogger = null;

public SSPCORSFilter( ){

    sSPLogger = new SSPLogger( getClass( ) );
    sSPLogger.info( "CORSFilter initialized" );
}

@Override
public void destroy( ) {

}

@Override
public void doFilter( ServletRequest req , ServletResponse res , FilterChain chain )
                throws IOException , ServletException {

    String origin = ( ( HttpServletRequest ) req ).getHeader( "Origin" );
    sSPLogger.info( "CORSFilter Request origin : " + origin );
    HttpServletResponse response = ( HttpServletResponse ) res;
    response.setHeader( "Access-Control-Allow-Origin" , origin );
    response.setHeader( "Access-Control-Allow-Credentials" , "true" );
    response.setHeader( "Access-Control-Allow-Methods" , "GET, POST, PUT, DELETE, OPTIONS" );
    // response.setHeader( "Access-Control-Allow-Headers" , "Content-Type,
    // Accept, X-Requested-With, oauth" );
    response.setHeader( "Access-Control-Request-Headers" ,
                    "Access-Control-Allow-Headers, Origin,Accept, X-Requested-With, Content-Type, Access-Control-Request-Method, Access-Control-Request-Headers, oauth, access-control-allow-credentials" );
    response.setHeader( "Access-Control-Allow-Headers" ,
                    "Access-Control-Allow-Headers, Origin,Accept, X-Requested-With, Content-Type, Access-Control-Request-Method, Access-Control-Request-Headers, oauth, access-control-allow-credentials" );

    chain.doFilter( req , res );
}

@Override
public void init( FilterConfig filterConfig ) {

}

}`

1
Please post more details about your error log on KUDU? - Jay Gong
Sorry for the late response. I fixed this issue. The root cause is dead lock occurred at db connection pool. So my request not getting DB connection.. after time out IIS server returning my request with status as "http-error-502-3-bad-gateway" - Babu Gali
Great! Thank you for your sharing.Please mark your answer to end this case later. - Jay Gong

1 Answers

2
votes

I fixed this issue after looking at thread dump. The root cause is dead lock occurred at db connection pool. So my request not getting DB connection.. after time out IIS server returning my request with status as "http-error-502-3-bad-gateway"