2
votes

I have the following problem with WSO2 API Manager and I suspect that it could be a problem related to certificate of the final HTTPS endpoint that it was registered.

I try to explain my situation in details:

First thing: I changed this section of the repository/conf/axis2/axis2.xml file in order to contact the 443 port instead the default 8243 port to call the registered endpoint over HTTPS (at the moment I can't change the registered end point port and I can't install a reverse proxy, but I have to test if the system works as expected, basically I need to call the final endpoint on the 443 port and I have to obtain the JSON response).

The original section that I changed is:

<transportReceiver name="https" class="org.apache.synapse.transport.passthru.PassThroughHttpSSLListener">
    <parameter name="port" locked="false">8243</parameter>
    <parameter name="non-blocking" locked="false">true</parameter>
    <!--parameter name="bind-address" locked="false">hostname or IP address</parameter-->
    <!--parameter name="WSDLEPRPrefix" locked="false">https://apachehost:port/somepath</parameter-->
   <parameter name="httpGetProcessor" locked="false">org.wso2.carbon.mediation.transport.handlers.PassThroughNHttpGetProcessor</parameter>
    <parameter name="keystore" locked="false">
        <KeyStore>
            <Location>repository/resources/security/wso2carbon.jks</Location>
            <Type>JKS</Type>
            <Password>wso2carbon</Password>
            <KeyPassword>wso2carbon</KeyPassword>
        </KeyStore>
    </parameter>
    <parameter name="truststore" locked="false">
        <TrustStore>
            <Location>repository/resources/security/client-truststore.jks</Location>
            <Type>JKS</Type>
            <Password>wso2carbon</Password>
        </TrustStore>
    </parameter>
    <!--<parameter name="SSLVerifyClient">require</parameter>
        supports optional|require or defaults to none -->
</transportReceiver>

I changed it in this way:

<transportReceiver name="https" class="org.apache.synapse.transport.passthru.PassThroughHttpSSLListener">
    <parameter name="port" locked="false">443</parameter>
    <parameter name="non-blocking" locked="false">true</parameter>
    <!--parameter name="bind-address" locked="false">hostname or IP address</parameter-->
    <!--parameter name="WSDLEPRPrefix" locked="false">https://apachehost:port/somepath</parameter-->
   <parameter name="httpGetProcessor" locked="false">org.wso2.carbon.mediation.transport.handlers.PassThroughNHttpGetProcessor</parameter>
    <parameter name="keystore" locked="false">
        <KeyStore>
            <Location>repository/resources/security/wso2carbon.jks</Location>
            <Type>JKS</Type>
            <Password>wso2carbon</Password>
            <KeyPassword>wso2carbon</KeyPassword>
        </KeyStore>
    </parameter>
    <parameter name="truststore" locked="false">
        <TrustStore>
            <Location>repository/resources/security/client-truststore.jks</Location>
            <Type>JKS</Type>
            <Password>wso2carbon</Password>
        </TrustStore>
    </parameter>
    <!--<parameter name="SSLVerifyClient">require</parameter>
        supports optional|require or defaults to none -->
</transportReceiver>

Basically I only changed the 8243 default port with 443 standard HTTPS port used to expose the final endpoint.

Now, executing the API from the Store portal I obtain a cURL command that works on the expected 443 port:

curl -k -X POST "https://ENDPOINT_IP_ADDRESS:443/puntualitest/v1.0.0/puntuali" -H "accept: application/json" -H "Content-Type: application/json" -H "Authorization: Bearer XXXXX-YYYY-ZZZZ-KKKK-WWWW" -d "{ \"header\": { \"msgUid\": \"a36bea3f-6dc6-49d7-9376-f31692930ba9\", \"timestamp\": 1567060509108, \"metadata\": { \"TRACKER_BIZID_REV_CODICE\": \"7175\", \"TRACKER_BIZID_REV_NUMERO\": \"545/2019\" }, \"codApplication\": null, \"codEnte\": null, \"invocationContext\": null, \"caller\": \"SRW\", \"user\": null, \"service\": \"crediti.invioPosizioneCreditoria\" }, \"body\": { \"@dto\": \"binary\", \"content\": \"PD94bWwgdmVyc2..............................+\" }}"

This seems correct, trying to perform the previous cURL command from the bash shell of the machine on which WSO2 API Manager is installed I obtain a JSON response from the API, this:

{"timestamp":"2020-02-29T12:13:54.630+0000","status":404,"error":"Not Found","message":"No message available","path":"/puntualitest/v1.0.0/puntuali"}

It contains an error message but I think that it is cause by a "wrong" payload, anyway it seems that the final registered API endpoint received my request, elaboreted it and return me a JSON message (is it this reasoning correct)?

The problem is that trying to perform the request directly from the inside of the Store portal of WSO2 API Manager I am obtaining the following error message:

<am:fault xmlns:am="http://wso2.org/apimanager">
  <am:code>101500</am:code>
  <am:type>Status report</am:type>
  <am:message>Runtime Error</am:message>
  <am:description>Error in Sender</am:description>
</am:fault>

Reading on the documantation it seems to me that the error having code 101500 could be related to a certificate problem:

WSO2 ESB 4.9.0: what means error 101500

The previous link referer to ESB product and not API Manager but I suspect that the problem could be the same. I suspect it also because into my log file (/usr/lib/wso2/wso2am/2.6.0/repository/logs/wso2carbon.log) when I perform the previous request from the Store portal I obtain the following error message:

TID: [-1] [] [2020-02-29 13:34:58,686] ERROR {org.apache.synapse.transport.passthru.SourceHandler} -  I/O error: Received fatal alert: certificate_unknown {org.apache.synapse.transport.passthru.SourceHandler}
javax.net.ssl.SSLException: Received fatal alert: certificate_unknown
        at sun.security.ssl.Alerts.getSSLException(Alerts.java:208)
        at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1647)
        at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1615)
        at sun.security.ssl.SSLEngineImpl.recvAlert(SSLEngineImpl.java:1781)
        at sun.security.ssl.SSLEngineImpl.readRecord(SSLEngineImpl.java:1070)
        at sun.security.ssl.SSLEngineImpl.readNetRecord(SSLEngineImpl.java:896)
        at sun.security.ssl.SSLEngineImpl.unwrap(SSLEngineImpl.java:766)
        at javax.net.ssl.SSLEngine.unwrap(SSLEngine.java:624)
        at org.apache.http.nio.reactor.ssl.SSLIOSession.doUnwrap(SSLIOSession.java:245)
        at org.apache.http.nio.reactor.ssl.SSLIOSession.doHandshake(SSLIOSession.java:280)
        at org.apache.http.nio.reactor.ssl.SSLIOSession.isAppInputReady(SSLIOSession.java:410)
        at org.apache.http.impl.nio.reactor.AbstractIODispatch.inputReady(AbstractIODispatch.java:119)
        at org.apache.http.impl.nio.reactor.BaseIOReactor.readable(BaseIOReactor.java:159)
        at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvent(AbstractIOReactor.java:338)
        at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvents(AbstractIOReactor.java:316)
        at org.apache.http.impl.nio.reactor.AbstractIOReactor.execute(AbstractIOReactor.java:277)
        at org.apache.http.impl.nio.reactor.BaseIOReactor.execute(BaseIOReactor.java:105)
        at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:586)
        at java.lang.Thread.run(Thread.java:748)
TID: [-1234] [] [2020-02-29 13:34:58,827]  WARN {org.wso2.carbon.apimgt.keymgt.service.thrift.APIKeyValidationServiceImpl} -  Invalid session id for thrift authenticator. {org.wso2.carbon.apimgt.keymgt.service.thrift.APIKeyValidationServiceImpl}
TID: [-1234] [] [2020-02-29 13:34:58,829] ERROR {org.wso2.carbon.apimgt.keymgt.service.thrift.APIKeyValidationServiceImpl} -  Error in invoking validate key via thrift.. {org.wso2.carbon.apimgt.keymgt.service.thrift.APIKeyValidationServiceImpl}
TID: [-1234] [] [2020-02-29 13:34:58,830]  WARN {org.wso2.carbon.apimgt.gateway.handlers.security.thrift.ThriftKeyValidatorClient} -  Login failed.. Authenticating again.. {org.wso2.carbon.apimgt.gateway.handlers.security.thrift.ThriftKeyValidatorClient}
TID: [-1234] [] [2020-02-29 13:34:58,846]  INFO {org.wso2.carbon.core.services.util.CarbonAuthenticationUtil} -  '[email protected] [-1234]' logged in at [2020-02-29 13:34:58,845+0100] from IP address  {org.wso2.carbon.core.services.util.CarbonAuthenticationUtil}
TID: [-1] [] [2020-02-29 13:34:58,941] ERROR {org.apache.synapse.transport.passthru.TargetHandler} -  I/O error: General SSLEngine problem {org.apache.synapse.transport.passthru.TargetHandler}
javax.net.ssl.SSLHandshakeException: General SSLEngine problem
        at sun.security.ssl.Handshaker.checkThrown(Handshaker.java:1521)
        at sun.security.ssl.SSLEngineImpl.checkTaskThrown(SSLEngineImpl.java:528)
        at sun.security.ssl.SSLEngineImpl.writeAppRecord(SSLEngineImpl.java:1197)
        at sun.security.ssl.SSLEngineImpl.wrap(SSLEngineImpl.java:1165)
        at javax.net.ssl.SSLEngine.wrap(SSLEngine.java:469)
        at org.apache.http.nio.reactor.ssl.SSLIOSession.doWrap(SSLIOSession.java:237)
        at org.apache.http.nio.reactor.ssl.SSLIOSession.doHandshake(SSLIOSession.java:271)
        at org.apache.http.nio.reactor.ssl.SSLIOSession.isAppInputReady(SSLIOSession.java:410)
        at org.apache.http.impl.nio.reactor.AbstractIODispatch.inputReady(AbstractIODispatch.java:119)
        at org.apache.http.impl.nio.reactor.BaseIOReactor.readable(BaseIOReactor.java:159)
        at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvent(AbstractIOReactor.java:338)
        at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvents(AbstractIOReactor.java:316)
        at org.apache.http.impl.nio.reactor.AbstractIOReactor.execute(AbstractIOReactor.java:277)
        at org.apache.http.impl.nio.reactor.BaseIOReactor.execute(BaseIOReactor.java:105)
        at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:586)
        at java.lang.Thread.run(Thread.java:748)
Caused by: javax.net.ssl.SSLHandshakeException: General SSLEngine problem
        at sun.security.ssl.Alerts.getSSLException(Alerts.java:192)
        at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1709)
        at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:318)
        at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:310)
        at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1639)
        at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:223)
        at sun.security.ssl.Handshaker.processLoop(Handshaker.java:1037)
        at sun.security.ssl.Handshaker$1.run(Handshaker.java:970)
        at sun.security.ssl.Handshaker$1.run(Handshaker.java:967)
        at java.security.AccessController.doPrivileged(Native Method)
        at sun.security.ssl.Handshaker$DelegatedTask.run(Handshaker.java:1459)
        at org.apache.http.nio.reactor.ssl.SSLIOSession.doRunTask(SSLIOSession.java:255)
        at org.apache.http.nio.reactor.ssl.SSLIOSession.doHandshake(SSLIOSession.java:293)
        ... 9 more
Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
        at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:397)
        at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:302)
        at sun.security.validator.Validator.validate(Validator.java:262)
        at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:324)
        at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:281)
        at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:136)
        at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1626)
        ... 17 more
Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
        at sun.security.provider.certpath.SunCertPathBuilder.build(SunCertPathBuilder.java:141)
        at sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:126)
        at java.security.cert.CertPathBuilder.build(CertPathBuilder.java:280)
        at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:392)
        ... 23 more
TID: [-1234] [] [2020-02-29 13:34:58,948]  WARN {org.apache.synapse.endpoints.EndpointContext} -  Endpoint : SirPuntuali--vv1.0.0_APIproductionEndpoint with address https://ENDPOINT_IP_ADDRESS/cmd/j/ will be marked SUSPENDED as it failed {org.apache.synapse.endpoints.EndpointContext}
TID: [-1234] [] [2020-02-29 13:34:58,948]  WARN {org.apache.synapse.endpoints.EndpointContext} -  Suspending endpoint : SirPuntuali--vv1.0.0_APIproductionEndpoint with address https://ENDPOINT_IP_ADDRESS/cmd/j/ - last suspend duration was : 30000ms and current suspend duration is : 30000ms - Next retry after : Sat Feb 29 13:35:28 CET 2020 {org.apache.synapse.endpoints.EndpointContext}
TID: [-1234] [] [2020-02-29 13:34:58,949]  INFO {org.apache.synapse.mediators.builtin.LogMediator} -  STATUS = Executing default 'fault' sequence, ERROR_CODE = 101500, ERROR_MESSAGE = Error in Sender {org.apache.synapse.mediators.builtin.LogMediator}
TID: [-1] [] [2020-02-29 13:34:58,979]  INFO {org.wso2.carbon.databridge.core.DataBridge} -  user admin connected {org.wso2.carbon.databridge.core.DataBridge}

So it seems that WSO2 API Manager is trying to send the request to the correct endpoint but there is a certificate problem. Is it this reasoning correct?

If this could be the problem now I have some doubts about what I have to do to solve my problem:

  1. Have I to obtain a certificate generated on the server hosting the final endpoint and have I to set it on my WSO2 API Manager or, on the contrary, have I to generate a certificate on the WSO2 API Manager machine and I have to provide it to the machine hosting the final API? Reading on the documentation it seems to me that I have to obtain a certificate from the API hosting machine and I have to upload this certificate into WSO2 API Manager application (as shown here: https://apim.docs.wso2.com/en/latest/learn/design-api/endpoints/certificates/). But I am not sure of this assumption.

  2. A self signed certificate is ok? In case what is the procedure to generate it and what kind of certificate I need to obtain (I have to provide precise information to the guys working on the final API machine)

  3. Probably a trivial question: the Store portal is generating a cURL request using the -k option that is used to ignore certificate (infact performing it directly in the shell it seems to work fine). Why sending the request from the Store portal is not working? I suspect that it generates a cURL request for test pourpose but that under the hood the API Manager is not performing a simple cURL request.

1

1 Answers

1
votes

The behavior is a bit strange. Just to explain what happens here, there are 2 HTTP calls involved.

  1. The client (curl or UI) to the gateway
  2. The gateway to the backend

As per the 2nd error trace, the problem is with the connection between the gateway and the backend. Answering your first question, to resolve this, you have to get the certificate of the backend endpoint and install it to the APIM's client-truststore.jks. You can either do it for each API via the UI, or you can directly install it to the jks file.

However, since this is independent of the client you use, you should see the same behavior for both cURL and UI. I don't get how it works for cURL.

Answering your 3rd question, the UI does not use curl inside to make the call to the gateway. And it has nothing to do with the above error either.