Websphere 6.1 Stops Automatically

4
votes

we're encountering an error in websphere version 6.1 running on a windows 2003 environment. The server suddenly stops but there was no one who issued a stop on the server.

here is the logs of the server

what is weird is we did not an issue a stop it just suddenly stops. it does not start after this

anyone encountered this error? what do i need to look at?

Tried to retrieve a localized string for a component that has not yet been added to the page. This can sometimes lead to an invalid or no localized resource returned. Make sure you are not calling Component#getString() inside your Component's constructor. Offending component: [MarkupContainer [Component id = content]]
[1/16/13 11:10:15:150 CST] 000008ec AdminHelper   A   ADMN1020I: An attempt is made to stop the server1 server. (User ID = defaultWIMFileBasedRealm/testadmin)
[1/16/13 11:10:15:259 CST] 00000010 TCPChannel    I   TCPC0002I: TCP Channel TCP_3 has stopped listening on host *  (IPv4) port 9043. 
[1/16/13 11:10:15:259 CST] 00000010 TCPChannel    I   TCPC0002I: TCP Channel TCP_2 has stopped listening on host *  (IPv4) port 9080. 
[1/16/13 11:10:15:275 CST] 00000010 TCPChannel    I   TCPC0002I: TCP Channel TCP_1 has stopped listening on host *  (IPv4) port 9060. 
[1/16/13 11:10:20:275 CST] 00000010 ApplicationMg A   WSVR0217I: Stopping application: 10012012-TEST-66-V105-tester-ufm_war
[1/16/13 11:10:20:306 CST] 00000010 ServletWrappe I   SRVE0253I: [10012012-TEST-66-V105-tester-ufm_war] [/ufm] [com_ibm_ws_rrd_webservices_service_remote_RRDServiceSOAPBindingImplLTPA]: Destroy successful.
[1/16/13 11:10:20:322 CST] 00000010 ServletWrappe I   SRVE0253I: [10012012-TEST-66-V105-tester-ufm_war] [/ufm] [com_ibm_ws_rrd_webservices_service_remote_RRDServiceSOAPBindingImplCustom]: Destroy successful.
[1/16/13 11:10:20:322 CST] 00000010 ServletWrappe I   SRVE0253I: [10012012-TEST-66-V105-tester-ufm_war] [/ufm] [com_ibm_ws_rrd_webservices_service_remote_RRDServiceSOAPBindingImpl]: Destroy successful.
[1/16/13 11:10:20:322 CST] 00000010 WebApp        A   SRVE0180I: [10012012-TEST-66-V105-tester-ufm_war#10012012-TEST-66-V105-tester-ufm.war] [/ufm] [Servlet.LOG]: Closing Spring root WebApplicationContext
[1/16/13 11:10:20:603 CST] 00000010 XmlWebApplica I org.apache.commons.logging.impl.Jdk14Logger info Closing org.springframework.web.context.support.XmlWebApplicationContext@61c461c4: display name [Root WebApplicationContext]; startup date [Fri Jan 11 15:48:22 CST 2013]; root of context hierarchy
[1/16/13 11:10:21:228 CST] 00000010 LocalSessionF I org.apache.commons.logging.impl.Jdk14Logger info Closing Hibernate SessionFactory
[1/16/13 11:10:21:228 CST] 00000010 SystemOut     O INFO  - SessionFactoryImpl         - closing
INFO  - essionFactoryObjectFactory - Unbinding factory from JNDI name: sessionFactory
INFO  - NamingHelper               - JNDI InitialContext properties:{}
INFO  - essionFactoryObjectFactory - Unbound factory from JNDI name: sessionFactory

[1/16/13 11:10:21:260 CST] 00000010 ServletWrappe I   SRVE0253I: [10012012-TEST-66-V105-tester-ufm_war] [/ufm] [context]: Destroy successful.

[1/16/13 11:10:21:431 CST] 00000010 ApplicationMg A   WSVR0220I: Application stopped: 10012012-TEST-66-V105-tester-ufm_war

[1/16/13 11:10:21:447 CST] 00000010 ApplicationMg A   WSVR0217I: Stopping application: SchedulerCalendars

[1/16/13 11:10:21:463 CST] 00000010 EJBContainerI I   WSVR0041I: Stopping EJB jar: Calendars.jar

[1/16/13 11:10:21:463 CST] 00000010 EJBContainerI I   WSVR0059I: EJB jar stopped: Calendars.jar

[1/16/13 11:10:21:478 CST] 00000010 ApplicationMg A   WSVR0220I: Application stopped: SchedulerCalendars

[1/16/13 11:10:21:494 CST] 00000010 ApplicationMg A   WSVR0217I: Stopping application: filetransfer

[1/16/13 11:10:21:494 CST] 00000010 ServletWrappe I   SRVE0253I: [filetransfer] [/FileTransfer] [transfer]: Destroy successful.

[1/16/13 11:10:21:525 CST] 00000010 ApplicationMg A   WSVR0220I: Application stopped: filetransfer

[1/16/13 11:10:21:525 CST] 00000010 ApplicationMg A   WSVR0217I: Stopping application: ManagementEJB

[1/16/13 11:10:21:525 CST] 00000010 EJBContainerI I   WSVR0041I: Stopping EJB jar: mejb.jar

[1/16/13 11:10:21:525 CST] 00000010 EJBContainerI I   WSVR0059I: EJB jar stopped: mejb.jar

[1/16/13 11:10:21:556 CST] 00000010 ApplicationMg A   WSVR0220I: Application stopped: ManagementEJB

[1/16/13 11:10:21:556 CST] 00000010 ApplicationMg A   WSVR0217I: Stopping application: 10012012-TEST-66-V105-test-management_war

[1/16/13 11:10:21:556 CST] 00000010 ServletWrappe I   SRVE0253I: [10012012-TEST-66-V105-test-management_war] [/] [com_ibm_ws_rrd_webservices_service_remote_RRDServiceSOAPBindingImplLTPA]: Destroy successful.

[1/16/13 11:10:21:572 CST] 00000010 ServletWrappe I   SRVE0253I: [10012012-TEST-66-V105-test-management_war] [/] [com_ibm_ws_rrd_webservices_service_remote_RRDServiceSOAPBindingImplCustom]: Destroy successful.

[1/16/13 11:10:21:572 CST] 00000010 ServletWrappe I   SRVE0253I: [10012012-TEST-66-V105-test-management_war] [/] [com_ibm_ws_rrd_webservices_service_remote_RRDServiceSOAPBindingImpl]: Destroy successful.

[1/16/13 11:10:21:572 CST] 00000010 ServletWrappe I   SRVE0253I: [10012012-TEST-66-V105-test-management_war] [/] [JasperObject]: Destroy successful.

[1/16/13 11:10:22:119 CST] 00000010 ServletWrappe I   SRVE0253I: [10012012-TEST-66-V105-test-management_war] [/] [wicket.test-management]: Destroy successful.

[1/16/13 11:10:22:119 CST] 00000010 WebApp        A   SRVE0180I: [10012012-TEST-66-V105-test-management_war#10012012-TEST-66-V105-test-management.war] [/] [Servlet.LOG]: Closing Spring root WebApplicationContext

[1/16/13 11:10:22:416 CST] 00000010 XmlWebApplica I org.apache.commons.logging.impl.Jdk14Logger info Closing org.springframework.web.context.support.XmlWebApplicationContext@55a955a9: display name [Root WebApplicationContext]; startup date [Thu Jan 10 15:33:28 CST 2013]; root of context hierarchy



[1/16/13 11:10:22:760 CST] 00000794 WebContainer  E   Servlet.has.become.temporarily.unavailable.for.service

[1/16/13 11:10:27:057 CST] 00000010 LocalSessionF I org.apache.commons.logging.impl.Jdk14Logger info Closing Hibernate SessionFactory
closing
[1/16/13 11:10:27:072 CST] 00000010 SystemOut     O Unbinding factory from JNDI name: sessionFactory
3
javawindowswebspherewebsphere-6.1
Definitely looks like a deliberate stop. Are you certain that first "Tried to retrieve" message is connected to the stop rather than to something earlier in the log? Does the stop happen immediately on starting or after it's been running for some period of time? At some certain time of day/week? – dbreaux
somehow, it seems that the new servers that are being setup was the one causing the server to stop. there was a virtual cloning done on the server to setup a new production environment. Tests are being done on that server, oddly enough, it seems to be correlated with the time that stop was being done on that server. Meaning, most probably, it was the one stopping the production servers. I just don't know why the new set of servers is able to control the old servers. So the new set of WebSphere Application servers were the one actually contorlling the servers causing them to stop. – grassbl8d
it was actually the new server that was stopping it. The newly cloned server when stopped, is issuing a stop command to the old one. Is there anything that I could change? – grassbl8d

3 Answers

0
votes

testadmin is the user that has requested for the server shutdown.

Are there other credentials on this server?

Is this "testadmin" credentials stored in the config file (sas/soap client properties)?

0
votes

we can see it clearly on the logs. The new cloned server is the one causing the server from which is cloned from to stop. The stopserver commands are issued on the cloned server, and it is affecting the production server. This is quite a really weird behavior....

Is there anything I should look at ?

Thanks

0
votes

Ticket to IBM Software Support would help you with root cause. Also I would run the mustgather utility to see all the logs that could be involved in the stoppage.