#1 2017-04-21 01:16:09

aaronmartin1651
Member
Registered: 2017-03-02

Report Server keeps locking up

Once or twice a week ReportServer will lock up.  Either the site never loads or eventually I get the following error message:

Error
500 Internal Server Error <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN"> <html><head> <title>500 Internal Server Error</title> </head><body>
Internal Server Error
<p>The server encountered an internal error or misconfiguration and was unable to complete your request.</p> <p>Please contact the server administrator at you@example.com to inform them of the time this error occurred, and the actions you performed just before this error.</p> <p>More information about this error may be available in the server error log.</p> </body></html> Statuscode: 500 EncodedResponse: <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN"> <html><head> <title>500 Internal Server Error</title> </head><body>
Internal Server Error
<p>The server encountered an internal error or misconfiguration and was unable to complete your request.</p> <p>Please contact the server administrator at you@example.com to inform them of the time this error occurred, and the actions you performed just before this error.</p> <p>More information about this error may be available in the server error log.</p> </body></html>
OK
Error


Why does this keep occurring and how can I prevent it?

Offline

#2 2017-04-21 05:55:17

eduardo
Administrator
Registered: 2016-11-01
Website

Re: Report Server keeps locking up

Hi aaronmartin1651,

can you please check your tomcat error logs ?

Regards,
Eduardo

Offline

#3 2017-04-21 12:07:14

aaronmartin1651
Member
Registered: 2017-03-02

Re: Report Server keeps locking up

I am seeing this error a lot:

----------> Parent Classloader:^M
java.net.URLClassLoader@548c4f57^M
=org.apache.xerces.util.XMLGrammarPoolImpl@20925027}]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.
21-Apr-2017 01:19:50.619 SEVERE [localhost-startStop-2] org.apache.catalina.loader.WebappClassLoaderBase.checkThreadLocalMapForLeaks The web application [reportserver] created a ThreadLocal with key of type [java.lang.ThreadLocal] (value [java.lang.ThreadLocal@e682e0a]) and a value of type [java.lang.Class] (value [class produce45kl45sub45per_1492652492092_280553]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.
21-Apr-2017 01:19:50.619 SEVERE [localhost-startStop-2] org.apache.catalina.loader.WebappClassLoaderBase.checkThreadLocalMapForLeaks The web application [reportserver] created a ThreadLocal with key of type [java.lang.InheritableThreadLocal] (value [java.lang.InheritableThreadLocal@3bf43fa9]) and a value of type [net.datenwerke.rs.base.service.reportengines.jasper.JasperReportEngine$3] (value [net.datenwerke.rs.base.service.reportengines.jasper.JasperReportEngine$3@fc57dfa]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.
21-Apr-2017 01:19:50.619 SEVERE [localhost-startStop-2] org.apache.catalina.loader.WebappClassLoaderBase.checkThreadLocalMapForLeaks The web application [reportserver] created a ThreadLocal with key of type [java.lang.ThreadLocal] (value [java.lang.ThreadLocal@51580577]) and a value of type [org.hibernate.jpa.internal.EntityManagerImpl] (value [org.hibernate.jpa.internal.EntityManagerImpl@6056b54e]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.
21-Apr-2017 01:19:50.619 SEVERE [localhost-startStop-2] org.apache.catalina.loader.WebappClassLoaderBase.checkThreadLocalMapForLeaks The web application [reportserver] created a ThreadLocal with key of type [java.lang.ThreadLocal] (value [java.lang.ThreadLocal@311a77bb]) and a value of type [org.apache.commons.collections.map.ReferenceMap] (value [{WebappClassLoader^M
  context: reportserver^M
  delegate: false^M
----------> Parent Classloader:^M

Offline

#4 2017-04-21 12:11:49

eduardo
Administrator
Registered: 2016-11-01
Website

Re: Report Server keeps locking up

Hi aaronmartin,

is this the error log when you are getting the 500 Internal Server Error ? These may be unrelated.. so please post the log entry when you get the 500 Internal Server Error

Regards,
Eduardo

Offline

#5 2017-04-21 23:09:59

aaronmartin1651
Member
Registered: 2017-03-02

Re: Report Server keeps locking up

There errors from the reportserver.2017-04-21.log file:
1-Apr-2017 01:19:50.257 SEVERE [ajp-apr-8009-exec-8] org.hibernate.engine.jdbc.spi.SqlExceptionHelper.logExceptions An SQLException was provoked by the following failure: java.lang.InterruptedException
21-Apr-2017 01:19:50.259 SEVERE [scheduler-thread-1] org.hibernate.engine.jdbc.spi.SqlExceptionHelper.logExceptions An SQLException was provoked by the following failure: java.lang.InterruptedException
21-Apr-2017 01:19:50.259 SEVERE [NioProcessor-1] org.hibernate.engine.jdbc.spi.SqlExceptionHelper.logExceptions An SQLException was provoked by the following failure: java.lang.InterruptedException
21-Apr-2017 01:19:50.261 SEVERE [NioProcessor-3] org.hibernate.engine.jdbc.spi.SqlExceptionHelper.logExceptions An SQLException was provoked by the following failure: java.lang.InterruptedException
21-Apr-2017 01:19:50.262 SEVERE [ajp-apr-8009-exec-10] org.hibernate.engine.jdbc.spi.SqlExceptionHelper.logExceptions An SQLException was provoked by the following failure: java.lang.InterruptedException
21-Apr-2017 01:19:50.263 SEVERE [dwdaemon-MaintenanceServiceImpl-1] org.hibernate.engine.jdbc.spi.SqlExceptionHelper.logExceptions An SQLException was provoked by the following failure: java.lang.InterruptedException
21-Apr-2017 01:19:50.257 SEVERE [ajp-apr-8009-exec-1] org.hibernate.engine.jdbc.spi.SqlExceptionHelper.logExceptions An SQLException was provoked by the following failure: java.lang.InterruptedException

Offline

#6 2017-04-21 23:12:33

aaronmartin1651
Member
Registered: 2017-03-02

Re: Report Server keeps locking up

This error message from catalina.2017-04-21.log file:
21-Apr-2017 01:19:50.496 WARNING [localhost-startStop-2] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesJdbc The web application [reportserver] registered the JDBC driver [org.hsqldb.jdbc.JDBCDriver] but failed to unregister it when the web application was stopped. To prevent a memory leak, the JDBC Driver has been forcibly unregistered.

Offline

#7 2017-04-21 23:15:16

aaronmartin1651
Member
Registered: 2017-03-02

Re: Report Server keeps locking up

Where else might I look for logs containing 500 errors?  I am looking in /opt/reportserver-versionnumber/apache-tomcat/logs and the above errors are all I am seeing...

Offline

#8 2017-05-02 13:50:39

eduardo
Administrator
Registered: 2016-11-01
Website

Re: Report Server keeps locking up

Hi aaronmartin1651,

we identified and fixed a memory leak with the following class:
net.datenwerke.rs.base.service.datasources.statementmanager.StatementManagerServiceImpl

Could you please check with for example Eclipse Memory Analyzer if the memory leak you describe is in this class?

This will be fixed in the next reportserver version.

In the meanwhile, restarting reportserver periodically, e.g. every night, should solve the problem.

Best regards,
Eduardo

Offline

Board footer

Powered by FluxBB