#1 2018-07-10 23:00:43

mkrause
Member
Registered: 2018-07-01

Reportserver stops responding after about 12-16 hours

Team,

A second help request. Am running tomcat7 on ubuntu 14 with the latest reportserver 7.
After about 12-16 hours, when going to the server, it stops responding to requests. Just waits and nothing appears in the browser …

If I specify an explicit path that is other than the root path where the reportserver is installed, the tomcat7 server does respond http status 404 error -- so tomcat7 is not entirely dead.

I have enabled logging properties as per directions, but am not seeing any explicit failures in the logs.  Any guidance to make the logs more explicit or get right to this would be appreciated.


The only working solution I have is to do a 'service  tomcat7 restart' command, and then suddenly reportserver starts responding.  I have 2gb allocated to maximum heap size, and this server generally has only 1 or 2 users at a time.  The max query size result is generally about 16mb  for a pre-canned report.


Hopefully that's enough to start on. Appreciate any comments or ideas in advance. Am not too familiar with tomcat/java app environments (much more comfortable on the apache/MySQL side of things), so any guidance here is appreciated.

Offline

#2 2018-07-11 06:43:55

eduardo
Administrator
Registered: 2016-11-01
Website

Re: Reportserver stops responding after about 12-16 hours

Hi mkrause,

which reportserver version exactly are you using (rsversion.properties) ?

Regards,
Eduardo

Offline

#3 2018-07-11 11:41:01

mkrause
Member
Registered: 2018-07-01

Re: Reportserver stops responding after about 12-16 hours

3.03 6002.  Tomcat version is 7.052, for ubuntu 14.

Last edited by mkrause (2018-07-11 11:41:46)

Offline

#4 2018-07-11 11:48:41

eduardo
Administrator
Registered: 2016-11-01
Website

Re: Reportserver stops responding after about 12-16 hours

Hi mkrause,

where exactly did you set this: " I have 2gb allocated to maximum heap size" ? Could you paste the settings ?

To change the logger settings, you can change: logging-rs.properties:
E.g.:

.level = FINEST
net.datenwerke.level = INFO

Regards,
Eduardo

Offline

#5 2018-07-11 13:47:52

mkrause
Member
Registered: 2018-07-01

Re: Reportserver stops responding after about 12-16 hours

In /etc/default/tomcat7 …  Just upped it back to 4g. 

JAVA_OPTS="-Djava.awt.headless=true -Xmx4g  -XX:+UseConcMarkSweepGC -Dfile.encoding=UTF8"

JAVA_OPTS="${JAVA_OPTS} -Drs.configdir=/opt/reportserver"

Offline

#6 2018-07-12 07:26:15

eduardo
Administrator
Registered: 2016-11-01
Website

Re: Reportserver stops responding after about 12-16 hours

Hi mkrause,

did you get any logs on the error by changing the logging-rs.properties file ?
For testing, install tomcat using its default settings (just download and install) and put reportserver into a subdirectory of your webapps directory. With this standard installation, please check if the error is still happening.

Regards,
Eduardo

Offline

#7 2018-07-12 18:44:12

mkrause
Member
Registered: 2018-07-01

Re: Reportserver stops responding after about 12-16 hours

I currently have logs set to INFO as FINE simply had too much output.


root:/var/log/tomcat7# more /opt/reportserver/logging-rs.prop
.level = INFO
net.datenwerke.level = INFO


The reportserver stopped responding at this state of the world, and most recent entries in the reportserver log were ---  I just disabled all SFTP access via permissions after I saw this.  This looks like outsiders maybe trying to exploit something here?  Please send me an email and I can send more detailed logs...  I have already set up in my server.xml a <Valve className="org.apache.catalina.valves.RemoteAddrValve"...> setting to whitelist only in-network users of ReportServer, and that appears to be working (so the other issue I am having with trying ReportServer outside root is no longer of concern).  ReportServer still failed to respond despite this -- even though tomcat7 is response on outside Reportserver queries...


[Thu Jul 12 13:53:07 EDT 2018] INFO: org.apache.sshd.common.session.AbstractSession$1IoSessionCloser operationComplete - Session null@/61.219.45.81:37385 closed
[Thu Jul 12 13:57:35 EDT 2018] INFO: org.apache.sshd.server.session.ServerSession <init> - Session created from /140.143.7.160:47368
[Thu Jul 12 13:57:40 EDT 2018] WARNING: net.datenwerke.rs.remoteaccess.service.sftp.SftpAuthenticator authenticate - null
java.lang.NullPointerException
        at net.datenwerke.security.service.security.SecurityServiceImpl.testRights_simpleTarget(SecurityServiceImpl.java:410)
        at net.datenwerke.security.service.security.SecurityServiceImpl.testRights(SecurityServiceImpl.java:333)
        at net.datenwerke.security.service.security.SecurityServiceImpl.checkRights(SecurityServiceImpl.java:246)
        at net.datenwerke.security.service.security.SecurityServiceImpl.checkRights(SecurityServiceImpl.java:234)
        at net.datenwerke.security.service.security.SecurityServiceImpl.checkRights(SecurityServiceImpl.java:219)
        at net.datenwerke.rs.remoteaccess.service.sftp.SftpAuthenticator.authenticate(SftpAuthenticator.java:73)
        at org.apache.sshd.server.auth.UserAuthPassword.checkPassword(UserAuthPassword.java:55)
        at org.apache.sshd.server.auth.UserAuthPassword.auth(UserAuthPassword.java:49)
        at org.apache.sshd.server.session.ServerSession.userAuth(ServerSession.java:392)
        at org.apache.sshd.server.session.ServerSession.handleMessage(ServerSession.java:201)
        at org.apache.sshd.common.session.AbstractSession.decode(AbstractSession.java:566)
        at org.apache.sshd.common.session.AbstractSession.messageReceived(AbstractSession.java:236)
        at org.apache.sshd.common.AbstractSessionIoHandler.messageReceived(AbstractSessionIoHandler.java:58)
        at org.apache.mina.core.filterchain.DefaultIoFilterChain$TailFilter.messageReceived(DefaultIoFilterChain.java:716)
        at org.apache.mina.core.filterchain.DefaultIoFilterChain.callNextMessageReceived(DefaultIoFilterChain.java:434)
        at org.apache.mina.core.filterchain.DefaultIoFilterChain.access$1200(DefaultIoFilterChain.java:46)
        at org.apache.mina.core.filterchain.DefaultIoFilterChain$EntryImpl$1.messageReceived(DefaultIoFilterChain.java:796)
        at org.apache.mina.core.filterchain.IoFilterAdapter.messageReceived(IoFilterAdapter.java:119)
        at org.apache.mina.core.filterchain.DefaultIoFilterChain.callNextMessageReceived(DefaultIoFilterChain.java:434)
        at org.apache.mina.core.filterchain.DefaultIoFilterChain.fireMessageReceived(DefaultIoFilterChain.java:426)
        at org.apache.mina.core.polling.AbstractPollingIoProcessor.read(AbstractPollingIoProcessor.java:697)
        at org.apache.mina.core.polling.AbstractPollingIoProcessor.process(AbstractPollingIoProcessor.java:651)
        at org.apache.mina.core.polling.AbstractPollingIoProcessor.process(AbstractPollingIoProcessor.java:640)
        at org.apache.mina.core.polling.AbstractPollingIoProcessor.access$600(AbstractPollingIoProcessor.java:67)
        at org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.run(AbstractPollingIoProcessor.java:1111)
        at org.apache.mina.util.NamePreservingRunnable.run(NamePreservingRunnable.java:64)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1152)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:622)
        at java.lang.Thread.run(Thread.java:748)

[Thu Jul 12 13:57:40 EDT 2018] INFO: org.apache.sshd.common.session.AbstractSession$1IoSessionCloser operationComplete - Session null@/140.143.7.160:47368 closed
[Thu Jul 12 14:27:35 EDT 2018] INFO: org.apache.sshd.server.session.ServerSession <init> - Session created from /111.19.140.93:35848

Offline

#8 2018-07-13 09:10:36

eduardo
Administrator
Registered: 2016-11-01
Website

Re: Reportserver stops responding after about 12-16 hours

Hi mkrause,

please post your /fileserver/etc/misc/misc.cf file.
Check here for the SFTP configuration: https://reportserver.net/en/guides/conf … roperties/ 4.9.4: Configuring the SFTP Server

Check this: "If you do not want to start the SFTP server simply supply an invalid path.". Try to supply an invalid path and restart reportserver. Are you still seeing these messages after the restart?

Regards,
Eduardo

Offline

#9 2018-07-13 12:58:33

mkrause
Member
Registered: 2018-07-01

Re: Reportserver stops responding after about 12-16 hours

ok just edited it away from the default so now port 8022 doesn't answer sftp queries as it did before.
Here's what it is now.   Btw I moved the logging-rs.properties levels both to CONFIG since the prior debugs were sparse in showing failure.  Will let you know if I have any failures going forward, and use FINE if can not find anything.

FWIW, the server crashed again prior to making this change.

<?xml version="1.0" encoding="UTF-8"?>
<configuration>
   <database>
      <oracle>
         <filter>
            <convertclobs>true</convertclobs>
         </filter>
      </oracle>
   </database>
   <remoteaccess>
      <sftp>
         <keylocation>/test/test.cf</keylocation>
         <port>8022</port>
      </sftp>
   </remoteaccess>
</configuration>

Offline

#10 2018-12-24 20:58:56

hw
Member
Registered: 2018-12-24

Re: Reportserver stops responding after about 12-16 hours

I am encountering the same problem with reportserver-3.0.4-1 on Linux.

It is the community edition of ReportServer.

I tried to find /fileserver/etc/misc/misc.cf, but there is no directory named fileserver, whether at the filesystem root or at the ReportServer root.  Neither is there a file named misc.cf.

Is my ReportServer installation missing something?

Then I installed reportserver-3.0.5-0.  There are no fileserver and misc.cf either.

Last edited by hw (2018-12-24 21:02:08)

Offline

#11 2018-12-25 16:56:51

eduardo
Administrator
Registered: 2016-11-01
Website

Re: Reportserver stops responding after about 12-16 hours

Hi hw,

these files are located in the internal reportserver file system. You can access the internal file system in Administration -> File System.
More details here: https://reportserver.net/en/guides/admi … le-System/

Regards,
Eduardo

Offline

Board footer

Powered by FluxBB