You are not logged in.
Hi,
I'm trying to setup a CSV datasource which uses a URL Connector.
Purpose is to get live monitoring status information from a Nagios/Check_MK server which does provide the info as csv.
The access to this URL has to be protected with username and password.
A datasource of type "Relational Database " provides two fields for username and password including the security mechanism to not send back the password to browser.
A datasource of type "CSV List" with "URL connector" does not provide such username and password fields.
I've also tried to include the username:password into URL as "http://username:password@IPofNagiosServer/check_mk/view.py/....", but always receive a 401 Authentication required.
Error message:
"
Error
The report could not be executed: java.lang.IllegalStateException: java.lang.IllegalStateException: java.io.IOException: Server returned HTTP response code: 401 for URL: http://username:password@IPofNagiosServer/check_mk/view.py?view_name=servicegroup&servicegroup=PROD&output_format=csv_export
"
On Nagios/Check_MK apache the username+password seem to not arrive. Apache httpd access.log:
172.0.0.2 - - [19/Jun/2015:17:06:09 +0200] "GET /check_mk/view.py?view_name=servicegroup&servicegroup=PROD&output_format=csv_export HTTP/1.1" 401 401 "-" "Java/1.7.0_79"
My first assumption:
Username + password in URL are not supported by java.net.URL / java.net.URLConnection and would need to be provided via HTTP header / connection property "Authentication: Basic <encoded username:password>"
I do not see this as a bug, but as an potential improvement/feature to support passwords for URL Connectors and to protect them in a similar way as you do already for Database connections.
Full exception in reportserver tomcat catalina.log:
Jun 19, 2015 5:06:09 PM net.datenwerke.rs.core.service.reportmanager.ReportExecutorServiceImpl execute
WARNING: java.lang.IllegalStateException: java.lang.IllegalStateException: java.io.IOException: Server returned HTTP response code: 401 for URL: http://username:password@IPofNagiosServer/check_mk/view.py?view_name=servicegroup&servicegroup=PROD&output_format=csv_export
java.lang.RuntimeException: java.lang.IllegalStateException: java.lang.IllegalStateException: java.io.IOException: Server returned HTTP response code: 401 for URL: http://username:password@IPofNagiosServer/check_mk/view.py?view_name=servicegroup&servicegroup=PROD&output_format=csv_export
at net.datenwerke.rs.base.service.datasources.helpers.CsvToInMemoryDb.getConnection(CsvToInMemoryDb.java:143)
at net.datenwerke.rs.base.service.datasources.table.transformers.CsvDatasourceTableTransformer.transform(CsvDatasourceTableTransformer.java:51)
at net.datenwerke.rs.base.service.datasources.table.transformers.CsvDatasourceTableTransformer.transform(CsvDatasourceTableTransformer.java:1)
at net.datenwerke.rs.core.service.datasourcemanager.datasourcetransformers.DatasourceDefinitionTransformerManager.transform(DatasourceDefinitionTransformerManager.java:49)
at net.datenwerke.rs.base.service.reportengines.table.TableReportEngine.doExecuteNormal(TableReportEngine.java:168)
at net.datenwerke.rs.base.service.reportengines.table.TableReportEngine.doExecuteNormal(TableReportEngine.java:148)
at net.datenwerke.rs.base.service.reportengines.table.TableReportEngine.doExecute(TableReportEngine.java:90)
at net.datenwerke.rs.core.service.reportmanager.engine.ReportEngine.execute(ReportEngine.java:127)
at net.datenwerke.rs.core.service.reportmanager.ReportExecutorServiceImpl.execute(ReportExecutorServiceImpl.java:186)
at net.datenwerke.rs.core.service.reportmanager.ReportExecutorServiceImpl.execute(ReportExecutorServiceImpl.java:119)
at net.datenwerke.rs.core.service.reportmanager.ReportExecutorServiceImpl.execute(ReportExecutorServiceImpl.java:106)
at net.datenwerke.rs.core.service.reportmanager.ReportExecutorServiceImpl.execute(ReportExecutorServiceImpl.java:101)
at net.datenwerke.rs.base.service.reportengines.table.TableReportUtilsImpl.getReturnedColumns(TableReportUtilsImpl.java:180)
at net.datenwerke.rs.base.service.reportengines.table.TableReportUtilsImpl.getReturnedPlainColumns(TableReportUtilsImpl.java:144)
at net.datenwerke.rs.base.server.table.TableReportUtilityServiceImpl.getReturnedColumns(TableReportUtilityServiceImpl.java:212)
at net.datenwerke.rs.base.server.table.TableReportUtilityServiceImpl$$EnhancerByGuice$$fd148473.CGLIB$getReturnedColumns$1(<generated>)
at net.datenwerke.rs.base.server.table.TableReportUtilityServiceImpl$$EnhancerByGuice$$fd148473$$FastClassByGuice$$997eb26c.invoke(<generated>)
at com.google.inject.internal.cglib.proxy.$MethodProxy.invokeSuper(MethodProxy.java:228)
at com.google.inject.internal.InterceptorStackCallback$InterceptedMethodInvocation.proceed(InterceptorStackCallback.java:72)
at com.google.inject.persist.jpa.JpaLocalTxnInterceptor.invoke(JpaLocalTxnInterceptor.java:66)
at com.google.inject.internal.InterceptorStackCallback$InterceptedMethodInvocation.proceed(InterceptorStackCallback.java:72)
at net.datenwerke.security.service.security.aop.SecurityCheckInterceptor.invoke(SecurityCheckInterceptor.java:110)
at com.google.inject.internal.InterceptorStackCallback$InterceptedMethodInvocation.proceed(InterceptorStackCallback.java:72)
at net.datenwerke.gf.service.gwtstacktrace.CatchStacktraceInterceptor.invoke(CatchStacktraceInterceptor.java:38)
at com.google.inject.internal.InterceptorStackCallback$InterceptedMethodInvocation.proceed(InterceptorStackCallback.java:72)
at com.google.inject.internal.InterceptorStackCallback.intercept(InterceptorStackCallback.java:52)
at net.datenwerke.rs.base.server.table.TableReportUtilityServiceImpl$$EnhancerByGuice$$fd148473.getReturnedColumns(<generated>)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at com.google.gwt.user.server.rpc.RPC.invokeAndEncodeResponse(RPC.java:561)
at com.google.gwt.user.server.rpc.RemoteServiceServlet.processCall(RemoteServiceServlet.java:208)
at net.datenwerke.rs.base.server.table.TableReportUtilityServiceImpl$$EnhancerByGuice$$fd148473.CGLIB$processCall$11(<generated>)
at net.datenwerke.rs.base.server.table.TableReportUtilityServiceImpl$$EnhancerByGuice$$fd148473$$FastClassByGuice$$997eb26c.invoke(<generated>)
at com.google.inject.internal.cglib.proxy.$MethodProxy.invokeSuper(MethodProxy.java:228)
at com.google.inject.internal.InterceptorStackCallback$InterceptedMethodInvocation.proceed(InterceptorStackCallback.java:72)
at net.datenwerke.security.service.security.aop.SecurityCheckInterceptor.invoke(SecurityCheckInterceptor.java:110)
at com.google.inject.internal.InterceptorStackCallback$InterceptedMethodInvocation.proceed(InterceptorStackCallback.java:72)
at net.datenwerke.gf.service.gwtstacktrace.CatchStacktraceInterceptor.invoke(CatchStacktraceInterceptor.java:38)
at com.google.inject.internal.InterceptorStackCallback$InterceptedMethodInvocation.proceed(InterceptorStackCallback.java:72)
at com.google.inject.internal.InterceptorStackCallback.intercept(InterceptorStackCallback.java:52)
at net.datenwerke.rs.base.server.table.TableReportUtilityServiceImpl$$EnhancerByGuice$$fd148473.processCall(<generated>)
at com.google.gwt.user.server.rpc.RemoteServiceServlet.processPost(RemoteServiceServlet.java:248)
at com.google.gwt.user.server.rpc.AbstractRemoteServiceServlet.doPost(AbstractRemoteServiceServlet.java:62)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:650)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:731)
at net.datenwerke.rs.base.server.table.TableReportUtilityServiceImpl$$EnhancerByGuice$$fd148473.CGLIB$service$23(<generated>)
at net.datenwerke.rs.base.server.table.TableReportUtilityServiceImpl$$EnhancerByGuice$$fd148473$$FastClassByGuice$$997eb26c.invoke(<generated>)
at com.google.inject.internal.cglib.proxy.$MethodProxy.invokeSuper(MethodProxy.java:228)
at com.google.inject.internal.InterceptorStackCallback$InterceptedMethodInvocation.proceed(InterceptorStackCallback.java:72)
at net.datenwerke.security.service.security.aop.SecurityCheckInterceptor.invoke(SecurityCheckInterceptor.java:110)
at com.google.inject.internal.InterceptorStackCallback$InterceptedMethodInvocation.proceed(InterceptorStackCallback.java:72)
at net.datenwerke.gf.service.gwtstacktrace.CatchStacktraceInterceptor.invoke(CatchStacktraceInterceptor.java:38)
at com.google.inject.internal.InterceptorStackCallback$InterceptedMethodInvocation.proceed(InterceptorStackCallback.java:72)
at com.google.inject.internal.InterceptorStackCallback.intercept(InterceptorStackCallback.java:52)
at net.datenwerke.rs.base.server.table.TableReportUtilityServiceImpl$$EnhancerByGuice$$fd148473.service(<generated>)
at com.google.inject.servlet.ServletDefinition.doService(ServletDefinition.java:263)
at com.google.inject.servlet.ServletDefinition.service(ServletDefinition.java:178)
at com.google.inject.servlet.ManagedServletPipeline.service(ManagedServletPipeline.java:91)
at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:62)
at com.google.inject.persist.PersistFilter.doFilter(PersistFilter.java:89)
at com.google.inject.servlet.FilterDefinition.doFilter(FilterDefinition.java:163)
at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:58)
at com.google.inject.servlet.ManagedFilterPipeline.dispatch(ManagedFilterPipeline.java:118)
at com.google.inject.servlet.GuiceFilter.doFilter(GuiceFilter.java:113)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:505)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:170)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103)
at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:957)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:423)
at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1079)
at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:620)
at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:316)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalStateException: java.lang.IllegalStateException: java.io.IOException: Server returned HTTP response code: 401 for URL: http://username:password@IPofNagiosServer/check_mk/view.py?view_name=servicegroup&servicegroup=PROD&output_format=csv_export
at net.datenwerke.rs.core.service.internaldb.InternalDbServiceImpl.getConnection(InternalDbServiceImpl.java:345)
at net.datenwerke.rs.base.service.datasources.helpers.CsvToInMemoryDb.getConnection(CsvToInMemoryDb.java:132)
... 82 more
Caused by: java.lang.IllegalStateException: java.io.IOException: Server returned HTTP response code: 401 for URL: http://username:password@IPofNagiosServer/check_mk/view.py?view_name=servicegroup&servicegroup=PROD&output_format=csv_export
at net.datenwerke.rs.base.service.datasources.helpers.CsvToInMemoryDb$1.open(CsvToInMemoryDb.java:86)
at net.datenwerke.rs.core.service.internaldb.InternalDbTableCreator.openStream(InternalDbTableCreator.java:79)
at net.datenwerke.rs.core.service.internaldb.InternalDbServiceImpl.getConnection(InternalDbServiceImpl.java:326)
... 83 more
Caused by: java.io.IOException: Server returned HTTP response code: 401 for URL: http://username:password@IPofNagiosServer/check_mk/view.py?view_name=servicegroup&servicegroup=PROD&output_format=csv_export
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1627)
at java.net.URL.openStream(URL.java:1037)
at net.datenwerke.rs.base.service.datasources.connectors.UrlDatasourceConnector.getDataStream(UrlDatasourceConnector.java:38)
at net.datenwerke.rs.base.service.datasources.definitions.FormatBasedDatasourceDefinition.getDataStream(FormatBasedDatasourceDefinition.java:46)
at net.datenwerke.rs.base.service.datasources.helpers.CsvToInMemoryDb$1.open(CsvToInMemoryDb.java:79)
... 85 more
Regards
Offline
Remark:
my current workaround is to load the csv via curl triggered by cron into local filesystem and use for the datasource URL file:///pathtocsvfile
Last edited by SteffenF (2015-06-19 15:50:22)
Offline
Hi,
you are right. For HTTP Authentication one indeed needs to do some extra work on the java side which is why this is currently not working. I've added a feature request ticket for this. In the mean time, if you are not happy with the curl based solution you could also do some hack in RS: you could point the URL to a ReportServer script (scriptAccess) and have the script go and get the CSV.
Cheers
-Arno
Offline
Couple questions about CSV datasources via URL:
1) Has support for http://username:password@aaa.bbb.ccc.ddd/URL been added?
2) Lets say we get a CSV datasource via http set up so that it does not need a username and password, how do you go about testing it?
Offline
To answer my question above, it does appear that http://username:password@aaa.bbb.ccc.ddd/URL works.
Offline
Hi,
what is the optimal syntax for putting files via curl to the local RS fileserver?
What is the correct Url of a fileserver folder?
Thanks,
Christoph
Last edited by itatwork (2020-07-06 11:33:39)
Offline
Hi Christoph,
did you check the SFTP feature? https://reportserver.net/en/guides/admi … TP-Server/
If you install sftp, you can put files easily via SFTP.
Regards,
Eduardo
Offline
Hi Eduardo,
thank you, i will try this.
Cheers,
Christoph
Offline