TUESDAY, FEBRUARY 07, 2012

Exporting a Report Directly to FTP

Scheduled Data Extract

S

ervice-now provides numerous ways of getting data into the system. The import set functionality supports pulling data in from an attachment, FTP(S) servers, JDBC data sources, etc. However, a common problem that many customers have is trying to get data out of their SN instance and into their data center. The simplest way to handle this currently is by having a report emailed to you on a scheduled basis. There has to be a better way!

Scheduled Data Extract ModulesThe good news is that there is a better way. The “Scheduled Data Extract” update set has been designed to allow a user to have reports, tables, or table views sent to a specific directory on a local mid server or FTP to a customer’s FTP/FTPS server on a scheduled basis. The “Scheduled Data Extract” update set has been battle-tested and is used by several large Service-now customers today.

On installation of the Data Extract update set, two new modules will be present within the Reports Application: ‘Scheduled Data Extract’ and ‘Data Extract History’.

How it works

Scenario #1:

Let’s say you want to have all of your users exported from Service-now to a particular computer in your corporate intranet. In this case you want it to end up on a generic Windows XP machine in a certain directory everyday at 12 AM. Once you’ve installed the scheduled data extract, you simply fill out the form in the following manner.

Scheduled Data Extract MID Server

Most of these fields should be self-explanatory.  The MID Server field asks which mid server you would like to have handle this request (Note that the ‘MID Server’ field will not be visible until you have installed the MID Server plugin as documented below).  In this scenario, this field should be set to the mid server that’s running on the machine that will be receiving the export from Service-now.  The Append timestamp field should be checked if you do not want to overwrite the file on your Windows XP machine each time Service-now exports its user table.

Scenario #2:

You want to export an incident report to an FTP server every Wednesday at 1 PM.  This report should be in CSV format.  Since the incident table contains so many fields, you want to limit the export to only include the fields that you’re interested in.  This screenshot represents the solution.  Notice the View field.  This allows you to specify which view you want to use.  It is by defining the view that you’re able to specify which fields should be exported.

Scheduled Data Extract FTP

Notice that the MID Server field is still a mandatory field.  This is because the mid server is what handles the grunt work to get the data from Service-now and FTPs it to your FTP/FTPS server.  The big advantage of this is that you do not need to open/modify ports to allow Service-now to send data to your FTP server.  It can be a local FTP server that you use internally but do not want to open up to the outside world.  The mid server is required to actually perform the transport of the exported file to your FTP location.

The ‘Scheduled Data Extract’ update set supports several secure FTP methods.  One method that is not supported is SFTP, which commonly confused with FTPS.

Although it is possible to use this export functionality to export large numbers of records (500K+) from your Service-now instance, you obviously should not do so without doing some performance testing first. Any large query against any database has the potential to adversely affect the performance of that database.

Prerequisites:

Related Links:

Comments

Posted On
Dec 08, 2011
Posted By
Jacob Andersen

Please update your SNFilePoster mid server script include with the latest code to make sure that we’re debugging a true problem. You can find the latest code here: http://www.servicenowguru.com/scheduled-file-poster-code/

Please update me with your results. Thanks!

Posted On
Dec 13, 2011
Posted By
pmendoza

I actually didn’t have the SNFilePoster script in my instance, but added the updated file you provided. I am still getting the error, however. Let me know if you need me to test something else. Thanks!

Posted On
Dec 13, 2011
Posted By
Jacob Andersen

Hmm, if you didn’t even have that script in your instance, then I imagine there is a problem altogether with this in your instance. I would suggest reapplying the latest update set. I just added a few things this update set this morning, so please reapply and re-test. If it still does not work, I’d be happy to take a look at your instance.

Posted On
Dec 14, 2011
Posted By
pmendoza

I applied the new update set, but I am still receiving the error.

Posted On
Dec 14, 2011
Posted By
Jacob Andersen

What is your SNC instance? I’ll hop on and take a look. I’ll also delete your post after I get on your instance so that others don’t see.

Thanks!

Posted On
Dec 14, 2011
Posted By
pmendoza

fxcmdev, thank you so much!

Posted On
Dec 15, 2011
Posted By
Jacob Andersen

pmendoza, I’m on the road this week and am away from my main computer. I took a quick look at your instance and cannot see right off-hand what the root problem is. I will continue to look at this on Monday.

Thanks!

Posted On
Dec 16, 2011
Posted By
pmendoza

I understand, thank you for taking the time in the first place! Have you been able to test the update set with Aspen yet?

Posted On
Jan 06, 2012
Posted By
brumiou

Hi,

I try to export a report in csv to a ftp server.
The file is created, but contains an Apache error :
This request requires HTTP authentication ().

Have you any idea where it comes from?

thanks a lot!

Posted On
Jan 25, 2012
Posted By
brumiou

Hi,

I still have the issue, and I checked, and as for pmendoza, I don’t have the SNFilePoster in the mid server script includes.

I add it, and updated the SNDataRetriever, but it doesn”t change anything.

If the issue was solved for pmendoza, maybe you can explain to me how you did? or what’s the solution?

Thanks a lot!

rgds

Posted On
Jan 25, 2012
Posted By
brumiou

Sorry to bother you again…

I think I discovered something…
first of all I forgot to tell you that we are on premise, so we are not hosted by service-now.
In the logs, I can see :
Probe: JavascriptProbe:SNDataRetriever *** Script: report: https://scrdev01.service-now.com/sys_report_template.do?jvar_report_id=51822452ff8210002098dfffffffff95&CSV
host: https://scrdev01.service-now.com/

but the adress is not correct!

Do you know where I can change this?
it’s not https but http, and it’s not neither zzzz.service-now.com.

Thanks for your time

rgds

Xavier

Posted On
Jan 26, 2012
Posted By
Jacob Andersen

You can modify how the instance name is defined by going to the sendDataExtractProbe business rule. The 3rd line in the script contains this code:

var host = gs.getProperty(“glide.servlet.uri”);

You can change it to whatever suits your environment. Good luck!

Posted On
Jan 27, 2012
Posted By
brumiou

Hi,

thanks for the quick reply!

I looked at the system property glide.servlet.uri, and it was already set correctly (http://scrdev01.solaris.nbb:16001/). So it seems that the SNDataRetriever use another property?
Maybe based on the instance name, and then a hardcoded value somewhere?

So for your last question, yes, I still have the problem :(

Posted On
Jan 26, 2012
Posted By
Jacob Andersen

Do you still have this problem? From your latest post, to looks like you got past that issue. Thanks!

Posted On
Feb 06, 2012
Posted By
Matt Haak

Is it possible to use this with the local Mid Server (mid.server.localhost) It appears from this community post that it should be possible (see Jacob Andersen’s comments) http://community.service-now.com/forum/1719?page=1 However, after installing your ScheduledDataExtract_2_6.xml it appears there’s a reference qualifier on the Mid Server lookup that specifically excludes localhost. I tried taking that off and doing an extract anyway but it appears it does’t work. Can you shed some light on this?

Posted On
Feb 06, 2012
Posted By
Mark Stanger

Due to some ServiceNow limitations, the localhost MID server option had to be removed.

Leave a Reply


Notify me of followup comments via e-mail. You can also subscribe without commenting.

Latest Comments

  • Mark Stanger: This linkage all happens for you if you use the task survey plugin. You can look on the wiki for more...
  • Vineeth: I want a way in which if a survey is filled in by the user the response are stored in the survey response...
  • Mark Stanger: This functionality doesn’t connect to an FTP server. See this line in the post above…...
  • Mark Stanger: The report page is back-end XML so there’s no way to directly manipulate the behavior of that...