NDS2-Client package

Joseph Areeda1John Zweizig2, Ed Maros2, Leo Singer2,

1California State University at Fullerton
2LIGO Laboratory, California Institute of Technology

Abstract

The Network Data Service (NDS2) package is a collection programs and libraries used to access LIGO data available at Tier 1 and Tier 2 clusters. Version 2 of the NDS protocol added client athentication and accesses data only from frame files. NDS2 is a proprietary TCP/IP protocol serving online and archived data from multiple servers available at various locations on the Internet.

The mailing list monitored by the NDS development team is the best place to send questions, comments, and suggestions:

The NDS2-client is distributed as several packages whose contents vary depending on the operating system. The components are:

  • Command line programs (user documentation below)

  • C programming libraries (API documentation at https://ldas-jobs.ligo.caltech.edu/~jzweizig/nds2-client-docs/

  • SWIG bindings to the C libraries for Python, Java (and Matlab), and Octave documentation is installed with the package in a location appropriate for the OS and available online at https://www.lsc-group.phys.uwm.edu/daswg/projects/nds-client/doc/manual/index.html

    • Sceintific Linux: /usr/share/doc/nds2-client/index.html
    • MacOs: /opt/local/share/doc/nds2-client/index.html
    • Ubuntu: /usr/share/doc/nds2-client/index.html
    • Windows 7: ?is it installed?

  • Compiled MEX files for Matlab and Octave

    To use these set the Matlab path to include the installation directory as returned by nds_client_config --matlabdir or nds_client_config --octavedir. There are .m files associated with each .mex file so "help function" works from the matlab or octave command window.

This document is usually installed as part of the client installation in an appropriate place for that OS

  • Sceintific Linux: /usr/share/doc/nds2-client/programs/index.html
  • MacOs: /opt/local/share/doc/nds2-client/programs/index.html
  • Ubuntu: /usr/share/doc/nds2-client/programs/index.html
  • Windows 7: No documentaiton is currently installed.

It is also available online at http://www.lsc-group.phys.uwm.edu/daswg/projects/nds-client/doc/manual/ .


Table of Contents

1. Command line programs
nds-client-config
nds2-tunnel (deprecated)
nds2_channel_source
nds_query
NDS2Frame

Chapter 1. Command line programs

nds-client-config


                nds-client-config [option]
                
                nds-client-config accepts the following arguments: 
                --help          Show the options
                --includedir    Include directory for (C headers)
                --javaclasspath Java class directory
                --libdir        Library directory (for linking)
                --matlabdir     Matlab function library path
                --octavedir     Octave function library path
                --pythondir     Python sources
                --pyexecdir     Python executable libraries
                --version       This nds2-client version
            

This commands displays all the important paths specified when the programs and libraries were compiled. It is installed so as to be available to running programs and is used to set environment variables or to find elements of the package.

One thing to be aware of is that on some systems such as MacOS, when a program is launched from the Window Manager (Finder or the Dock) the executable path is not set. If the programs are not installed in the system directories on Linux

nds2-tunnel (deprecated)

nds2-tunnel <hostt> [<portt>]

A convenience function for opening an ssh tunnel to an open NDS2 server which only listens on localhost. If no port is given the standard 31200 is used for port numbeer.

Example:

If an authenticated server is installed at ldas-pcdev4.ligo.caltech.edu and you can ssh into that machine entering ns2-tunnel ldas-pcdev4.ligo.caltech.edu will allow you to use localhost as the host address for subsequent nds2 programs or library calls.

This utility has been deprecated and may be removed in a future release.

nds2_channel_source


                nds2_channel_source --help
                Command syntax: 
                nds_channel_source [-n <servert>] [-p <portt>
                        [-s <start-gps>] channel-list
                                where: 
                                -n      Specify server IP address
                                -p      Specify port number [31200]
                                -s      Specify start time (as GPS)
                                -v      Verbose printout (dump channel data)
                                <channel-list> Space delimited channel list
            

nds2_channel_source reports the times of the appropriate frames that should contain the channel data specified. For efficiency reasons, the NDS2 server does not confirm that the channel actually has data in all the relevant frames.

Example:


                $ nds2_channel_source -n nds.ligo.caltech.edu -s $(tconvert 10/25/13) H1:PSL-ISS_PDA_OUT_DQ
                Requested source list: 
                H1:PSL-ISS_PDA_OUT_DQ {H-H1_R:1053900000-inf}
            

The output indicates that the H1:PSL-ISS_PDA_OUT_DQ is most likely avaialable on 25-OCT-2013 on the NDS2 server at Caltech with data in hte H-H!_R frames. The inf indicator at the end signifies the channel is still being acquired.

nds_query


                Command syntax: 
                nds_query [--version] [--server-version] [--server-protocol]
                [--fast-fetch]
                [-g] [-l] [-n <server>] [-p <port>]
                        [-s <start-gps>] [-e <end-gps>
                        [-d <delta-t>][<channel-list>]
                                    
                        where: 
                        -d      Time stride [2]
                        -e      End gps time [start_gps + delta]
                        -g      Request a list of channel groups (NDS1 only) -- currently disabled
                        -l      List available channels
                        -n      Specify server IP address
                        -p      Specify port number [31200]
                        -s      Specify start time (as GPS)
                        -t      Specify default channel type.
                        -v      Verbose printout (dump channel data)
                        -1      NDS1 protocol
                        -2      NDS2 protocol
                        --fast-fetch       optimize channel data retrieval
                        --sample-format    printf format for data dump
                        --samples-per-line samples dumped per line
                        --version          print version of client and exit
                        --server-version   print version of server and exit
                        --server-protocol  print protocol used by the server and exit
                        <channel-list> Space delimited channel list
                            If the protocol is not specified, nds_query tries
                            nds2 protocol, and if this fails, reverts to nds1
                            protocol
            

nds_query is a powerful tool that brings most of the features of the NDS2 client library to the command line. It is used most often to verify that the server and client are working properly.

To use any nds2-client with a protected server the user must have a valid Kerberos ticket. See (?) for a brief introduction to using Kerberos.

Example: nds_query -l -n nds.ligo.caltech.edu -t reduced | head

This command will query the nds2 server at Caltech for a list of reduced data set channels and print the first few from the list.

Example: nds_query -n nds.ligo-wa.caltech.edu -s 1064620816 -e 1064620826 H1:PSL-ISS_PDA_OUT_DQ

The above command will print a summary of 10 seconds of data for the specified channel from the NDS2 server at the Hanford Observatory. If the verbose switch (-v) is used all the data will be printed as ascii values. The output looks like:


                    Data for 2 seconds starting at GPS: 1064620816
                    Channel                                  type    nWords
                    H1:PSL-ISS_PDA_OUT_DQ                    real_4  65536
                    Data for 2 seconds starting at GPS: 1064620818
                    Channel                                  type    nWords
                    H1:PSL-ISS_PDA_OUT_DQ                    real_4  65536
                    Data for 2 seconds starting at GPS: 1064620820
                    Channel                                  type    nWords
                    H1:PSL-ISS_PDA_OUT_DQ                    real_4  65536
                    Data for 2 seconds starting at GPS: 1064620822
                    Channel                                  type    nWords
                    H1:PSL-ISS_PDA_OUT_DQ                    real_4  65536
                    Data for 2 seconds starting at GPS: 1064620824
                    Channel                                  type    nWords
                    H1:PSL-ISS_PDA_OUT_DQ                    real_4  65536
                

NDS2Frame


                NDS2Frame [--server <server-ip>] [--port <port>] [--online] [--help] 
                    [--start-gps <start-gps>] [--end-gps <end-gps>] [--stride <stride>
                    [--out-frame <file-name>] [--debug <debug-lvl>] [--compress <c-mode>]
                    <channel-1> [... <channel-n>]

            <server-ip>  Server address:port [localhost:31200]
            <port>       Server port [0]
            --online     Get online data
            --help       Print this message.
            <start-gps>  Start GPS time (ignored in online requests)
            <end-gps>    End GPS time (number of data seconds online)
            <stride>     Number of seconds per request (frame length)
            <file-name>  Output file name template [NDS2Frame-%s-%1n.gwf]
            <debug-lvl>  Debug print level
            <c-mode>     Compression mode: number, "gzip", "gzip-differential", 
                         or "gzip-or-zero-suppress"
            <channel-1> ... <channel-n>  Channels to be written to frame
            

NDS2Frame is not installed with the nds2-client package. It is part of the gds-core package but uses the NDS2 client to retrieve data from an NDS server and write the results in a frame file. This can be very convenient for making local copies of data in the standard LIGO format. The frame contains only channels of interest and one frame can cover all data needed for processing.