Difference between revisions of "HOWTO get ns-3 data into SciPy"

From Nsnam
Jump to: navigation, search
(Ways to analyze data)
(Ways to visualize data)
Line 201: Line 201:
  
 
= Ways to analyze data =
 
= Ways to analyze data =
 
= Ways to visualize data =
 
  
 
= Ways to analyze data =
 
= Ways to analyze data =
 
= Ways to visualize data =
 
= Ways to visualize data =

Revision as of 16:50, 27 January 2011

Ways to get data into the framework

Files

Text Files

Use numpy.loadtxt to load arrays from text files.

Here is a description of numpy.loadtxt from its help message:

  loadtxt(fname, dtype=<type 'float'>, comments='#', delimiter=None, converters=None, skiprows=0, usecols=None, unpack=False)
      Load data from a text file.
      
      Each row in the text file must have the same number of values.
      
      Parameters
      ----------
      fname : file or string
          File or filename to read.  If the filename extension is ``.gz`` or
          ``.bz2``, the file is first decompressed.
      dtype : data-type
          Data type of the resulting array.  If this is a record data-type,
          the resulting array will be 1-dimensional, and each row will be
          interpreted as an element of the array.   In this case, the number
          of columns used must match the number of fields in the data-type.
      comments : string, optional
          The character used to indicate the start of a comment.
      delimiter : string, optional
          The string used to separate values.  By default, this is any
          whitespace.
      converters : {}
          A dictionary mapping column number to a function that will convert
          that column to a float.  E.g., if column 0 is a date string:
          ``converters = {0: datestr2num}``. Converters can also be used to
          provide a default value for missing data:
          ``converters = {3: lambda s: float(s or 0)}``.
      skiprows : int
          Skip the first `skiprows` lines.
      usecols : sequence
          Which columns to read, with 0 being the first.  For example,
          ``usecols = (1,4,5)`` will extract the 2nd, 5th and 6th columns.
      unpack : bool
          If True, the returned array is transposed, so that arguments may be
          unpacked using ``x, y, z = loadtxt(...)``
      
      Returns
      -------
      out : ndarray
          Data read from the text file.
      
      See Also
      --------
      scipy.io.loadmat : reads Matlab(R) data files
      
      Examples
      --------
      >>> from StringIO import StringIO   # StringIO behaves like a file object
      >>> c = StringIO("0 1\n2 3")
      >>> np.loadtxt(c)
      array([[ 0.,  1.],
             [ 2.,  3.]])
      
      >>> d = StringIO("M 21 72\nF 35 58")
      >>> np.loadtxt(d, dtype={'names': ('gender', 'age', 'weight'),
      ...                      'formats': ('S1', 'i4', 'f4')})
      array([('M', 21, 72.0), ('F', 35, 58.0)],
            dtype=[('gender', '|S1'), ('age', '<i4'), ('weight', '<f4')])
      
      >>> c = StringIO("1,0,2\n3,0,4")
      >>> x,y = np.loadtxt(c, delimiter=',', usecols=(0,2), unpack=True)
      >>> x
      array([ 1.,  3.])
      >>> y
      array([ 2.,  4.])

Note that Comma Separated Value (CSV) files can be read by specifying the comma character as the delimiter for numpy.loadtxt.

NetCDF Files

SciPy can handle Network Common Data Form (NetCDF) files, which use a self-describing, machine-independent data format that supports the creation, access, and sharing of array-oriented scientific data.

NetCDF data is:

  • Self-Describing. A netCDF file includes information about the data it contains.
  • Portable. A netCDF file can be accessed by computers with different ways of storing integers, characters, and floating-point numbers.
  • Scalable. A small subset of a large dataset may be accessed efficiently.
  • Appendable. Data may be appended to a properly structured netCDF file without copying the dataset or redefining its structure.
  • Sharable. One writer and multiple readers may simultaneously access the same netCDF file.
  • Archivable. Access to all earlier forms of netCDF data will be supported by current and future versions of the software.

An example NetCDF file follows:

    netcdf sfc_pres_temp {
    dimensions:
    	latitude = 6 ;
    	longitude = 12 ;
    variables:
    	float latitude(latitude) ;
    		latitude:units = "degrees_north" ;
    	float longitude(longitude) ;
    		longitude:units = "degrees_east" ;
    	float pressure(latitude, longitude) ;
    		pressure:units = "hPa" ;
    	float temperature(latitude, longitude) ;
    		temperature:units = "celsius" ;
    data:
    
     latitude = 25, 30, 35, 40, 45, 50 ;
    
     longitude = -125, -120, -115, -110, -105, -100, -95, -90, -85, -80, -75, -70 ;
    
     pressure =
      900, 906, 912, 918, 924, 930, 936, 942, 948, 954, 960, 966,
      901, 907, 913, 919, 925, 931, 937, 943, 949, 955, 961, 967,
      902, 908, 914, 920, 926, 932, 938, 944, 950, 956, 962, 968,
      903, 909, 915, 921, 927, 933, 939, 945, 951, 957, 963, 969,
      904, 910, 916, 922, 928, 934, 940, 946, 952, 958, 964, 970,
      905, 911, 917, 923, 929, 935, 941, 947, 953, 959, 965, 971 ;
    
     temperature =
      9, 10.5, 12, 13.5, 15, 16.5, 18, 19.5, 21, 22.5, 24, 25.5,
      9.25, 10.75, 12.25, 13.75, 15.25, 16.75, 18.25, 19.75, 21.25, 22.75, 24.25,
        25.75,
      9.5, 11, 12.5, 14, 15.5, 17, 18.5, 20, 21.5, 23, 24.5, 26,
      9.75, 11.25, 12.75, 14.25, 15.75, 17.25, 18.75, 20.25, 21.75, 23.25, 24.75,
        26.25,
      10, 11.5, 13, 14.5, 16, 17.5, 19, 20.5, 22, 23.5, 25, 26.5,
      10.25, 11.75, 13.25, 14.75, 16.25, 17.75, 19.25, 20.75, 22.25, 23.75,
        25.25, 26.75 ;
    }

MATLAB Data Files

SciPy can handle MATLAB format data files.

See the MATLAB web site for details on MATLAB format files.

Matrix Market Files

SciPy can handle Matrix Market (MM) format files, which are a set of human readable, ASCII-based file formats designed to facilitate the exchange of matrix data.

If you had the following sparse matrix,

            1    0      0       6      0     
            0   10.5    0       0      0     
            0    0    .015      0      0     
            0  250.5    0     -280    33.32  
            0    0      0       0     12     

then it would be represented as follows:

  %%MatrixMarket matrix coordinate real general
  %=================================================================================
  %
  % This ASCII file represents a sparse MxN matrix with L 
  % nonzeros in the following Matrix Market format:
  %
  % +----------------------------------------------+
  % |%%MatrixMarket matrix coordinate real general | <--- header line
  % |%                                             | <--+
  % |% comments                                    |    |-- 0 or more comment lines
  % |%                                             | <--+         
  % |    M  N  L                                   | <--- rows, columns, entries
  % |    I1  J1  A(I1, J1)                         | <--+
  % |    I2  J2  A(I2, J2)                         |    |
  % |    I3  J3  A(I3, J3)                         |    |-- L lines
  % |        . . .                                 |    |
  % |    IL JL  A(IL, JL)                          | <--+
  % +----------------------------------------------+   
  %
  % Indices are 1-based, i.e. A(1,1) is the first element.
  %
  %=================================================================================
    5  5  8
      1     1   1.000e+00
      2     2   1.050e+01
      3     3   1.500e-02
      1     4   6.000e+00
      4     2   2.505e+02
      4     4  -2.800e+02
      4     5   3.332e+01
      5     5   1.200e+01

Databases

SciPy can interface with PyTables, a hierarchical database package designed to efficiently manage large amounts of data using HDF5.

Pipes

Scipy does not have any direct support for pipes.

Python, however, can read the standard output of external pipes and write to standard input for other pipes to use. So, you could use the Python code that calls scipy to interact with pipes in the normal way.

Sockets

SciPy does not have any direct support for sockets.

Python, however, does with the standard socket module. So, you could use Python to interact with a socket using the standard Python socket module.

Ways to visualize data

Ways to analyze data

Ways to analyze data

Ways to visualize data