schain-jc Commit - r1125:ba0b6ae662be

merge 2.3

José Chávez -

r1125:ba0b6ae662be

parent child

Context file:

r1125:ba0b6ae662be

Collapse all files

Dockerfile created 644 +26 0

@@ -0,0 +1,26
	1	FROM python:2.7-slim
	2
	3	RUN apt-get clean && apt-get update && apt-get install -y --no-install-recommends \
	4	git \
	5	gcc \
	6	libpng-dev \
	7	libfreetype6-dev \
	8	libopenblas-dev \
	9	liblapack-dev \
	10	libatlas-base-dev \
	11	libssl-dev \
	12	libhdf5-dev \
	13	&& git clone --branch v2.3 --depth 1 \
	14	http://jro-dev.igp.gob.pe/rhodecode/schain \
	15	&& pip install numpy \
	16	&& cd schain \
	17	&& pip install . \
	18	&& rm -rf * \
	19	&& apt-get purge -y --auto-remove git gcc \
	20	&& rm -rf /var/lib/apt/lists/*
	21
	22	ENV BACKEND="Agg"
	23
	24	VOLUME /data
	25
	26	ENTRYPOINT ["schain"]

README.md +17 -15

             # Signal Chain
             ## Introduction
             Signal Chain (SCh) is a radar data processing library developed using [Python](www.python.org) at JRO. SCh provides modules to read, write, process and plot data.
             ## Installation
-            Install system dependencies, clone the latest version from [git](http://jro-dev.igp.gob.pe/rhodecode/schain/) and install it as a normal python package.
+            Install system dependencies, clone the latest version from [here](http://jro-dev.igp.gob.pe/rhodecode/schain/) and install it as a normal python package.
             ### Linux based system
             ```
             $ sudo apt-get install python-pip python-dev gfortran libpng-dev freetype* libblas-dev liblapack-dev libatlas-base-dev python-qt4 python-tk libssl-dev libhdf5-dev
             $ git clone http://jro-dev.igp.gob.pe/rhodecode/schain/
             $ cd schain
             $ sudo pip install ./
             ```
-            **It is recommended to install schain in a virtual environment**
-            ```
-            $ sudo pip install virtualenv
-            $ virtualenv /path/to/virtual --system-site-packages
-            $ source /path/to/virtual/bin/activate
-            (virtual) $ cd schain
-            (virtual) $ pip install ./
-            ```
             ### MAC Os
             ```
+            $ brew install python
             $ brew install cartr/qt4/pyqt
             $ git clone http://jro-dev.igp.gob.pe/rhodecode/schain/
             $ cd schain
             $ pip install ./
             ```
-            if ```pip install ./``` does not work, install a proper python enviroment, and repeat the steps.
+            **It is recommended to install schain in a virtual environment**
             ```
-            $ brew install python
+            $ virtualenv /path/to/virtual
+            $ source /path/to/virtual/bin/activate
+            (virtual) $ cd schain
+            (virtual) $ pip install ./
+            (virtual) $ bash link_PyQt4.sh
             ```
-            ### GUI Installation
+            ### Docker
+            Download Dockerfile from the repository, and create a docker image
             ```
-            $ sudo apt-get install python-pip python-dev gfortran libpng-dev freetype* libblas-dev liblapack-dev libatlas-base-dev python-qt4 python-tk libssl-dev libhdf5-dev
+            $ docker build -t schain .
-            $ (virtual) bash link_PyQt4.sh
             ```
+            You can run a container using an xml file or a schain script also you need to mount a volume for the data input and for the output files/plots
+            ```
+            $ docker run -it --rm --volume /path/to/host/data:/data schain xml /data/test.xml
+            $ docker run -it --rm --volume /path/to/host/data:/data --entrypoint=/bin/python schain /data/test.py
+            ```
             ## First Script
             Read Spectra data (.pdata) - remove dc - plot spectra & RTI
             Import SCh and creating a project
             ```python
             #!/usr/bin/python
             from schainpy.controller import Project
             controller = Project()
             controller.setup(id = '100',
                              name='test',
                              description='Basic experiment')
             ```
             Adding read unit and operations
             ```python
             read_unit = controller.addReadUnit(datatype='Spectra',
                                                path='/path/to/pdata/',
                                                startDate='2014/01/31',
                                                endDate='2014/03/31',
                                                startTime='00:00:00',
                                                endTime='23:59:59',
                                                online=0,
                                                walk=0)
             proc_unit = controller.addProcUnit(datatype='Spectra',
                                                inputId=read_unit.getId())
             op = proc_unit.addOperation(name='selectChannels')
             op.addParameter(name='channelList', value='0,1', format='intlist')
             op = proc_unit.addOperation(name='selectHeights')
             op.addParameter(name='minHei', value='80', format='float')
             op.addParameter(name='maxHei', value='200', format='float')
             op = proc_unit.addOperation(name='removeDC')
             ```
             Plotting data & start project
             ```python
             op = proc_unit.addOperation(name='SpectraPlot', optype='other')
             op.addParameter(name='id', value='1', format='int')
             op.addParameter(name='wintitle', value='Spectra', format='str')
             op = procUnitConfObj1.addOperation(name='RTIPlot', optype='other')
             op.addParameter(name='id', value='2', format='int')
             op.addParameter(name='wintitle', value='RTI', format='str')
             controller.start()
             ```
             Full script
             ```python
             #!/usr/bin/python
             from schainpy.controller import Project
             controller = Project()
             controller.setup(id = '100',
                              name='test',
                              description='Basic experiment')
             read_unit = controller.addReadUnit(datatype='Spectra',
                                                path='/path/to/pdata/',
                                                startDate='2014/01/31',
                                                endDate='2014/03/31',
                                                startTime='00:00:00',
                                                endTime='23:59:59',
                                                online=0,
                                                walk=0)
             proc_unit = controller.addProcUnit(datatype='Spectra',
                                                inputId=read_unit.getId())
             op = proc_unit.addOperation(name='selectChannels')
             op.addParameter(name='channelList', value='0,1', format='intlist')
             op = proc_unit.addOperation(name='selectHeights')
             op.addParameter(name='minHei', value='80', format='float')
             op.addParameter(name='maxHei', value='200', format='float')
             op = proc_unit.addOperation(name='removeDC')
             op = proc_unit.addOperation(name='SpectraPlot', optype='other')
             op.addParameter(name='id', value='6', format='int')
             op.addParameter(name='wintitle', value='Spectra', format='str')
             op = procUnitConfObj1.addOperation(name='RTIPlot', optype='other')
             op.addParameter(name='id', value='2', format='int')
             op.addParameter(name='wintitle', value='RTI', format='str')
             controller.start()
             ```

link_PyQt4.sh +5 -7

             #!/bin/bash
             # This hook is run after a new virtualenv is activated.
-            # ~/.virtualenvs/postmkvirtualenv
-            libs=( PyQt4 sip.so )
             python_version=python$(python -c "import sys; print (str(sys.version_info[0])+'.'+str(sys.version_info[1]))")
             var=( $(which -a $python_version) )
             get_python_lib_cmd="from distutils.sysconfig import get_python_lib; print (get_python_lib())"
             lib_virtualenv_path=$(python -c "$get_python_lib_cmd")
             lib_system_path=$(${var[-1]} -c "$get_python_lib_cmd")
+            sip_path=$(ls $lib_system_path/sip*.so)
-            for lib in ${libs[@]}
+            echo "Linking Qt4..."
-            do
+            ln -s $lib_system_path/PyQt4 $lib_virtualenv_path/PyQt4
-                ln -s $lib_system_path/$lib $lib_virtualenv_path/$lib
+            echo "Linking SIP..."
-            done
+            ln -s $sip_path $lib_virtualenv_path/sip.so

schainpy/cli/README.md ~~schaincli/README.md~~ renamed 0 0

NO CONTENT: file renamed from schaincli/README.md to schainpy/cli/README.md

schainpy/cli/__init__.py ~~schaincli/__init__.py~~ renamed 0 0

NO CONTENT: file renamed from schaincli/__init__.py to schainpy/cli/__init__.py

schainpy/cli/cli.py ~~schaincli/cli.py~~ renamed +1 -1

             import click
             import schainpy
             import subprocess
             import os
             import sys
             import glob
             save_stdout = sys.stdout
             sys.stdout = open('trash', 'w')
             from multiprocessing import cpu_count
-            from schaincli import templates
             from schainpy.controller import Project
             from schainpy.model import Operation, ProcessingUnit
             from schainpy.utils import log
             from importlib import import_module
             from pydoc import locate
             from fuzzywuzzy import process
             from schainpy.utils import paramsFinder
+            import templates
             sys.stdout = save_stdout
             def print_version(ctx, param, value):
                 if not value or ctx.resilient_parsing:
                     return
                 click.echo(schainpy.__version__)
                 ctx.exit()
             cliLogger = log.makelogger('schain cli')
             PREFIX = 'experiment'
             @click.command()
             @click.option('--version', '-v', is_flag=True, callback=print_version, help='SChain version', type=str)
             @click.argument('command', default='run', required=True)
             @click.argument('nextcommand', default=None, required=False, type=str)
             def main(command, nextcommand, version):
                 """COMMAND LINE INTERFACE FOR SIGNAL CHAIN - JICAMARCA RADIO OBSERVATORY \n
                     Available commands.\n
                     --xml: runs a schain XML generated file\n
                     run: runs any python script starting 'experiment_'\n
                     generate: generates a template schain script\n
                     search: return avilable operations, procs or arguments of the give operation/proc\n"""
                 if command == 'xml':
                     runFromXML(nextcommand)
                 elif command == 'generate':
                     generate()
                 elif command == 'test':
                     test()
                 elif command == 'run':
                     runschain(nextcommand)
                 elif command == 'search':
                     search(nextcommand)
                 else:
                     log.error('Command {} is not defined'.format(command))
             def check_module(possible, instance):
                 def check(x):
                     try:
                         instancia = locate('schainpy.model.{}'.format(x))
                         return isinstance(instancia(), instance)
                     except Exception as e:
                         return False
                 clean = clean_modules(possible)
                 return [x for x in clean if check(x)]
             def clean_modules(module):
                 noEndsUnder = [x for x in module if not x.endswith('__')]
                 noStartUnder = [x for x in noEndsUnder if not x.startswith('__')]
                 noFullUpper = [x for x in noStartUnder if not x.isupper()]
                 return noFullUpper
             def search(nextcommand):
                 if nextcommand is None:
                     log.error('There is no Operation/ProcessingUnit to search')
                 elif nextcommand == 'procs':
                     procs = paramsFinder.getProcs()
                     log.success(
                         'Current ProcessingUnits are:\n\033[1m{}\033[0m'.format('\n'.join(procs)))
                 elif nextcommand == 'operations':
                     operations = paramsFinder.getOperations()
                     log.success('Current Operations are:\n\033[1m{}\033[0m'.format(
                         '\n'.join(operations)))
                 else:
                     try:
                         args = paramsFinder.getArgs(nextcommand)
                         log.warning(
                             'Use this feature with caution. It may not return all the allowed arguments')
                         if len(args) == 0:
                             log.success('{} has no arguments'.format(nextcommand))
                         else:
                             log.success('Showing {} arguments:\n\033[1m{}\033[0m'.format(
                                 nextcommand, '\n'.join(args)))
                     except Exception as e:
                         log.error('Module {} does not exists'.format(nextcommand))
                         allModules = paramsFinder.getAll()
                         similar = process.extractOne(nextcommand, allModules)[0]
                         log.success('Showing {} instead'.format(similar))
                         search(similar)
             def runschain(nextcommand):
                 if nextcommand is None:
                     currentfiles = glob.glob('./{}_*.py'.format(PREFIX))
                     numberfiles = len(currentfiles)
                     if numberfiles > 1:
                         log.error('There is more than one file to run')
                     elif numberfiles == 1:
                         subprocess.call(['python ' + currentfiles[0]], shell=True)
                     else:
                         log.error('There is no file to run')
                 else:
                     try:
                         subprocess.call(['python ' + nextcommand], shell=True)
                     except Exception as e:
                         log.error("I cannot run the file. Does it exists?")
             def basicInputs():
                 inputs = {}
                 inputs['desc'] = click.prompt(
                     'Enter a description', default="A schain project", type=str)
                 inputs['name'] = click.prompt(
                     'Name of the project', default="project", type=str)
                 inputs['path'] = click.prompt('Data path', default=os.getcwd(
                 ), type=click.Path(exists=True, resolve_path=True))
                 inputs['startDate'] = click.prompt(
                     'Start date', default='1970/01/01', type=str)
                 inputs['endDate'] = click.prompt(
                     'End date', default='2017/12/31', type=str)
                 inputs['startHour'] = click.prompt(
                     'Start hour', default='00:00:00', type=str)
                 inputs['endHour'] = click.prompt('End hour', default='23:59:59', type=str)
                 inputs['figpath'] = inputs['path'] + '/figs'
                 return inputs
             def generate():
                 inputs = basicInputs()
                 inputs['multiprocess'] = click.confirm('Is this a multiprocess script?')
                 if inputs['multiprocess']:
                     inputs['nProcess'] = click.prompt(
                         'How many process?', default=cpu_count(), type=int)
                     current = templates.multiprocess.format(**inputs)
                 else:
                     current = templates.basic.format(**inputs)
                 scriptname = '{}_{}.py'.format(PREFIX, inputs['name'])
                 script = open(scriptname, 'w')
                 try:
                     script.write(current)
                     log.success('Script {} generated'.format(scriptname))
                 except Exception as e:
                     log.error('I cannot create the file. Do you have writing permissions?')
             def test():
                 log.warning('testing')
             def runFromXML(filename):
                 controller = Project()
                 if not controller.readXml(filename):
                     return
                 controller.start()
                 return

schainpy/cli/templates.py ~~schaincli/templates.py~~ renamed 0 0

NO CONTENT: file renamed from schaincli/templates.py to schainpy/cli/templates.py

schainpy/cli/tests/__init__.py ~~schaincli/tests/__init__.py~~ renamed 0 0

NO CONTENT: file renamed from schaincli/tests/__init__.py to schainpy/cli/tests/__init__.py

schainpy/cli/tests/test_cli.py ~~schaincli/tests/test_cli.py~~ renamed +1 -1

             import pytest
             from click.testing import CliRunner
-            from schaincli import cli
+            from schainpy.cli import cli
             @pytest.fixture
             def runner():
                 return CliRunner()
             def test_cli(runner):
                 result = runner.invoke(cli.main)
                 assert result.exit_code == 0
                 assert not result.exception
                 assert result.output.strip() == 'Hello, world.'
             def test_cli_with_option(runner):
                 result = runner.invoke(cli.main, ['--as-cowboy'])
                 assert not result.exception
                 assert result.exit_code == 0
                 assert result.output.strip() == 'Howdy, world.'
             def test_cli_with_arg(runner):
                 result = runner.invoke(cli.main, ['Jicamarca'])
                 assert result.exit_code == 0
                 assert not result.exception
                 assert result.output.strip() == 'Hello, Jicamarca.'

schainpy/controller.py +10 -4

             '''
             Created on September , 2012
             @author:
             '''
             import sys
             import ast
             import datetime
             import traceback
             import math
             import time
             from multiprocessing import Process, cpu_count
             from xml.etree.ElementTree import ElementTree, Element, SubElement, tostring
             from xml.dom import minidom
             import schainpy
             import schainpy.admin
             from schainpy.model import *
             from schainpy.utils import log
             DTYPES = {
                 'Voltage': '.r',
                 'Spectra': '.pdata'
             }
             def MPProject(project, n=cpu_count()):
                 '''
                 Project wrapper to run schain in n processes
                 '''
                 rconf = project.getReadUnitObj()
                 op = rconf.getOperationObj('run')
                 dt1 = op.getParameterValue('startDate')
                 dt2 = op.getParameterValue('endDate')
+                tm1 = op.getParameterValue('startTime')
+                tm2 = op.getParameterValue('endTime')
                 days = (dt2 - dt1).days
                 for day in range(days + 1):
                     skip = 0
                     cursor = 0
                     processes = []
                     dt = dt1 + datetime.timedelta(day)
                     dt_str = dt.strftime('%Y/%m/%d')
                     reader = JRODataReader()
                     paths, files = reader.searchFilesOffLine(path=rconf.path,
                                                              startDate=dt,
                                                              endDate=dt,
+                                                             startTime=tm1,
+                                                             endTime=tm2,
                                                              ext=DTYPES[rconf.datatype])
                     nFiles = len(files)
                     if nFiles == 0:
                         continue
                     skip = int(math.ceil(nFiles / n))
                     while nFiles > cursor * skip:
                         rconf.update(startDate=dt_str, endDate=dt_str, cursor=cursor,
                                      skip=skip)
                         p = project.clone()
                         p.start()
                         processes.append(p)
                         cursor += 1
                     def beforeExit(exctype, value, trace):
                         for process in processes:
                             process.terminate()
                             process.join()
                         print traceback.print_tb(trace)
                     sys.excepthook = beforeExit
                     for process in processes:
                         process.join()
                         process.terminate()
                     time.sleep(3)
             class ParameterConf():
                 id = None
                 name = None
                 value = None
                 format = None
                 __formated_value = None
                 ELEMENTNAME = 'Parameter'
                 def __init__(self):
                     self.format = 'str'
                 def getElementName(self):
                     return self.ELEMENTNAME
                 def getValue(self):
                     value = self.value
                     format = self.format
                     if self.__formated_value != None:
                         return self.__formated_value
                     if format == 'obj':
                         return value
                     if format == 'str':
                         self.__formated_value = str(value)
                         return self.__formated_value
                     if value == '':
                         raise ValueError, '%s: This parameter value is empty' % self.name
                     if format == 'list':
                         strList = value.split(',')
                         self.__formated_value = strList
                         return self.__formated_value
                     if format == 'intlist':
                         '''
                         Example:
                             value = (0,1,2)
                         '''
                         new_value = ast.literal_eval(value)
                         if type(new_value) not in (tuple, list):
                             new_value = [int(new_value)]
                         self.__formated_value = new_value
                         return self.__formated_value
                     if format == 'floatlist':
                         '''
                         Example:
                             value = (0.5, 1.4, 2.7)
                         '''
                         new_value = ast.literal_eval(value)
                         if type(new_value) not in (tuple, list):
                             new_value = [float(new_value)]
                         self.__formated_value = new_value
                         return self.__formated_value
                     if format == 'date':
                         strList = value.split('/')
                         intList = [int(x) for x in strList]
                         date = datetime.date(intList[0], intList[1], intList[2])
                         self.__formated_value = date
                         return self.__formated_value
                     if format == 'time':
                         strList = value.split(':')
                         intList = [int(x) for x in strList]
                         time = datetime.time(intList[0], intList[1], intList[2])
                         self.__formated_value = time
                         return self.__formated_value
                     if format == 'pairslist':
                         '''
                         Example:
                             value = (0,1),(1,2)
                         '''
                         new_value = ast.literal_eval(value)
                         if type(new_value) not in (tuple, list):
                             raise ValueError, '%s has to be a tuple or list of pairs' % value
                         if type(new_value[0]) not in (tuple, list):
                             if len(new_value) != 2:
                                 raise ValueError, '%s has to be a tuple or list of pairs' % value
                             new_value = [new_value]
                         for thisPair in new_value:
                             if len(thisPair) != 2:
                                 raise ValueError, '%s has to be a tuple or list of pairs' % value
                         self.__formated_value = new_value
                         return self.__formated_value
                     if format == 'multilist':
                         '''
                         Example:
                             value = (0,1,2),(3,4,5)
                         '''
                         multiList = ast.literal_eval(value)
                         if type(multiList[0]) == int:
                             multiList = ast.literal_eval('(' + value + ')')
                         self.__formated_value = multiList
                         return self.__formated_value
                     if format == 'bool':
                         value = int(value)
                     if format == 'int':
                         value = float(value)
                     format_func = eval(format)
                     self.__formated_value = format_func(value)
                     return self.__formated_value
                 def updateId(self, new_id):
                     self.id = str(new_id)
                 def setup(self, id, name, value, format='str'):
                     self.id = str(id)
                     self.name = name
                     if format == 'obj':
                         self.value = value
                     else:
                         self.value = str(value)
                     self.format = str.lower(format)
                     self.getValue()
                     return 1
                 def update(self, name, value, format='str'):
                     self.name = name
                     self.value = str(value)
                     self.format = format
                 def makeXml(self, opElement):
                     if self.name not in ('queue',):
                         parmElement = SubElement(opElement, self.ELEMENTNAME)
                         parmElement.set('id', str(self.id))
                         parmElement.set('name', self.name)
                         parmElement.set('value', self.value)
                         parmElement.set('format', self.format)
                 def readXml(self, parmElement):
                     self.id = parmElement.get('id')
                     self.name = parmElement.get('name')
                     self.value = parmElement.get('value')
                     self.format = str.lower(parmElement.get('format'))
                     # Compatible with old signal chain version
                     if self.format == 'int' and self.name == 'idfigure':
                         self.name = 'id'
                 def printattr(self):
                     print 'Parameter[%s]: name = %s, value = %s, format = %s' % (self.id, self.name, self.value, self.format)
             class OperationConf():
                 id = None
                 name = None
                 priority = None
                 type = None
                 parmConfObjList = []
                 ELEMENTNAME = 'Operation'
                 def __init__(self):
                     self.id = '0'
                     self.name = None
                     self.priority = None
                     self.type = 'self'
                 def __getNewId(self):
                     return int(self.id) * 10 + len(self.parmConfObjList) + 1
                 def updateId(self, new_id):
                     self.id = str(new_id)
                     n = 1
                     for parmObj in self.parmConfObjList:
                         idParm = str(int(new_id) * 10 + n)
                         parmObj.updateId(idParm)
                         n += 1
                 def getElementName(self):
                     return self.ELEMENTNAME
                 def getParameterObjList(self):
                     return self.parmConfObjList
                 def getParameterObj(self, parameterName):
                     for parmConfObj in self.parmConfObjList:
                         if parmConfObj.name != parameterName:
                             continue
                         return parmConfObj
                     return None
                 def getParameterObjfromValue(self, parameterValue):
                     for parmConfObj in self.parmConfObjList:
                         if parmConfObj.getValue() != parameterValue:
                             continue
                         return parmConfObj.getValue()
                     return None
                 def getParameterValue(self, parameterName):
                     parameterObj = self.getParameterObj(parameterName)
                 #         if not parameterObj:
                 #             return None
                     value = parameterObj.getValue()
                     return value
                 def getKwargs(self):
                     kwargs = {}
                     for parmConfObj in self.parmConfObjList:
                         if self.name == 'run' and parmConfObj.name == 'datatype':
                             continue
                         kwargs[parmConfObj.name] = parmConfObj.getValue()
                     return kwargs
                 def setup(self, id, name, priority, type):
                     self.id = str(id)
                     self.name = name
                     self.type = type
                     self.priority = priority
                     self.parmConfObjList = []
                 def removeParameters(self):
                     for obj in self.parmConfObjList:
                         del obj
                     self.parmConfObjList = []
                 def addParameter(self, name, value, format='str'):
                     if value is None:
                         return None
                     id = self.__getNewId()
                     parmConfObj = ParameterConf()
                     if not parmConfObj.setup(id, name, value, format):
                         return None
                     self.parmConfObjList.append(parmConfObj)
                     return parmConfObj
                 def changeParameter(self, name, value, format='str'):
                     parmConfObj = self.getParameterObj(name)
                     parmConfObj.update(name, value, format)
                     return parmConfObj
                 def makeXml(self, procUnitElement):
                     opElement = SubElement(procUnitElement, self.ELEMENTNAME)
                     opElement.set('id', str(self.id))
                     opElement.set('name', self.name)
                     opElement.set('type', self.type)
                     opElement.set('priority', str(self.priority))
                     for parmConfObj in self.parmConfObjList:
                         parmConfObj.makeXml(opElement)
                 def readXml(self, opElement):
                     self.id = opElement.get('id')
                     self.name = opElement.get('name')
                     self.type = opElement.get('type')
                     self.priority = opElement.get('priority')
                     # Compatible with old signal chain version
                     # Use of 'run' method instead 'init'
                     if self.type == 'self' and self.name == 'init':
                         self.name = 'run'
                     self.parmConfObjList = []
                     parmElementList = opElement.iter(ParameterConf().getElementName())
                     for parmElement in parmElementList:
                         parmConfObj = ParameterConf()
                         parmConfObj.readXml(parmElement)
                         # Compatible with old signal chain version
                         # If an 'plot' OPERATION is found, changes name operation by the value of its type PARAMETER
                         if self.type != 'self' and self.name == 'Plot':
                             if parmConfObj.format == 'str' and parmConfObj.name == 'type':
                                 self.name = parmConfObj.value
                                 continue
                         self.parmConfObjList.append(parmConfObj)
                 def printattr(self):
                     print '%s[%s]: name = %s, type = %s, priority = %s' % (self.ELEMENTNAME,
                                                                            self.id,
                                                                            self.name,
                                                                            self.type,
                                                                            self.priority)
                     for parmConfObj in self.parmConfObjList:
                         parmConfObj.printattr()
                 def createObject(self, plotter_queue=None):
                     if self.type == 'self':
                         raise ValueError, 'This operation type cannot be created'
                     if self.type == 'plotter':
                         if not plotter_queue:
                             raise ValueError, 'plotter_queue is not defined. Use:\nmyProject = Project()\nmyProject.setPlotterQueue(plotter_queue)'
                         opObj = Plotter(self.name, plotter_queue)
                     if self.type == 'external' or self.type == 'other':
                         className = eval(self.name)
                         kwargs = self.getKwargs()
                         opObj = className(**kwargs)
                     return opObj
             class ProcUnitConf():
                 id = None
                 name = None
                 datatype = None
                 inputId = None
                 parentId = None
                 opConfObjList = []
                 procUnitObj = None
                 opObjList = []
                 ELEMENTNAME = 'ProcUnit'
                 def __init__(self):
                     self.id = None
                     self.datatype = None
                     self.name = None
                     self.inputId = None
                     self.opConfObjList = []
                     self.procUnitObj = None
                     self.opObjDict = {}
                 def __getPriority(self):
                     return len(self.opConfObjList) + 1
                 def __getNewId(self):
                     return int(self.id) * 10 + len(self.opConfObjList) + 1
                 def getElementName(self):
                     return self.ELEMENTNAME
                 def getId(self):
                     return self.id
                 def updateId(self, new_id, parentId=parentId):
                     new_id = int(parentId) * 10 + (int(self.id) % 10)
                     new_inputId = int(parentId) * 10 + (int(self.inputId) % 10)
                     # If this proc unit has not inputs
                     if self.inputId == '0':
                         new_inputId = 0
                     n = 1
                     for opConfObj in self.opConfObjList:
                         idOp = str(int(new_id) * 10 + n)
                         opConfObj.updateId(idOp)
                         n += 1
                     self.parentId = str(parentId)
                     self.id = str(new_id)
                     self.inputId = str(new_inputId)
                 def getInputId(self):
                     return self.inputId
                 def getOperationObjList(self):
                     return self.opConfObjList
                 def getOperationObj(self, name=None):
                     for opConfObj in self.opConfObjList:
                         if opConfObj.name != name:
                             continue
                         return opConfObj
                     return None
                 def getOpObjfromParamValue(self, value=None):
                     for opConfObj in self.opConfObjList:
                         if opConfObj.getParameterObjfromValue(parameterValue=value) != value:
                             continue
                         return opConfObj
                     return None
                 def getProcUnitObj(self):
                     return self.procUnitObj
                 def setup(self, id, name, datatype, inputId, parentId=None):
                     # Compatible with old signal chain version
                     if datatype == None and name == None:
                         raise ValueError, 'datatype or name should be defined'
                     if name == None:
                         if 'Proc' in datatype:
                             name = datatype
                         else:
                             name = '%sProc' % (datatype)
                     if datatype == None:
                         datatype = name.replace('Proc', '')
                     self.id = str(id)
                     self.name = name
                     self.datatype = datatype
                     self.inputId = inputId
                     self.parentId = parentId
                     self.opConfObjList = []
                     self.addOperation(name='run', optype='self')
                 def removeOperations(self):
                     for obj in self.opConfObjList:
                         del obj
                     self.opConfObjList = []
                     self.addOperation(name='run')
                 def addParameter(self, **kwargs):
                     '''
                     Add parameters to 'run' operation
                     '''
                     opObj = self.opConfObjList[0]
                     opObj.addParameter(**kwargs)
                     return opObj
                 def addOperation(self, name, optype='self'):
                     id = self.__getNewId()
                     priority = self.__getPriority()
                     opConfObj = OperationConf()
                     opConfObj.setup(id, name=name, priority=priority, type=optype)
                     self.opConfObjList.append(opConfObj)
                     return opConfObj
                 def makeXml(self, projectElement):
                     procUnitElement = SubElement(projectElement, self.ELEMENTNAME)
                     procUnitElement.set('id', str(self.id))
                     procUnitElement.set('name', self.name)
                     procUnitElement.set('datatype', self.datatype)
                     procUnitElement.set('inputId', str(self.inputId))
                     for opConfObj in self.opConfObjList:
                         opConfObj.makeXml(procUnitElement)
                 def readXml(self, upElement):
                     self.id = upElement.get('id')
                     self.name = upElement.get('name')
                     self.datatype = upElement.get('datatype')
                     self.inputId = upElement.get('inputId')
                     if self.ELEMENTNAME == 'ReadUnit':
                         self.datatype = self.datatype.replace('Reader', '')
                     if self.ELEMENTNAME == 'ProcUnit':
                         self.datatype = self.datatype.replace('Proc', '')
                     if self.inputId == 'None':
                         self.inputId = '0'
                     self.opConfObjList = []
                     opElementList = upElement.iter(OperationConf().getElementName())
                     for opElement in opElementList:
                         opConfObj = OperationConf()
                         opConfObj.readXml(opElement)
                         self.opConfObjList.append(opConfObj)
                 def printattr(self):
                     print '%s[%s]: name = %s, datatype = %s, inputId = %s' % (self.ELEMENTNAME,
                                                                               self.id,
                                                                               self.name,
                                                                               self.datatype,
                                                                               self.inputId)
                     for opConfObj in self.opConfObjList:
                         opConfObj.printattr()
                 def getKwargs(self):
                     opObj = self.opConfObjList[0]
                     kwargs = opObj.getKwargs()
                     return kwargs
                 def createObjects(self, plotter_queue=None):
                     className = eval(self.name)
                     kwargs = self.getKwargs()
                     procUnitObj = className(**kwargs)
                     for opConfObj in self.opConfObjList:
                         if opConfObj.type == 'self' and self.name == 'run':
                             continue
                         elif opConfObj.type == 'self':
                             procUnitObj.addOperationKwargs(
                                 opConfObj.id, **opConfObj.getKwargs())
                             continue
                         opObj = opConfObj.createObject(plotter_queue)
                         self.opObjDict[opConfObj.id] = opObj
                         procUnitObj.addOperation(opObj, opConfObj.id)
                     self.procUnitObj = procUnitObj
                     return procUnitObj
                 def run(self):
                     is_ok = False
                     for opConfObj in self.opConfObjList:
                         kwargs = {}
                         for parmConfObj in opConfObj.getParameterObjList():
                             if opConfObj.name == 'run' and parmConfObj.name == 'datatype':
                                 continue
                             kwargs[parmConfObj.name] = parmConfObj.getValue()
                         sts = self.procUnitObj.call(opType=opConfObj.type,
                                                     opName=opConfObj.name,
                                                     opId=opConfObj.id)
                         is_ok = is_ok or sts
                     return is_ok
                 def close(self):
                     for opConfObj in self.opConfObjList:
                         if opConfObj.type == 'self':
                             continue
                         opObj = self.procUnitObj.getOperationObj(opConfObj.id)
                         opObj.close()
                     self.procUnitObj.close()
                     return
             class ReadUnitConf(ProcUnitConf):
                 path = None
                 startDate = None
                 endDate = None
                 startTime = None
                 endTime = None
                 ELEMENTNAME = 'ReadUnit'
                 def __init__(self):
                     self.id = None
                     self.datatype = None
                     self.name = None
                     self.inputId = None
                     self.parentId = None
                     self.opConfObjList = []
                     self.opObjList = []
                 def getElementName(self):
                     return self.ELEMENTNAME
                 def setup(self, id, name, datatype, path='', startDate='', endDate='',
                           startTime='', endTime='', parentId=None, server=None, **kwargs):
                     # Compatible with old signal chain version
                     if datatype == None and name == None:
                         raise ValueError, 'datatype or name should be defined'
                     if name == None:
                         if 'Reader' in datatype:
                             name = datatype
                             datatype = name.replace('Reader','')
                         else:
                             name = '{}Reader'.format(datatype)
                     if datatype == None:
                         if 'Reader' in name:
                             datatype = name.replace('Reader','')
                         else:
                             datatype = name
                             name = '{}Reader'.format(name)
                     self.id = id
                     self.name = name
                     self.datatype = datatype
                     if path != '':
                         self.path = os.path.abspath(path)
                     self.startDate = startDate
                     self.endDate = endDate
                     self.startTime = startTime
                     self.endTime = endTime
                     self.inputId = '0'
                     self.parentId = parentId
                     self.server = server
                     self.addRunOperation(**kwargs)
                 def update(self, **kwargs):
                     if 'datatype' in kwargs:
                         datatype = kwargs.pop('datatype')
                         if 'Reader' in datatype:
                             self.name = datatype
                         else:
                             self.name = '%sReader' % (datatype)
                         self.datatype = self.name.replace('Reader', '')
                     attrs = ('path', 'startDate', 'endDate',
                              'startTime', 'endTime', 'parentId')
                     for attr in attrs:
                         if attr in kwargs:
                             setattr(self, attr, kwargs.pop(attr))
                     self.inputId = '0'
                     self.updateRunOperation(**kwargs)
                 def removeOperations(self):
                     for obj in self.opConfObjList:
                         del obj
                     self.opConfObjList = []
                 def addRunOperation(self, **kwargs):
                     opObj = self.addOperation(name='run', optype='self')
                     if self.server is None:
                         opObj.addParameter(
                             name='datatype', value=self.datatype, format='str')
                         opObj.addParameter(name='path', value=self.path, format='str')
                         opObj.addParameter(
                             name='startDate', value=self.startDate, format='date')
                         opObj.addParameter(
                             name='endDate', value=self.endDate, format='date')
                         opObj.addParameter(
                             name='startTime', value=self.startTime, format='time')
                         opObj.addParameter(
                             name='endTime', value=self.endTime, format='time')
                         for key, value in kwargs.items():
                             opObj.addParameter(name=key, value=value,
                                                format=type(value).__name__)
                     else:
                         opObj.addParameter(name='server', value=self.server, format='str')
                     return opObj
                 def updateRunOperation(self, **kwargs):
                     opObj = self.getOperationObj(name='run')
                     opObj.removeParameters()
                     opObj.addParameter(name='datatype', value=self.datatype, format='str')
                     opObj.addParameter(name='path', value=self.path, format='str')
                     opObj.addParameter(
                         name='startDate', value=self.startDate, format='date')
                     opObj.addParameter(name='endDate', value=self.endDate, format='date')
                     opObj.addParameter(
                         name='startTime', value=self.startTime, format='time')
                     opObj.addParameter(name='endTime', value=self.endTime, format='time')
                     for key, value in kwargs.items():
                         opObj.addParameter(name=key, value=value,
                                            format=type(value).__name__)
                     return opObj
                 def readXml(self, upElement):
                     self.id = upElement.get('id')
                     self.name = upElement.get('name')
                     self.datatype = upElement.get('datatype')
                     self.inputId = upElement.get('inputId')
                     if self.ELEMENTNAME == 'ReadUnit':
                         self.datatype = self.datatype.replace('Reader', '')
                     if self.inputId == 'None':
                         self.inputId = '0'
                     self.opConfObjList = []
                     opElementList = upElement.iter(OperationConf().getElementName())
                     for opElement in opElementList:
                         opConfObj = OperationConf()
                         opConfObj.readXml(opElement)
                         self.opConfObjList.append(opConfObj)
                         if opConfObj.name == 'run':
                             self.path = opConfObj.getParameterValue('path')
                             self.startDate = opConfObj.getParameterValue('startDate')
                             self.endDate = opConfObj.getParameterValue('endDate')
                             self.startTime = opConfObj.getParameterValue('startTime')
                             self.endTime = opConfObj.getParameterValue('endTime')
             class Project(Process):
                 id = None
                 # name = None
                 description = None
                 filename = None
                 procUnitConfObjDict = None
                 ELEMENTNAME = 'Project'
                 plotterQueue = None
                 def __init__(self, plotter_queue=None):
                     Process.__init__(self)
                     self.id = None
                     # self.name = None
                     self.description = None
                     self.plotterQueue = plotter_queue
                     self.procUnitConfObjDict = {}
                 def __getNewId(self):
                     idList = self.procUnitConfObjDict.keys()
                     id = int(self.id) * 10
                     while True:
                         id += 1
                         if str(id) in idList:
                             continue
                         break
                     return str(id)
                 def getElementName(self):
                     return self.ELEMENTNAME
                 def getId(self):
                     return self.id
                 def updateId(self, new_id):
                     self.id = str(new_id)
                     keyList = self.procUnitConfObjDict.keys()
                     keyList.sort()
                     n = 1
                     newProcUnitConfObjDict = {}
                     for procKey in keyList:
                         procUnitConfObj = self.procUnitConfObjDict[procKey]
                         idProcUnit = str(int(self.id) * 10 + n)
                         procUnitConfObj.updateId(idProcUnit, parentId=self.id)
                         newProcUnitConfObjDict[idProcUnit] = procUnitConfObj
                         n += 1
                     self.procUnitConfObjDict = newProcUnitConfObjDict
                 def setup(self, id, name='', description=''):
                     print
                     print '*' * 60
                     print '   Starting SIGNAL CHAIN PROCESSING v%s ' % schainpy.__version__
                     print '*' * 60
                     print
                     self.id = str(id)
                     self.description = description
                 def update(self, name, description):
                     self.description = description
                 def clone(self):
                     p = Project()
                     p.procUnitConfObjDict = self.procUnitConfObjDict
                     return p
                 def addReadUnit(self, id=None, datatype=None, name=None, **kwargs):
                     if id is None:
                         idReadUnit = self.__getNewId()
                     else:
                         idReadUnit = str(id)
                     readUnitConfObj = ReadUnitConf()
                     readUnitConfObj.setup(idReadUnit, name, datatype,
                                           parentId=self.id, **kwargs)
                     self.procUnitConfObjDict[readUnitConfObj.getId()] = readUnitConfObj
                     return readUnitConfObj
                 def addProcUnit(self, inputId='0', datatype=None, name=None):
                     idProcUnit = self.__getNewId()
                     procUnitConfObj = ProcUnitConf()
                     procUnitConfObj.setup(idProcUnit, name, datatype,
                                           inputId, parentId=self.id)
                     self.procUnitConfObjDict[procUnitConfObj.getId()] = procUnitConfObj
                     return procUnitConfObj
                 def removeProcUnit(self, id):
                     if id in self.procUnitConfObjDict.keys():
                         self.procUnitConfObjDict.pop(id)
                 def getReadUnitId(self):
                     readUnitConfObj = self.getReadUnitObj()
                     return readUnitConfObj.id
                 def getReadUnitObj(self):
                     for obj in self.procUnitConfObjDict.values():
                         if obj.getElementName() == 'ReadUnit':
                             return obj
                     return None
                 def getProcUnitObj(self, id=None, name=None):
                     if id != None:
                         return self.procUnitConfObjDict[id]
                     if name != None:
                         return self.getProcUnitObjByName(name)
                     return None
                 def getProcUnitObjByName(self, name):
                     for obj in self.procUnitConfObjDict.values():
                         if obj.name == name:
                             return obj
                     return None
                 def procUnitItems(self):
                     return self.procUnitConfObjDict.items()
                 def makeXml(self):
                     projectElement = Element('Project')
                     projectElement.set('id', str(self.id))
                     projectElement.set('name', self.name)
                     projectElement.set('description', self.description)
                     for procUnitConfObj in self.procUnitConfObjDict.values():
                         procUnitConfObj.makeXml(projectElement)
                     self.projectElement = projectElement
                 def writeXml(self, filename=None):
                     if filename == None:
                         if self.filename:
                             filename = self.filename
                         else:
                             filename = 'schain.xml'
                     if not filename:
                         print 'filename has not been defined. Use setFilename(filename) for do it.'
                         return 0
                     abs_file = os.path.abspath(filename)
                     if not os.access(os.path.dirname(abs_file), os.W_OK):
                         print 'No write permission on %s' % os.path.dirname(abs_file)
                         return 0
                     if os.path.isfile(abs_file) and not(os.access(abs_file, os.W_OK)):
                         print 'File %s already exists and it could not be overwriten' % abs_file
                         return 0
                     self.makeXml()
                     ElementTree(self.projectElement).write(abs_file, method='xml')
                     self.filename = abs_file
                     return 1
                 def readXml(self, filename=None):
                     if not filename:
                         print 'filename is not defined'
                         return 0
                     abs_file = os.path.abspath(filename)
                     if not os.path.isfile(abs_file):
                         print '%s file does not exist' % abs_file
                         return 0
                     self.projectElement = None
                     self.procUnitConfObjDict = {}
                     try:
                         self.projectElement = ElementTree().parse(abs_file)
                     except:
                         print 'Error reading %s, verify file format' % filename
                         return 0
                     self.project = self.projectElement.tag
                     self.id = self.projectElement.get('id')
                     self.name = self.projectElement.get('name')
                     self.description = self.projectElement.get('description')
                     readUnitElementList = self.projectElement.iter(
                         ReadUnitConf().getElementName())
                     for readUnitElement in readUnitElementList:
                         readUnitConfObj = ReadUnitConf()
                         readUnitConfObj.readXml(readUnitElement)
                         if readUnitConfObj.parentId == None:
                             readUnitConfObj.parentId = self.id
                         self.procUnitConfObjDict[readUnitConfObj.getId()] = readUnitConfObj
                     procUnitElementList = self.projectElement.iter(
                         ProcUnitConf().getElementName())
                     for procUnitElement in procUnitElementList:
                         procUnitConfObj = ProcUnitConf()
                         procUnitConfObj.readXml(procUnitElement)
                         if procUnitConfObj.parentId == None:
                             procUnitConfObj.parentId = self.id
                         self.procUnitConfObjDict[procUnitConfObj.getId()] = procUnitConfObj
                     self.filename = abs_file
                     return 1
                 def printattr(self):
                     print 'Project[%s]: name = %s, description = %s' % (self.id,
                                                                         self.name,
                                                                         self.description)
                     for procUnitConfObj in self.procUnitConfObjDict.values():
                         procUnitConfObj.printattr()
                 def createObjects(self):
                     for procUnitConfObj in self.procUnitConfObjDict.values():
                         procUnitConfObj.createObjects(self.plotterQueue)
                 def __connect(self, objIN, thisObj):
                     thisObj.setInput(objIN.getOutputObj())
                 def connectObjects(self):
                     for thisPUConfObj in self.procUnitConfObjDict.values():
                         inputId = thisPUConfObj.getInputId()
                         if int(inputId) == 0:
                             continue
                         # Get input object
                         puConfINObj = self.procUnitConfObjDict[inputId]
                         puObjIN = puConfINObj.getProcUnitObj()
                         # Get current object
                         thisPUObj = thisPUConfObj.getProcUnitObj()
                         self.__connect(puObjIN, thisPUObj)
                 def __handleError(self, procUnitConfObj, send_email=False):
                     import socket
                     err = traceback.format_exception(sys.exc_info()[0],
                                                      sys.exc_info()[1],
                                                      sys.exc_info()[2])
                     print '***** Error occurred in %s *****' % (procUnitConfObj.name)
                     print '***** %s' % err[-1]
                     message = ''.join(err)
                     sys.stderr.write(message)
                     if not send_email:
                         return
                     subject = 'SChain v%s: Error running %s\n' % (
                         schainpy.__version__, procUnitConfObj.name)
                     subtitle = '%s: %s\n' % (
                         procUnitConfObj.getElementName(), procUnitConfObj.name)
                     subtitle += 'Hostname: %s\n' % socket.gethostbyname(
                         socket.gethostname())
                     subtitle += 'Working directory: %s\n' % os.path.abspath('./')
                     subtitle += 'Configuration file: %s\n' % self.filename
                     subtitle += 'Time: %s\n' % str(datetime.datetime.now())
                     readUnitConfObj = self.getReadUnitObj()
                     if readUnitConfObj:
                         subtitle += '\nInput parameters:\n'
                         subtitle += '[Data path = %s]\n' % readUnitConfObj.path
                         subtitle += '[Data type = %s]\n' % readUnitConfObj.datatype
                         subtitle += '[Start date = %s]\n' % readUnitConfObj.startDate
                         subtitle += '[End date = %s]\n' % readUnitConfObj.endDate
                         subtitle += '[Start time = %s]\n' % readUnitConfObj.startTime
                         subtitle += '[End time = %s]\n' % readUnitConfObj.endTime
                     adminObj = schainpy.admin.SchainNotify()
                     adminObj.sendAlert(message=message,
                                        subject=subject,
                                        subtitle=subtitle,
                                        filename=self.filename)
                 def isPaused(self):
                     return 0
                 def isStopped(self):
                     return 0
                 def runController(self):
                     '''
                     returns 0 when this process has been stopped, 1 otherwise
                     '''
                     if self.isPaused():
                         print 'Process suspended'
                         while True:
                             time.sleep(0.1)
                             if not self.isPaused():
                                 break
                             if self.isStopped():
                                 break
                         print 'Process reinitialized'
                     if self.isStopped():
                         print 'Process stopped'
                         return 0
                     return 1
                 def setFilename(self, filename):
                     self.filename = filename
                 def setPlotterQueue(self, plotter_queue):
                     raise NotImplementedError, 'Use schainpy.controller_api.ControllerThread instead Project class'
                 def getPlotterQueue(self):
                     raise NotImplementedError, 'Use schainpy.controller_api.ControllerThread instead Project class'
                 def useExternalPlotter(self):
                     raise NotImplementedError, 'Use schainpy.controller_api.ControllerThread instead Project class'
                 def run(self):
                     log.success('Starting {}'.format(self.name))
+                    self.start_time = time.time()
                     self.createObjects()
                     self.connectObjects()
                     keyList = self.procUnitConfObjDict.keys()
                     keyList.sort()
                     while(True):
                         is_ok = False
                         for procKey in keyList:
                             procUnitConfObj = self.procUnitConfObjDict[procKey]
                             try:
                                 sts = procUnitConfObj.run()
                                 is_ok = is_ok or sts
                             except KeyboardInterrupt:
                                 is_ok = False
                                 break
                             except ValueError, e:
                                 time.sleep(0.5)
                                 self.__handleError(procUnitConfObj, send_email=True)
                                 is_ok = False
                                 break
                             except:
                                 time.sleep(0.5)
                                 self.__handleError(procUnitConfObj)
                                 is_ok = False
                                 break
                         # If every process unit finished so end process
                         if not(is_ok):
                             break
                         if not self.runController():
                             break
                     # Closing every process
                     for procKey in keyList:
                         procUnitConfObj = self.procUnitConfObjDict[procKey]
                         procUnitConfObj.close()
-                    log.success('{} finished'.format(self.name))
+                    log.success('{} finished (time: {}s)'.format(
+                        self.name,
+                        time.time()-self.start_time))

schainpy/gui/schainGUI +1 -2

             #!/usr/bin/env python
             import os
             import sys
             from schainpy.utils import log
             try:
                 from PyQt4 import QtCore, QtGui
                 from PyQt4.QtGui import QApplication
             except:
                 log.error(
-                    'You should install PtQt4 module in order to run the GUI. See the README.')
+                    'You should install PyQt4 module in order to run the GUI. See the README.')
                 sys.exit()
             from schainpy.gui.viewcontroller.initwindow import InitWindow
             from schainpy.gui.viewcontroller.basicwindow import BasicWindow
             from schainpy.gui.viewcontroller.workspace import Workspace
             def main():
                 app = QtGui.QApplication(sys.argv)
                 Welcome = InitWindow()
                 if not Welcome.exec_():
                     sys.exit(-1)
                 WorkPathspace = Workspace()
                 if not WorkPathspace.exec_():
                     sys.exit(-1)
                 MainGUI = BasicWindow()
                 MainGUI.setWorkSpaceGUI(WorkPathspace.dirComBox.currentText())
                 MainGUI.show()
                 sys.exit(app.exec_())
             if __name__ == "__main__":
                 main()

schainpy/model/data/jroheaderIO.py +1 -1

             '''
             $Author: murco $
             $Id: JROHeaderIO.py 151 2012-10-31 19:00:51Z murco $
             '''
             import sys
             import numpy
             import copy
             import datetime
             import inspect
             SPEED_OF_LIGHT = 299792458
             SPEED_OF_LIGHT = 3e8
             BASIC_STRUCTURE = numpy.dtype([
                 ('nSize', '<u4'),
                 ('nVersion', '<u2'),
                 ('nDataBlockId', '<u4'),
                 ('nUtime', '<u4'),
                 ('nMilsec', '<u2'),
                 ('nTimezone', '<i2'),
                 ('nDstflag', '<i2'),
                 ('nErrorCount', '<u4')
             ])
             SYSTEM_STRUCTURE = numpy.dtype([
                 ('nSize', '<u4'),
                 ('nNumSamples', '<u4'),
                 ('nNumProfiles', '<u4'),
                 ('nNumChannels', '<u4'),
                 ('nADCResolution', '<u4'),
                 ('nPCDIOBusWidth', '<u4'),
             ])
             RADAR_STRUCTURE = numpy.dtype([
                                          ('nSize', '<u4'),
                                          ('nExpType', '<u4'),
                                          ('nNTx', '<u4'),
                                          ('fIpp', '<f4'),
                                          ('fTxA', '<f4'),
                                          ('fTxB', '<f4'),
                                          ('nNumWindows', '<u4'),
                                          ('nNumTaus', '<u4'),
                                          ('nCodeType', '<u4'),
                                          ('nLine6Function', '<u4'),
                                          ('nLine5Function', '<u4'),
                                          ('fClock', '<f4'),
                                          ('nPrePulseBefore', '<u4'),
                                          ('nPrePulseAfter', '<u4'),
                                          ('sRangeIPP', '<a20'),
                                          ('sRangeTxA', '<a20'),
                                          ('sRangeTxB', '<a20'),
             ])
             SAMPLING_STRUCTURE = numpy.dtype(
                 [('h0', '<f4'), ('dh', '<f4'), ('nsa', '<u4')])
             PROCESSING_STRUCTURE = numpy.dtype([
                 ('nSize', '<u4'),
                 ('nDataType', '<u4'),
                 ('nSizeOfDataBlock', '<u4'),
                 ('nProfilesperBlock', '<u4'),
                 ('nDataBlocksperFile', '<u4'),
                 ('nNumWindows', '<u4'),
                 ('nProcessFlags', '<u4'),
                 ('nCoherentIntegrations', '<u4'),
                 ('nIncoherentIntegrations', '<u4'),
                 ('nTotalSpectra', '<u4')
             ])
             class Header(object):
                 def __init__(self):
                     raise NotImplementedError
                 def copy(self):
                     return copy.deepcopy(self)
                 def read(self):
                     raise NotImplementedError
                 def write(self):
                     raise NotImplementedError
                 def getAllowedArgs(self):
                     args = inspect.getargspec(self.__init__).args
                     try:
                         args.remove('self')
                     except:
                         pass
                     return args
                 def getAsDict(self):
                     args = self.getAllowedArgs()
                     asDict = {}
                     for x in args:
                         asDict[x] = self[x]
                     return asDict
                 def __getitem__(self, name):
                     return getattr(self, name)
                 def printInfo(self):
                     message = "#" * 50 + "\n"
                     message += self.__class__.__name__.upper() + "\n"
                     message += "#" * 50 + "\n"
                     keyList = self.__dict__.keys()
                     keyList.sort()
                     for key in keyList:
                         message += "%s = %s" % (key, self.__dict__[key]) + "\n"
                     if "size" not in keyList:
                         attr = getattr(self, "size")
                         if attr:
                             message += "%s = %s" % ("size", attr) + "\n"
                     print message
             class BasicHeader(Header):
                 size = None
                 version = None
                 dataBlock = None
                 utc = None
                 ltc = None
                 miliSecond = None
                 timeZone = None
                 dstFlag = None
                 errorCount = None
                 datatime = None
                 structure = BASIC_STRUCTURE
                 __LOCALTIME = None
                 def __init__(self, useLocalTime=True):
                     self.size = 24
                     self.version = 0
                     self.dataBlock = 0
                     self.utc = 0
                     self.miliSecond = 0
                     self.timeZone = 0
                     self.dstFlag = 0
                     self.errorCount = 0
                     self.useLocalTime = useLocalTime
                 def read(self, fp):
                     self.length = 0
                     try:
                         if hasattr(fp, 'read'):
                             header = numpy.fromfile(fp, BASIC_STRUCTURE, 1)
                         else:
                             header = numpy.fromstring(fp, BASIC_STRUCTURE, 1)
                     except Exception, e:
                         print "BasicHeader: "
                         print e
                         return 0
                     self.size = int(header['nSize'][0])
                     self.version = int(header['nVersion'][0])
                     self.dataBlock = int(header['nDataBlockId'][0])
                     self.utc = int(header['nUtime'][0])
                     self.miliSecond = int(header['nMilsec'][0])
                     self.timeZone = int(header['nTimezone'][0])
                     self.dstFlag = int(header['nDstflag'][0])
                     self.errorCount = int(header['nErrorCount'][0])
                     if self.size < 24:
                         return 0
                     self.length = header.nbytes
                     return 1
                 def write(self, fp):
                     headerTuple = (self.size, self.version, self.dataBlock, self.utc,
                                    self.miliSecond, self.timeZone, self.dstFlag, self.errorCount)
                     header = numpy.array(headerTuple, BASIC_STRUCTURE)
                     header.tofile(fp)
                     return 1
                 def get_ltc(self):
                     return self.utc - self.timeZone * 60
                 def set_ltc(self, value):
                     self.utc = value + self.timeZone * 60
                 def get_datatime(self):
                     return datetime.datetime.utcfromtimestamp(self.ltc)
                 ltc = property(get_ltc, set_ltc)
                 datatime = property(get_datatime)
             class SystemHeader(Header):
                 size = None
                 nSamples = None
                 nProfiles = None
                 nChannels = None
                 adcResolution = None
                 pciDioBusWidth = None
                 structure = SYSTEM_STRUCTURE
                 def __init__(self, nSamples=0, nProfiles=0, nChannels=0, adcResolution=14, pciDioBusWidth=0):
                     self.size = 24
                     self.nSamples = nSamples
                     self.nProfiles = nProfiles
                     self.nChannels = nChannels
                     self.adcResolution = adcResolution
                     self.pciDioBusWidth = pciDioBusWidth
                 def read(self, fp):
                     self.length = 0
                     try:
                         startFp = fp.tell()
                     except Exception, e:
                         startFp = None
                         pass
                     try:
                         if hasattr(fp, 'read'):
                             header = numpy.fromfile(fp, SYSTEM_STRUCTURE, 1)
                         else:
                             header = numpy.fromstring(fp, SYSTEM_STRUCTURE, 1)
                     except Exception, e:
                         print "System Header: " + str(e)
                         return 0
                     self.size = header['nSize'][0]
                     self.nSamples = header['nNumSamples'][0]
                     self.nProfiles = header['nNumProfiles'][0]
                     self.nChannels = header['nNumChannels'][0]
                     self.adcResolution = header['nADCResolution'][0]
                     self.pciDioBusWidth = header['nPCDIOBusWidth'][0]
                     if startFp is not None:
                         endFp = self.size + startFp
                         if fp.tell() > endFp:
                             sys.stderr.write(
                                 "Warning %s: Size value read from System Header is lower than it has to be\n" % fp.name)
                             return 0
                         if fp.tell() < endFp:
                             sys.stderr.write(
                                 "Warning %s: Size value read from System Header size is greater than it has to be\n" % fp.name)
                             return 0
                     self.length = header.nbytes
                     return 1
                 def write(self, fp):
                     headerTuple = (self.size, self.nSamples, self.nProfiles,
                                    self.nChannels, self.adcResolution, self.pciDioBusWidth)
                     header = numpy.array(headerTuple, SYSTEM_STRUCTURE)
                     header.tofile(fp)
                     return 1
             class RadarControllerHeader(Header):
                 expType = None
                 nTx = None
                 ipp = None
                 txA = None
                 txB = None
                 nWindows = None
                 numTaus = None
                 codeType = None
                 line6Function = None
                 line5Function = None
                 fClock = None
                 prePulseBefore = None
                 prePulseAfter = None
                 rangeIpp = None
                 rangeTxA = None
                 rangeTxB = None
                 structure = RADAR_STRUCTURE
                 __size = None
                 def __init__(self, expType=2, nTx=1,
                              ipp=None, txA=0, txB=0,
                              nWindows=None, nHeights=None, firstHeight=None, deltaHeight=None,
                              numTaus=0, line6Function=0, line5Function=0, fClock=None,
                              prePulseBefore=0, prePulseAfter=0,
                              codeType=0, nCode=0, nBaud=0, code=None,
                              flip1=0, flip2=0):
                     #         self.size = 116
                     self.expType = expType
                     self.nTx = nTx
                     self.ipp = ipp
                     self.txA = txA
                     self.txB = txB
                     self.rangeIpp = ipp
                     self.rangeTxA = txA
                     self.rangeTxB = txB
                     self.nWindows = nWindows
                     self.numTaus = numTaus
                     self.codeType = codeType
                     self.line6Function = line6Function
                     self.line5Function = line5Function
                     self.fClock = fClock
                     self.prePulseBefore = prePulseBefore
                     self.prePulseAfter = prePulseAfter
                     self.nHeights = nHeights
                     self.firstHeight = firstHeight
                     self.deltaHeight = deltaHeight
                     self.samplesWin = nHeights
                     self.nCode = nCode
                     self.nBaud = nBaud
                     self.code = code
                     self.flip1 = flip1
                     self.flip2 = flip2
                     self.code_size = int(numpy.ceil(self.nBaud / 32.)) * self.nCode * 4
             #         self.dynamic = numpy.array([],numpy.dtype('byte'))
                     if self.fClock is None and self.deltaHeight is not None:
                         self.fClock = 0.15 / (deltaHeight * 1e-6)  # 0.15Km / (height * 1u)
                 def read(self, fp):
                     self.length = 0
                     try:
                         startFp = fp.tell()
                     except Exception, e:
                         startFp = None
                         pass
                     try:
                         if hasattr(fp, 'read'):
                             header = numpy.fromfile(fp, RADAR_STRUCTURE, 1)
                         else:
                             header = numpy.fromstring(fp, RADAR_STRUCTURE, 1)
                         self.length += header.nbytes
                     except Exception, e:
                         print "RadarControllerHeader: " + str(e)
                         return 0
                     size = int(header['nSize'][0])
                     self.expType = int(header['nExpType'][0])
                     self.nTx = int(header['nNTx'][0])
                     self.ipp = float(header['fIpp'][0])
                     self.txA = float(header['fTxA'][0])
                     self.txB = float(header['fTxB'][0])
                     self.nWindows = int(header['nNumWindows'][0])
                     self.numTaus = int(header['nNumTaus'][0])
                     self.codeType = int(header['nCodeType'][0])
                     self.line6Function = int(header['nLine6Function'][0])
                     self.line5Function = int(header['nLine5Function'][0])
                     self.fClock = float(header['fClock'][0])
                     self.prePulseBefore = int(header['nPrePulseBefore'][0])
                     self.prePulseAfter = int(header['nPrePulseAfter'][0])
                     self.rangeIpp = header['sRangeIPP'][0]
                     self.rangeTxA = header['sRangeTxA'][0]
                     self.rangeTxB = header['sRangeTxB'][0]
                     try:
                         if hasattr(fp, 'read'):
                             samplingWindow = numpy.fromfile(
                                 fp, SAMPLING_STRUCTURE, self.nWindows)
                         else:
                             samplingWindow = numpy.fromstring(
                                 fp[self.length:], SAMPLING_STRUCTURE, self.nWindows)
                         self.length += samplingWindow.nbytes
                     except Exception, e:
                         print "RadarControllerHeader: " + str(e)
                         return 0
                     self.nHeights = int(numpy.sum(samplingWindow['nsa']))
                     self.firstHeight = samplingWindow['h0']
                     self.deltaHeight = samplingWindow['dh']
                     self.samplesWin = samplingWindow['nsa']
                     try:
                         if hasattr(fp, 'read'):
                             self.Taus = numpy.fromfile(fp, '<f4', self.numTaus)
                         else:
                             self.Taus = numpy.fromstring(
                                 fp[self.length:], '<f4', self.numTaus)
                         self.length += self.Taus.nbytes
                     except Exception, e:
                         print "RadarControllerHeader: " + str(e)
                         return 0
                     self.code_size = 0
                     if self.codeType != 0:
                         try:
                             if hasattr(fp, 'read'):
                                 self.nCode = numpy.fromfile(fp, '<u4', 1)[0]
                                 self.length += self.nCode.nbytes
                                 self.nBaud = numpy.fromfile(fp, '<u4', 1)[0]
                                 self.length += self.nBaud.nbytes
                             else:
                                 self.nCode = numpy.fromstring(
                                     fp[self.length:], '<u4', 1)[0]
                                 self.length += self.nCode.nbytes
                                 self.nBaud = numpy.fromstring(
                                     fp[self.length:], '<u4', 1)[0]
                                 self.length += self.nBaud.nbytes
                         except Exception, e:
                             print "RadarControllerHeader: " + str(e)
                             return 0
                         code = numpy.empty([self.nCode, self.nBaud], dtype='i1')
                         for ic in range(self.nCode):
                             try:
                                 if hasattr(fp, 'read'):
                                     temp = numpy.fromfile(fp, 'u4', int(
                                         numpy.ceil(self.nBaud / 32.)))
                                 else:
                                     temp = numpy.fromstring(
                                         fp, 'u4', int(numpy.ceil(self.nBaud / 32.)))
                                 self.length += temp.nbytes
                             except Exception, e:
                                 print "RadarControllerHeader: " + str(e)
                                 return 0
                             for ib in range(self.nBaud - 1, -1, -1):
                                 code[ic, ib] = temp[ib / 32] % 2
                                 temp[ib / 32] = temp[ib / 32] / 2
                         self.code = 2.0 * code - 1.0
                         self.code_size = int(numpy.ceil(self.nBaud / 32.)) * self.nCode * 4
             #             if self.line5Function == RCfunction.FLIP:
             #                 self.flip1 = numpy.fromfile(fp,'<u4',1)
             #
             #             if self.line6Function == RCfunction.FLIP:
             #                 self.flip2 = numpy.fromfile(fp,'<u4',1)
                     if startFp is not None:
                         endFp = size + startFp
                         if fp.tell() != endFp:
                             #             fp.seek(endFp)
                             print "%s: Radar Controller Header size is not consistent: from data [%d] != from header field [%d]" % (fp.name, fp.tell() - startFp, size)
                 #             return 0
                         if fp.tell() > endFp:
                             sys.stderr.write(
                                 "Warning %s: Size value read from Radar Controller header is lower than it has to be\n" % fp.name)
                 #             return 0
                         if fp.tell() < endFp:
                             sys.stderr.write(
                                 "Warning %s: Size value read from Radar Controller header is greater than it has to be\n" % fp.name)
                     return 1
                 def write(self, fp):
                     headerTuple = (self.size,
                                    self.expType,
                                    self.nTx,
                                    self.ipp,
                                    self.txA,
                                    self.txB,
                                    self.nWindows,
                                    self.numTaus,
                                    self.codeType,
                                    self.line6Function,
                                    self.line5Function,
                                    self.fClock,
                                    self.prePulseBefore,
                                    self.prePulseAfter,
                                    self.rangeIpp,
                                    self.rangeTxA,
                                    self.rangeTxB)
                     header = numpy.array(headerTuple, RADAR_STRUCTURE)
                     header.tofile(fp)
                     sampleWindowTuple = (
                         self.firstHeight, self.deltaHeight, self.samplesWin)
                     samplingWindow = numpy.array(sampleWindowTuple, SAMPLING_STRUCTURE)
                     samplingWindow.tofile(fp)
                     if self.numTaus > 0:
                         self.Taus.tofile(fp)
                     if self.codeType != 0:
                         nCode = numpy.array(self.nCode, '<u4')
                         nCode.tofile(fp)
                         nBaud = numpy.array(self.nBaud, '<u4')
                         nBaud.tofile(fp)
                         code1 = (self.code + 1.0) / 2.
                         for ic in range(self.nCode):
-                            tempx = numpy.zeros(int(self.nBaud / 32.))
+                            tempx = numpy.zeros(int(numpy.ceil(self.nBaud / 32.)))
                             start = 0
                             end = 32
                             for i in range(len(tempx)):
                                 code_selected = code1[ic, start:end]
                                 for j in range(len(code_selected) - 1, -1, -1):
                                     if code_selected[j] == 1:
                                         tempx[i] = tempx[i] + \
 **(len(code_selected) - 1 - j)
                                 start = start + 32
                                 end = end + 32
                             tempx = tempx.astype('u4')
                             tempx.tofile(fp)
             #         if self.line5Function == RCfunction.FLIP:
             #             self.flip1.tofile(fp)
             #
             #         if self.line6Function == RCfunction.FLIP:
             #             self.flip2.tofile(fp)
                     return 1
                 def get_ippSeconds(self):
                     '''
                     '''
                     ippSeconds = 2.0 * 1000 * self.ipp / SPEED_OF_LIGHT
                     return ippSeconds
                 def set_ippSeconds(self, ippSeconds):
                     '''
                     '''
                     self.ipp = ippSeconds * SPEED_OF_LIGHT / (2.0 * 1000)
                     return
                 def get_size(self):
                     self.__size = 116 + 12 * self.nWindows + 4 * self.numTaus
                     if self.codeType != 0:
                         self.__size += 4 + 4 + 4 * self.nCode * \
                             numpy.ceil(self.nBaud / 32.)
                     return self.__size
                 def set_size(self, value):
                     raise IOError, "size is a property and it cannot be set, just read"
                     return
                 ippSeconds = property(get_ippSeconds, set_ippSeconds)
                 size = property(get_size, set_size)
             class ProcessingHeader(Header):
                 #     size = None
                 dtype = None
                 blockSize = None
                 profilesPerBlock = None
                 dataBlocksPerFile = None
                 nWindows = None
                 processFlags = None
                 nCohInt = None
                 nIncohInt = None
                 totalSpectra = None
                 structure = PROCESSING_STRUCTURE
                 flag_dc = None
                 flag_cspc = None
                 def __init__(self, dtype=0, blockSize=0, profilesPerBlock=0, dataBlocksPerFile=0, nWindows=0, processFlags=0, nCohInt=0,
                              nIncohInt=0, totalSpectra=0, nHeights=0, firstHeight=0, deltaHeight=0, samplesWin=0, spectraComb=0, nCode=0,
                              code=0, nBaud=None, shif_fft=False, flag_dc=False, flag_cspc=False, flag_decode=False, flag_deflip=False
                              ):
                     #         self.size = 0
                     self.dtype = dtype
                     self.blockSize = blockSize
                     self.profilesPerBlock = 0
                     self.dataBlocksPerFile = 0
                     self.nWindows = 0
                     self.processFlags = 0
                     self.nCohInt = 0
                     self.nIncohInt = 0
                     self.totalSpectra = 0
                     self.nHeights = 0
                     self.firstHeight = 0
                     self.deltaHeight = 0
                     self.samplesWin = 0
                     self.spectraComb = 0
                     self.nCode = None
                     self.code = None
                     self.nBaud = None
                     self.shif_fft = False
                     self.flag_dc = False
                     self.flag_cspc = False
                     self.flag_decode = False
                     self.flag_deflip = False
                     self.length = 0
                 def read(self, fp):
                     self.length = 0
                     try:
                         startFp = fp.tell()
                     except Exception, e:
                         startFp = None
                         pass
                     try:
                         if hasattr(fp, 'read'):
                             header = numpy.fromfile(fp, PROCESSING_STRUCTURE, 1)
                         else:
                             header = numpy.fromstring(fp, PROCESSING_STRUCTURE, 1)
                         self.length += header.nbytes
                     except Exception, e:
                         print "ProcessingHeader: " + str(e)
                         return 0
                     size = int(header['nSize'][0])
                     self.dtype = int(header['nDataType'][0])
                     self.blockSize = int(header['nSizeOfDataBlock'][0])
                     self.profilesPerBlock = int(header['nProfilesperBlock'][0])
                     self.dataBlocksPerFile = int(header['nDataBlocksperFile'][0])
                     self.nWindows = int(header['nNumWindows'][0])
                     self.processFlags = header['nProcessFlags']
                     self.nCohInt = int(header['nCoherentIntegrations'][0])
                     self.nIncohInt = int(header['nIncoherentIntegrations'][0])
                     self.totalSpectra = int(header['nTotalSpectra'][0])
                     try:
                         if hasattr(fp, 'read'):
                             samplingWindow = numpy.fromfile(
                                 fp, SAMPLING_STRUCTURE, self.nWindows)
                         else:
                             samplingWindow = numpy.fromstring(
                                 fp[self.length:], SAMPLING_STRUCTURE, self.nWindows)
                         self.length += samplingWindow.nbytes
                     except Exception, e:
                         print "ProcessingHeader: " + str(e)
                         return 0
                     self.nHeights = int(numpy.sum(samplingWindow['nsa']))
                     self.firstHeight = float(samplingWindow['h0'][0])
                     self.deltaHeight = float(samplingWindow['dh'][0])
                     self.samplesWin = samplingWindow['nsa'][0]
                     try:
                         if hasattr(fp, 'read'):
                             self.spectraComb = numpy.fromfile(
                                 fp, 'u1', 2 * self.totalSpectra)
                         else:
                             self.spectraComb = numpy.fromstring(
                                 fp[self.length:], 'u1', 2 * self.totalSpectra)
                         self.length += self.spectraComb.nbytes
                     except Exception, e:
                         print "ProcessingHeader: " + str(e)
                         return 0
                     if ((self.processFlags & PROCFLAG.DEFINE_PROCESS_CODE) == PROCFLAG.DEFINE_PROCESS_CODE):
                         self.nCode = int(numpy.fromfile(fp, '<u4', 1))
                         self.nBaud = int(numpy.fromfile(fp, '<u4', 1))
                         self.code = numpy.fromfile(
                             fp, '<f4', self.nCode * self.nBaud).reshape(self.nCode, self.nBaud)
                     if ((self.processFlags & PROCFLAG.EXP_NAME_ESP) == PROCFLAG.EXP_NAME_ESP):
                         exp_name_len = int(numpy.fromfile(fp, '<u4', 1))
                         exp_name = numpy.fromfile(fp, 'u1', exp_name_len + 1)
                     if ((self.processFlags & PROCFLAG.SHIFT_FFT_DATA) == PROCFLAG.SHIFT_FFT_DATA):
                         self.shif_fft = True
                     else:
                         self.shif_fft = False
                     if ((self.processFlags & PROCFLAG.SAVE_CHANNELS_DC) == PROCFLAG.SAVE_CHANNELS_DC):
                         self.flag_dc = True
                     else:
                         self.flag_dc = False
                     if ((self.processFlags & PROCFLAG.DECODE_DATA) == PROCFLAG.DECODE_DATA):
                         self.flag_decode = True
                     else:
                         self.flag_decode = False
                     if ((self.processFlags & PROCFLAG.DEFLIP_DATA) == PROCFLAG.DEFLIP_DATA):
                         self.flag_deflip = True
                     else:
                         self.flag_deflip = False
                     nChannels = 0
                     nPairs = 0
                     pairList = []
                     for i in range(0, self.totalSpectra * 2, 2):
                         if self.spectraComb[i] == self.spectraComb[i + 1]:
                             nChannels = nChannels + 1  # par de canales iguales
                         else:
                             nPairs = nPairs + 1  # par de canales diferentes
                             pairList.append((self.spectraComb[i], self.spectraComb[i + 1]))
                     self.flag_cspc = False
                     if nPairs > 0:
                         self.flag_cspc = True
                     if startFp is not None:
                         endFp = size + startFp
                         if fp.tell() > endFp:
                             sys.stderr.write(
                                 "Warning: Processing header size is lower than it has to be")
                             return 0
                         if fp.tell() < endFp:
                             sys.stderr.write(
                                 "Warning: Processing header size is greater than it is considered")
                     return 1
                 def write(self, fp):
                     # Clear DEFINE_PROCESS_CODE
                     self.processFlags = self.processFlags & (~PROCFLAG.DEFINE_PROCESS_CODE)
                     headerTuple = (self.size,
                                    self.dtype,
                                    self.blockSize,
                                    self.profilesPerBlock,
                                    self.dataBlocksPerFile,
                                    self.nWindows,
                                    self.processFlags,
                                    self.nCohInt,
                                    self.nIncohInt,
                                    self.totalSpectra)
                     header = numpy.array(headerTuple, PROCESSING_STRUCTURE)
                     header.tofile(fp)
                     if self.nWindows != 0:
                         sampleWindowTuple = (
                             self.firstHeight, self.deltaHeight, self.samplesWin)
                         samplingWindow = numpy.array(sampleWindowTuple, SAMPLING_STRUCTURE)
                         samplingWindow.tofile(fp)
                     if self.totalSpectra != 0:
                         #             spectraComb = numpy.array([],numpy.dtype('u1'))
                         spectraComb = self.spectraComb
                         spectraComb.tofile(fp)
             #         if self.processFlags & PROCFLAG.DEFINE_PROCESS_CODE == PROCFLAG.DEFINE_PROCESS_CODE:
             #             nCode = numpy.array([self.nCode], numpy.dtype('u4')) #Probar con un dato que almacene codigo, hasta el momento no se hizo la prueba
             #             nCode.tofile(fp)
             #
             #             nBaud = numpy.array([self.nBaud], numpy.dtype('u4'))
             #             nBaud.tofile(fp)
             #
             #             code = self.code.reshape(self.nCode*self.nBaud)
             #             code = code.astype(numpy.dtype('<f4'))
             #             code.tofile(fp)
                     return 1
                 def get_size(self):
                     self.__size = 40 + 12 * self.nWindows + 2 * self.totalSpectra
             #         if self.processFlags & PROCFLAG.DEFINE_PROCESS_CODE == PROCFLAG.DEFINE_PROCESS_CODE:
             #             self.__size += 4 + 4 + 4*self.nCode*numpy.ceil(self.nBaud/32.)
             #             self.__size += 4 + 4 + 4 * self.nCode * self.nBaud
                     return self.__size
                 def set_size(self, value):
                     raise IOError, "size is a property and it cannot be set, just read"
                     return
                 size = property(get_size, set_size)
             class RCfunction:
                 NONE = 0
                 FLIP = 1
                 CODE = 2
                 SAMPLING = 3
                 LIN6DIV256 = 4
                 SYNCHRO = 5
             class nCodeType:
                 NONE = 0
                 USERDEFINE = 1
                 BARKER2 = 2
                 BARKER3 = 3
                 BARKER4 = 4
                 BARKER5 = 5
                 BARKER7 = 6
                 BARKER11 = 7
                 BARKER13 = 8
                 AC128 = 9
                 COMPLEMENTARYCODE2 = 10
                 COMPLEMENTARYCODE4 = 11
                 COMPLEMENTARYCODE8 = 12
                 COMPLEMENTARYCODE16 = 13
                 COMPLEMENTARYCODE32 = 14
                 COMPLEMENTARYCODE64 = 15
                 COMPLEMENTARYCODE128 = 16
                 CODE_BINARY28 = 17
             class PROCFLAG:
                 COHERENT_INTEGRATION = numpy.uint32(0x00000001)
                 DECODE_DATA = numpy.uint32(0x00000002)
                 SPECTRA_CALC = numpy.uint32(0x00000004)
                 INCOHERENT_INTEGRATION = numpy.uint32(0x00000008)
                 POST_COHERENT_INTEGRATION = numpy.uint32(0x00000010)
                 SHIFT_FFT_DATA = numpy.uint32(0x00000020)
                 DATATYPE_CHAR = numpy.uint32(0x00000040)
                 DATATYPE_SHORT = numpy.uint32(0x00000080)
                 DATATYPE_LONG = numpy.uint32(0x00000100)
                 DATATYPE_INT64 = numpy.uint32(0x00000200)
                 DATATYPE_FLOAT = numpy.uint32(0x00000400)
                 DATATYPE_DOUBLE = numpy.uint32(0x00000800)
                 DATAARRANGE_CONTIGUOUS_CH = numpy.uint32(0x00001000)
                 DATAARRANGE_CONTIGUOUS_H = numpy.uint32(0x00002000)
                 DATAARRANGE_CONTIGUOUS_P = numpy.uint32(0x00004000)
                 SAVE_CHANNELS_DC = numpy.uint32(0x00008000)
                 DEFLIP_DATA = numpy.uint32(0x00010000)
                 DEFINE_PROCESS_CODE = numpy.uint32(0x00020000)
                 ACQ_SYS_NATALIA = numpy.uint32(0x00040000)
                 ACQ_SYS_ECHOTEK = numpy.uint32(0x00080000)
                 ACQ_SYS_ADRXD = numpy.uint32(0x000C0000)
                 ACQ_SYS_JULIA = numpy.uint32(0x00100000)
                 ACQ_SYS_XXXXXX = numpy.uint32(0x00140000)
                 EXP_NAME_ESP = numpy.uint32(0x00200000)
                 CHANNEL_NAMES_ESP = numpy.uint32(0x00400000)
                 OPERATION_MASK = numpy.uint32(0x0000003F)
                 DATATYPE_MASK = numpy.uint32(0x00000FC0)
                 DATAARRANGE_MASK = numpy.uint32(0x00007000)
                 ACQ_SYS_MASK = numpy.uint32(0x001C0000)
             dtype0 = numpy.dtype([('real', '<i1'), ('imag', '<i1')])
             dtype1 = numpy.dtype([('real', '<i2'), ('imag', '<i2')])
             dtype2 = numpy.dtype([('real', '<i4'), ('imag', '<i4')])
             dtype3 = numpy.dtype([('real', '<i8'), ('imag', '<i8')])
             dtype4 = numpy.dtype([('real', '<f4'), ('imag', '<f4')])
             dtype5 = numpy.dtype([('real', '<f8'), ('imag', '<f8')])
             NUMPY_DTYPE_LIST = [dtype0, dtype1, dtype2, dtype3, dtype4, dtype5]
             PROCFLAG_DTYPE_LIST = [PROCFLAG.DATATYPE_CHAR,
                                    PROCFLAG.DATATYPE_SHORT,
                                    PROCFLAG.DATATYPE_LONG,
                                    PROCFLAG.DATATYPE_INT64,
                                    PROCFLAG.DATATYPE_FLOAT,
                                    PROCFLAG.DATATYPE_DOUBLE]
             DTYPE_WIDTH = [1, 2, 4, 8, 4, 8]
             def get_dtype_index(numpy_dtype):
                 index = None
                 for i in range(len(NUMPY_DTYPE_LIST)):
                     if numpy_dtype == NUMPY_DTYPE_LIST[i]:
                         index = i
                         break
                 return index
             def get_numpy_dtype(index):
                 return NUMPY_DTYPE_LIST[index]
             def get_procflag_dtype(index):
                 return PROCFLAG_DTYPE_LIST[index]
             def get_dtype_width(index):
                 return DTYPE_WIDTH[index]

schainpy/model/graphics/jroplot_data.py +27 -17

             import os
             import time
             import glob
             import datetime
             from multiprocessing import Process
             import zmq
             import numpy
             import matplotlib
             import matplotlib.pyplot as plt
             from mpl_toolkits.axes_grid1 import make_axes_locatable
             from matplotlib.ticker import FuncFormatter, LinearLocator, MultipleLocator
             from schainpy.model.proc.jroproc_base import Operation
             from schainpy.utils import log
             jet_values = matplotlib.pyplot.get_cmap('jet', 100)(numpy.arange(100))[10:90]
             blu_values = matplotlib.pyplot.get_cmap(
                 'seismic_r', 20)(numpy.arange(20))[10:15]
             ncmap = matplotlib.colors.LinearSegmentedColormap.from_list(
                 'jro', numpy.vstack((blu_values, jet_values)))
             matplotlib.pyplot.register_cmap(cmap=ncmap)
-            CMAPS = [plt.get_cmap(s) for s in ('jro', 'jet', 'RdBu_r', 'seismic')]
+            CMAPS = [plt.get_cmap(s) for s in ('jro', 'jet', 'viridis', 'plasma', 'inferno', 'Greys', 'seismic', 'bwr', 'coolwarm')]
             def figpause(interval):
                 backend = plt.rcParams['backend']
                 if backend in matplotlib.rcsetup.interactive_bk:
                     figManager = matplotlib._pylab_helpers.Gcf.get_active()
                     if figManager is not None:
                         canvas = figManager.canvas
                         if canvas.figure.stale:
                             canvas.draw()
                         canvas.start_event_loop(interval)
                         return
             class PlotData(Operation, Process):
                 '''
                 Base class for Schain plotting operations
                 '''
                 CODE = 'Figure'
                 colormap = 'jro'
                 bgcolor = 'white'
                 CONFLATE = False
-                __MAXNUMX = 80
                 __missing = 1E30
                 __attrs__ = ['show', 'save', 'xmin', 'xmax', 'ymin', 'ymax', 'zmin', 'zmax',
                              'zlimits', 'xlabel', 'ylabel', 'xaxis','cb_label', 'title',
                              'colorbar', 'bgcolor', 'width', 'height', 'localtime', 'oneFigure',
                              'showprofile', 'decimation']
                 def __init__(self, **kwargs):
                     Operation.__init__(self, plot=True, **kwargs)
                     Process.__init__(self)
                     self.kwargs['code'] = self.CODE
                     self.mp = False
                     self.data = None
                     self.isConfig = False
                     self.figures = []
                     self.axes = []
                     self.cb_axes = []
                     self.localtime = kwargs.pop('localtime', True)
                     self.show = kwargs.get('show', True)
                     self.save = kwargs.get('save', False)
                     self.colormap = kwargs.get('colormap', self.colormap)
                     self.colormap_coh = kwargs.get('colormap_coh', 'jet')
                     self.colormap_phase = kwargs.get('colormap_phase', 'RdBu_r')
                     self.colormaps = kwargs.get('colormaps', None)
                     self.bgcolor = kwargs.get('bgcolor', self.bgcolor)
                     self.showprofile = kwargs.get('showprofile', False)
                     self.title = kwargs.get('wintitle', self.CODE.upper())
                     self.cb_label = kwargs.get('cb_label', None)
                     self.cb_labels = kwargs.get('cb_labels', None)
                     self.xaxis = kwargs.get('xaxis', 'frequency')
                     self.zmin = kwargs.get('zmin', None)
                     self.zmax = kwargs.get('zmax', None)
                     self.zlimits = kwargs.get('zlimits', None)
                     self.xmin = kwargs.get('xmin', None)
                     self.xmax = kwargs.get('xmax', None)
                     self.xrange = kwargs.get('xrange', 24)
                     self.ymin = kwargs.get('ymin', None)
                     self.ymax = kwargs.get('ymax', None)
                     self.xlabel = kwargs.get('xlabel', None)
-                    self.__MAXNUMY = kwargs.get('decimation', 200)
+                    self.decimation = kwargs.get('decimation', None)
                     self.showSNR = kwargs.get('showSNR', False)
                     self.oneFigure = kwargs.get('oneFigure', True)
                     self.width = kwargs.get('width', None)
                     self.height = kwargs.get('height', None)
                     self.colorbar = kwargs.get('colorbar', True)
                     self.factors = kwargs.get('factors', [1, 1, 1, 1, 1, 1, 1, 1])
                     self.titles = kwargs.get('titles', [])
                     self.polar = False
                 def __fmtTime(self, x, pos):
                     '''
                     '''
                     return '{}'.format(self.getDateTime(x).strftime('%H:%M'))
                 def __setup(self):
                     '''
                     Common setup for all figures, here figures and axes are created
                     '''
                     if self.CODE not in self.data:
                         raise ValueError(log.error('Missing data for {}'.format(self.CODE),
                                                    self.name))
                     self.setup()
                     self.time_label = 'LT' if self.localtime else 'UTC'
                     if self.data.localtime:
                         self.getDateTime = datetime.datetime.fromtimestamp
                     else:
                         self.getDateTime = datetime.datetime.utcfromtimestamp
                     if self.width is None:
                         self.width = 8
                     self.figures = []
                     self.axes = []
                     self.cb_axes = []
                     self.pf_axes = []
                     self.cmaps = []
                     size = '15%' if self.ncols == 1 else '30%'
                     pad = '4%' if self.ncols == 1 else '8%'
                     if self.oneFigure:
                         if self.height is None:
                             self.height = 1.4 * self.nrows + 1
                         fig = plt.figure(figsize=(self.width, self.height),
                                          edgecolor='k',
                                          facecolor='w')
                         self.figures.append(fig)
                         for n in range(self.nplots):
                             ax = fig.add_subplot(self.nrows, self.ncols,
                                                  n + 1, polar=self.polar)
                             ax.tick_params(labelsize=8)
                             ax.firsttime = True
                             ax.index = 0
                             ax.press = None
                             self.axes.append(ax)
                             if self.showprofile:
                                 cax = self.__add_axes(ax, size=size, pad=pad)
                                 cax.tick_params(labelsize=8)
                                 self.pf_axes.append(cax)
                     else:
                         if self.height is None:
                             self.height = 3
                         for n in range(self.nplots):
                             fig = plt.figure(figsize=(self.width, self.height),
                                              edgecolor='k',
                                              facecolor='w')
                             ax = fig.add_subplot(1, 1, 1, polar=self.polar)
                             ax.tick_params(labelsize=8)
                             ax.firsttime = True
                             ax.index = 0
                             ax.press = None
                             self.figures.append(fig)
                             self.axes.append(ax)
                             if self.showprofile:
                                 cax = self.__add_axes(ax, size=size, pad=pad)
                                 cax.tick_params(labelsize=8)
                                 self.pf_axes.append(cax)
                     for n in range(self.nrows):
                         if self.colormaps is not None:
                             cmap = plt.get_cmap(self.colormaps[n])
                         else:
                             cmap = plt.get_cmap(self.colormap)
                         cmap.set_bad(self.bgcolor, 1.)
                         self.cmaps.append(cmap)
                     for fig in self.figures:
                         fig.canvas.mpl_connect('key_press_event', self.OnKeyPress)
                         fig.canvas.mpl_connect('scroll_event', self.OnBtnScroll)
                         fig.canvas.mpl_connect('button_press_event', self.onBtnPress)
                         fig.canvas.mpl_connect('motion_notify_event', self.onMotion)
                         fig.canvas.mpl_connect('button_release_event', self.onBtnRelease)
                         if self.show:
                             fig.show()
                 def OnKeyPress(self, event):
                     '''
                     Event for pressing keys (up, down) change colormap
                     '''
                     ax = event.inaxes
                     if ax in self.axes:
                         if event.key == 'down':
                             ax.index += 1
                         elif event.key == 'up':
                             ax.index -= 1
                         if ax.index < 0:
                             ax.index = len(CMAPS) - 1
                         elif ax.index == len(CMAPS):
                             ax.index = 0
                         cmap = CMAPS[ax.index]
                         ax.cbar.set_cmap(cmap)
                         ax.cbar.draw_all()
                         ax.plt.set_cmap(cmap)
                         ax.cbar.patch.figure.canvas.draw()
                         self.colormap = cmap.name
                 def OnBtnScroll(self, event):
                     '''
                     Event for scrolling, scale figure
                     '''
                     cb_ax = event.inaxes
                     if cb_ax in [ax.cbar.ax for ax in self.axes if ax.cbar]:
                         ax = [ax for ax in self.axes if cb_ax == ax.cbar.ax][0]
                         pt = ax.cbar.ax.bbox.get_points()[:, 1]
                         nrm = ax.cbar.norm
                         vmin, vmax, p0, p1, pS = (
                             nrm.vmin, nrm.vmax, pt[0], pt[1], event.y)
                         scale = 2 if event.step == 1 else 0.5
                         point = vmin + (vmax - vmin) / (p1 - p0) * (pS - p0)
                         ax.cbar.norm.vmin = point - scale * (point - vmin)
                         ax.cbar.norm.vmax = point - scale * (point - vmax)
                         ax.plt.set_norm(ax.cbar.norm)
                         ax.cbar.draw_all()
                         ax.cbar.patch.figure.canvas.draw()
                 def onBtnPress(self, event):
                     '''
                     Event for mouse button press
                     '''
                     cb_ax = event.inaxes
                     if cb_ax is None:
                         return
                     if cb_ax in [ax.cbar.ax for ax in self.axes if ax.cbar]:
                         cb_ax.press = event.x, event.y
                     else:
                         cb_ax.press = None
                 def onMotion(self, event):
                     '''
                     Event for move inside colorbar
                     '''
                     cb_ax = event.inaxes
                     if cb_ax is None:
                         return
                     if cb_ax not in [ax.cbar.ax for ax in self.axes if ax.cbar]:
                         return
                     if cb_ax.press is None:
                         return
                     ax = [ax for ax in self.axes if cb_ax == ax.cbar.ax][0]
                     xprev, yprev = cb_ax.press
                     dx = event.x - xprev
                     dy = event.y - yprev
                     cb_ax.press = event.x, event.y
                     scale = ax.cbar.norm.vmax - ax.cbar.norm.vmin
                     perc = 0.03
                     if event.button == 1:
                         ax.cbar.norm.vmin -= (perc * scale) * numpy.sign(dy)
                         ax.cbar.norm.vmax -= (perc * scale) * numpy.sign(dy)
                     elif event.button == 3:
                         ax.cbar.norm.vmin -= (perc * scale) * numpy.sign(dy)
                         ax.cbar.norm.vmax += (perc * scale) * numpy.sign(dy)
                     ax.cbar.draw_all()
                     ax.plt.set_norm(ax.cbar.norm)
                     ax.cbar.patch.figure.canvas.draw()
                 def onBtnRelease(self, event):
                     '''
                     Event for mouse button release
                     '''
                     cb_ax = event.inaxes
                     if cb_ax is not None:
                         cb_ax.press = None
                 def __add_axes(self, ax, size='30%', pad='8%'):
                     '''
                     Add new axes to the given figure
                     '''
                     divider = make_axes_locatable(ax)
                     nax = divider.new_horizontal(size=size, pad=pad)
                     ax.figure.add_axes(nax)
                     return nax
                     self.setup()
                 def setup(self):
                     '''
                     This method should be implemented in the child class, the following
                     attributes should be set:
                     self.nrows: number of rows
                     self.ncols: number of cols
                     self.nplots: number of plots (channels or pairs)
                     self.ylabel: label for Y axes
                     self.titles: list of axes title
                     '''
                     raise(NotImplementedError, 'Implement this method in child class')
                 def fill_gaps(self, x_buffer, y_buffer, z_buffer):
                     '''
                     Create a masked array for missing data
                     '''
                     if x_buffer.shape[0] < 2:
                         return x_buffer, y_buffer, z_buffer
                     deltas = x_buffer[1:] - x_buffer[0:-1]
                     x_median = numpy.median(deltas)
                     index = numpy.where(deltas > 5 * x_median)
                     if len(index[0]) != 0:
                         z_buffer[::, index[0], ::] = self.__missing
                         z_buffer = numpy.ma.masked_inside(z_buffer,
 .99 * self.__missing,
 .01 * self.__missing)
                     return x_buffer, y_buffer, z_buffer
                 def decimate(self):
                     # dx = int(len(self.x)/self.__MAXNUMX) + 1
-                    dy = int(len(self.y) / self.__MAXNUMY) + 1
+                    dy = int(len(self.y) / self.decimation) + 1
                     # x = self.x[::dx]
                     x = self.x
                     y = self.y[::dy]
                     z = self.z[::, ::, ::dy]
                     return x, y, z
                 def format(self):
                     '''
                     Set min and max values, labels, ticks and titles
                     '''
                     if self.xmin is None:
                         xmin = self.min_time
                     else:
                         if self.xaxis is 'time':
                             dt = self.getDateTime(self.min_time)
                             xmin = (dt.replace(hour=int(self.xmin), minute=0, second=0) -
                                     datetime.datetime(1970, 1, 1)).total_seconds()
                             if self.data.localtime:
                                 xmin += time.timezone
                         else:
                             xmin = self.xmin
                     if self.xmax is None:
                         xmax = xmin + self.xrange * 60 * 60
                     else:
                         if self.xaxis is 'time':
                             dt = self.getDateTime(self.max_time)
                             xmax = (dt.replace(hour=int(self.xmax), minute=59, second=59) -
                                     datetime.datetime(1970, 1, 1) + datetime.timedelta(seconds=1)).total_seconds()
                             if self.data.localtime:
                                 xmax += time.timezone
                         else:
                             xmax = self.xmax
                     ymin = self.ymin if self.ymin else numpy.nanmin(self.y)
                     ymax = self.ymax if self.ymax else numpy.nanmax(self.y)
-                    Y = numpy.array([10, 20, 50, 100, 200, 500, 1000, 2000])
+                    Y = numpy.array([5, 10, 20, 50, 100, 200, 500, 1000, 2000])
                     i = 1 if numpy.where(ymax-ymin < Y)[0][0] < 0 else numpy.where(ymax-ymin < Y)[0][0]
                     ystep = Y[i] / 5
                     for n, ax in enumerate(self.axes):
                         if ax.firsttime:
                             ax.set_facecolor(self.bgcolor)
                             ax.yaxis.set_major_locator(MultipleLocator(ystep))
                             if self.xaxis is 'time':
                                 ax.xaxis.set_major_formatter(FuncFormatter(self.__fmtTime))
                                 ax.xaxis.set_major_locator(LinearLocator(9))
                             if self.xlabel is not None:
                                 ax.set_xlabel(self.xlabel)
                             ax.set_ylabel(self.ylabel)
                             ax.firsttime = False
                             if self.showprofile:
                                 self.pf_axes[n].set_ylim(ymin, ymax)
                                 self.pf_axes[n].set_xlim(self.zmin, self.zmax)
                                 self.pf_axes[n].set_xlabel('dB')
                                 self.pf_axes[n].grid(b=True, axis='x')
                                 [tick.set_visible(False)
                                  for tick in self.pf_axes[n].get_yticklabels()]
                             if self.colorbar:
                                 ax.cbar = plt.colorbar(
                                     ax.plt, ax=ax, fraction=0.05, pad=0.02, aspect=10)
                                 ax.cbar.ax.tick_params(labelsize=8)
                                 ax.cbar.ax.press = None
                                 if self.cb_label:
                                     ax.cbar.set_label(self.cb_label, size=8)
                                 elif self.cb_labels:
                                     ax.cbar.set_label(self.cb_labels[n], size=8)
                             else:
                                 ax.cbar = None
                         if not self.polar:
                             ax.set_xlim(xmin, xmax)
                             ax.set_ylim(ymin, ymax)
                             ax.set_title('{} - {} {}'.format(
                                 self.titles[n],
                                 self.getDateTime(self.max_time).strftime('%H:%M:%S'),
                                 self.time_label),
                                 size=8)
                         else:
                             ax.set_title('{}'.format(self.titles[n]), size=8)
                             ax.set_ylim(0, 90)
                             ax.set_yticks(numpy.arange(0, 90, 20))
                             ax.yaxis.labelpad = 40
                 def __plot(self):
                     '''
                     '''
                     log.success('Plotting', self.name)
-                    self.plot()
+                    try:
-                    self.format()
+                        self.plot()
+                        self.format()
+                    except:
+                        log.warning('{} Plot could not be updated... check data'.format(self.CODE), self.name)
                     for n, fig in enumerate(self.figures):
                         if self.nrows == 0 or self.nplots == 0:
                             log.warning('No data', self.name)
                             fig.text(0.5, 0.5, 'No Data', fontsize='large', ha='center')
                             fig.canvas.manager.set_window_title(self.CODE)
                             continue
                         fig.tight_layout()
                         fig.canvas.manager.set_window_title('{} - {}'.format(self.title,
                                                                              self.getDateTime(self.max_time).strftime('%Y/%m/%d')))
                         fig.canvas.draw()
                         if self.save and self.data.ended:
                             channels = range(self.nrows)
                             if self.oneFigure:
                                 label = ''
                             else:
                                 label = '_{}'.format(channels[n])
                             figname = os.path.join(
                                 self.save,
                                 '{}{}_{}.png'.format(
                                     self.CODE,
                                     label,
                                     self.getDateTime(self.saveTime).strftime(
-                                        '%y%m%d_%H%M%S'),
+                                        '%Y%m%d_%H%M%S'),
                                 )
                             )
                             log.log('Saving figure: {}'.format(figname), self.name)
                             fig.savefig(figname)
                 def plot(self):
                     '''
                     '''
                     raise(NotImplementedError, 'Implement this method in child class')
                 def run(self):
                     log.success('Starting', self.name)
                     context = zmq.Context()
                     receiver = context.socket(zmq.SUB)
                     receiver.setsockopt(zmq.SUBSCRIBE, '')
                     receiver.setsockopt(zmq.CONFLATE, self.CONFLATE)
                     if 'server' in self.kwargs['parent']:
                         receiver.connect(
                             'ipc:///tmp/{}.plots'.format(self.kwargs['parent']['server']))
                     else:
                         receiver.connect("ipc:///tmp/zmq.plots")
                     while True:
                         try:
                             self.data = receiver.recv_pyobj(flags=zmq.NOBLOCK)
                             if self.data.localtime and self.localtime:
                                 self.times = self.data.times
                             elif self.data.localtime and not self.localtime:
                                 self.times = self.data.times + time.timezone
                             elif not self.data.localtime and self.localtime:
                                 self.times = self.data.times - time.timezone
                             else:
                                 self.times = self.data.times
                             self.min_time = self.times[0]
                             self.max_time = self.times[-1]
                             if self.isConfig is False:
                                 self.__setup()
                                 self.isConfig = True
                             self.__plot()
                         except zmq.Again as e:
                             log.log('Waiting for data...')
                             if self.data:
                                 figpause(self.data.throttle)
                             else:
                                 time.sleep(2)
                 def close(self):
                     if self.data:
                         self.__plot()
             class PlotSpectraData(PlotData):
                 '''
                 Plot for Spectra data
                 '''
                 CODE = 'spc'
                 colormap = 'jro'
                 def setup(self):
                     self.nplots = len(self.data.channels)
                     self.ncols = int(numpy.sqrt(self.nplots) + 0.9)
                     self.nrows = int((1.0 * self.nplots / self.ncols) + 0.9)
                     self.width = 3.4 * self.ncols
                     self.height = 3 * self.nrows
                     self.cb_label = 'dB'
                     if self.showprofile:
                         self.width += 0.8 * self.ncols
-                    self.ylabel = 'Range [Km]'
+                    self.ylabel = 'Range [km]'
                 def plot(self):
                     if self.xaxis == "frequency":
                         x = self.data.xrange[0]
                         self.xlabel = "Frequency (kHz)"
                     elif self.xaxis == "time":
                         x = self.data.xrange[1]
                         self.xlabel = "Time (ms)"
                     else:
                         x = self.data.xrange[2]
                         self.xlabel = "Velocity (m/s)"
                     if self.CODE == 'spc_mean':
                         x = self.data.xrange[2]
                         self.xlabel = "Velocity (m/s)"
                     self.titles = []
                     y = self.data.heights
                     self.y = y
                     z = self.data['spc']
                     for n, ax in enumerate(self.axes):
                         noise = self.data['noise'][n][-1]
                         if self.CODE == 'spc_mean':
                             mean = self.data['mean'][n][-1]
                         if ax.firsttime:
                             self.xmax = self.xmax if self.xmax else numpy.nanmax(x)
                             self.xmin = self.xmin if self.xmin else -self.xmax
                             self.zmin = self.zmin if self.zmin else numpy.nanmin(z)
                             self.zmax = self.zmax if self.zmax else numpy.nanmax(z)
                             ax.plt = ax.pcolormesh(x, y, z[n].T,
                                                    vmin=self.zmin,
                                                    vmax=self.zmax,
                                                    cmap=plt.get_cmap(self.colormap)
                                                    )
                             if self.showprofile:
                                 ax.plt_profile = self.pf_axes[n].plot(
                                     self.data['rti'][n][-1], y)[0]
                                 ax.plt_noise = self.pf_axes[n].plot(numpy.repeat(noise, len(y)), y,
                                                                     color="k", linestyle="dashed", lw=1)[0]
                             if self.CODE == 'spc_mean':
                                 ax.plt_mean = ax.plot(mean, y, color='k')[0]
                         else:
                             ax.plt.set_array(z[n].T.ravel())
                             if self.showprofile:
                                 ax.plt_profile.set_data(self.data['rti'][n][-1], y)
                                 ax.plt_noise.set_data(numpy.repeat(noise, len(y)), y)
                             if self.CODE == 'spc_mean':
                                 ax.plt_mean.set_data(mean, y)
                         self.titles.append('CH {}: {:3.2f}dB'.format(n, noise))
                         self.saveTime = self.max_time
             class PlotCrossSpectraData(PlotData):
                 CODE = 'cspc'
                 zmin_coh = None
                 zmax_coh = None
                 zmin_phase = None
                 zmax_phase = None
                 def setup(self):
                     self.ncols = 4
                     self.nrows = len(self.data.pairs)
                     self.nplots = self.nrows * 4
                     self.width = 3.4 * self.ncols
                     self.height = 3 * self.nrows
-                    self.ylabel = 'Range [Km]'
+                    self.ylabel = 'Range [km]'
                     self.showprofile = False
                 def plot(self):
                     if self.xaxis == "frequency":
                         x = self.data.xrange[0]
                         self.xlabel = "Frequency (kHz)"
                     elif self.xaxis == "time":
                         x = self.data.xrange[1]
                         self.xlabel = "Time (ms)"
                     else:
                         x = self.data.xrange[2]
                         self.xlabel = "Velocity (m/s)"
                     self.titles = []
                     y = self.data.heights
                     self.y = y
                     spc = self.data['spc']
                     cspc = self.data['cspc']
                     for n in range(self.nrows):
                         noise = self.data['noise'][n][-1]
                         pair = self.data.pairs[n]
                         ax = self.axes[4 * n]
                         ax3 = self.axes[4 * n + 3]
                         if ax.firsttime:
                             self.xmax = self.xmax if self.xmax else numpy.nanmax(x)
                             self.xmin = self.xmin if self.xmin else -self.xmax
                             self.zmin = self.zmin if self.zmin else numpy.nanmin(spc)
                             self.zmax = self.zmax if self.zmax else numpy.nanmax(spc)
                             ax.plt = ax.pcolormesh(x, y, spc[pair[0]].T,
                                                    vmin=self.zmin,
                                                    vmax=self.zmax,
                                                    cmap=plt.get_cmap(self.colormap)
                                                    )
                         else:
                             ax.plt.set_array(spc[pair[0]].T.ravel())
                         self.titles.append('CH {}: {:3.2f}dB'.format(n, noise))
                         ax = self.axes[4 * n + 1]
                         if ax.firsttime:
                             ax.plt = ax.pcolormesh(x, y, spc[pair[1]].T,
                                                    vmin=self.zmin,
                                                    vmax=self.zmax,
                                                    cmap=plt.get_cmap(self.colormap)
                                                    )
                         else:
                             ax.plt.set_array(spc[pair[1]].T.ravel())
                         self.titles.append('CH {}: {:3.2f}dB'.format(n, noise))
                         out = cspc[n] / numpy.sqrt(spc[pair[0]] * spc[pair[1]])
                         coh = numpy.abs(out)
                         phase = numpy.arctan2(out.imag, out.real) * 180 / numpy.pi
                         ax = self.axes[4 * n + 2]
                         if ax.firsttime:
                             ax.plt = ax.pcolormesh(x, y, coh.T,
                                                    vmin=0,
                                                    vmax=1,
                                                    cmap=plt.get_cmap(self.colormap_coh)
                                                    )
                         else:
                             ax.plt.set_array(coh.T.ravel())
                         self.titles.append(
                             'Coherence Ch{} * Ch{}'.format(pair[0], pair[1]))
                         ax = self.axes[4 * n + 3]
                         if ax.firsttime:
                             ax.plt = ax.pcolormesh(x, y, phase.T,
                                                    vmin=-180,
                                                    vmax=180,
                                                    cmap=plt.get_cmap(self.colormap_phase)
                                                    )
                         else:
                             ax.plt.set_array(phase.T.ravel())
                         self.titles.append('Phase CH{} * CH{}'.format(pair[0], pair[1]))
                         self.saveTime = self.max_time
             class PlotSpectraMeanData(PlotSpectraData):
                 '''
                 Plot for Spectra and Mean
                 '''
                 CODE = 'spc_mean'
                 colormap = 'jro'
             class PlotRTIData(PlotData):
                 '''
                 Plot for RTI data
                 '''
                 CODE = 'rti'
                 colormap = 'jro'
                 def setup(self):
                     self.xaxis = 'time'
                     self.ncols = 1
                     self.nrows = len(self.data.channels)
                     self.nplots = len(self.data.channels)
-                    self.ylabel = 'Range [Km]'
+                    self.ylabel = 'Range [km]'
                     self.cb_label = 'dB'
                     self.titles = ['{} Channel {}'.format(
                         self.CODE.upper(), x) for x in range(self.nrows)]
                 def plot(self):
                     self.x = self.times
                     self.y = self.data.heights
                     self.z = self.data[self.CODE]
                     self.z = numpy.ma.masked_invalid(self.z)
-                    for n, ax in enumerate(self.axes):
+                    if self.decimation is None:
+                        x, y, z = self.fill_gaps(self.x, self.y, self.z)
+                    else:
                         x, y, z = self.fill_gaps(*self.decimate())
+                    for n, ax in enumerate(self.axes):
                         self.zmin = self.zmin if self.zmin else numpy.min(self.z)
                         self.zmax = self.zmax if self.zmax else numpy.max(self.z)
                         if ax.firsttime:
                             ax.plt = ax.pcolormesh(x, y, z[n].T,
                                                    vmin=self.zmin,
                                                    vmax=self.zmax,
                                                    cmap=plt.get_cmap(self.colormap)
                                                    )
                             if self.showprofile:
                                 ax.plot_profile = self.pf_axes[n].plot(
                                     self.data['rti'][n][-1], self.y)[0]
                                 ax.plot_noise = self.pf_axes[n].plot(numpy.repeat(self.data['noise'][n][-1], len(self.y)), self.y,
                                                                      color="k", linestyle="dashed", lw=1)[0]
                         else:
                             ax.collections.remove(ax.collections[0])
                             ax.plt = ax.pcolormesh(x, y, z[n].T,
                                                    vmin=self.zmin,
                                                    vmax=self.zmax,
                                                    cmap=plt.get_cmap(self.colormap)
                                                    )
                             if self.showprofile:
                                 ax.plot_profile.set_data(self.data['rti'][n][-1], self.y)
                                 ax.plot_noise.set_data(numpy.repeat(
                                     self.data['noise'][n][-1], len(self.y)), self.y)
                     self.saveTime = self.min_time
             class PlotCOHData(PlotRTIData):
                 '''
                 Plot for Coherence data
                 '''
                 CODE = 'coh'
                 def setup(self):
                     self.xaxis = 'time'
                     self.ncols = 1
                     self.nrows = len(self.data.pairs)
                     self.nplots = len(self.data.pairs)
-                    self.ylabel = 'Range [Km]'
+                    self.ylabel = 'Range [km]'
                     if self.CODE == 'coh':
                         self.cb_label = ''
                         self.titles = [
                             'Coherence Map Ch{} * Ch{}'.format(x[0], x[1]) for x in self.data.pairs]
                     else:
                         self.cb_label = 'Degrees'
                         self.titles = [
                             'Phase Map Ch{} * Ch{}'.format(x[0], x[1]) for x in self.data.pairs]
             class PlotPHASEData(PlotCOHData):
                 '''
                 Plot for Phase map data
                 '''
                 CODE = 'phase'
                 colormap = 'seismic'
             class PlotNoiseData(PlotData):
                 '''
                 Plot for noise
                 '''
                 CODE = 'noise'
                 def setup(self):
                     self.xaxis = 'time'
                     self.ncols = 1
                     self.nrows = 1
                     self.nplots = 1
                     self.ylabel = 'Intensity [dB]'
                     self.titles = ['Noise']
                     self.colorbar = False
                 def plot(self):
                     x = self.times
                     xmin = self.min_time
                     xmax = xmin + self.xrange * 60 * 60
                     Y = self.data[self.CODE]
                     if self.axes[0].firsttime:
                         for ch in self.data.channels:
                             y = Y[ch]
                             self.axes[0].plot(x, y, lw=1, label='Ch{}'.format(ch))
                         plt.legend()
                     else:
                         for ch in self.data.channels:
                             y = Y[ch]
                             self.axes[0].lines[ch].set_data(x, y)
                     self.ymin = numpy.nanmin(Y) - 5
                     self.ymax = numpy.nanmax(Y) + 5
                     self.saveTime = self.min_time
             class PlotSNRData(PlotRTIData):
                 '''
                 Plot for SNR Data
                 '''
                 CODE = 'snr'
                 colormap = 'jet'
             class PlotDOPData(PlotRTIData):
                 '''
                 Plot for DOPPLER Data
                 '''
                 CODE = 'dop'
                 colormap = 'jet'
             class PlotSkyMapData(PlotData):
                 '''
                 Plot for meteors detection data
                 '''
                 CODE = 'param'
                 def setup(self):
                     self.ncols = 1
                     self.nrows = 1
                     self.width = 7.2
                     self.height = 7.2
                     self.nplots = 1
                     self.xlabel = 'Zonal Zenith Angle (deg)'
                     self.ylabel = 'Meridional Zenith Angle (deg)'
                     self.polar = True
                     self.ymin = -180
                     self.ymax = 180
                     self.colorbar = False
                 def plot(self):
                     arrayParameters = numpy.concatenate(self.data['param'])
                     error = arrayParameters[:, -1]
                     indValid = numpy.where(error == 0)[0]
                     finalMeteor = arrayParameters[indValid, :]
                     finalAzimuth = finalMeteor[:, 3]
                     finalZenith = finalMeteor[:, 4]
                     x = finalAzimuth * numpy.pi / 180
                     y = finalZenith
                     ax = self.axes[0]
                     if ax.firsttime:
                         ax.plot = ax.plot(x, y, 'bo', markersize=5)[0]
                     else:
                         ax.plot.set_data(x, y)
                     dt1 = self.getDateTime(self.min_time).strftime('%y/%m/%d %H:%M:%S')
                     dt2 = self.getDateTime(self.max_time).strftime('%y/%m/%d %H:%M:%S')
                     title = 'Meteor Detection Sky Map\n %s - %s \n Number of events: %5.0f\n' % (dt1,
                                                                                                  dt2,
                                                                                                  len(x))
                     self.titles[0] = title
                     self.saveTime = self.max_time
             class PlotParamData(PlotRTIData):
                 '''
                 Plot for data_param object
                 '''
                 CODE = 'param'
                 colormap = 'seismic'
                 def setup(self):
                     self.xaxis = 'time'
                     self.ncols = 1
                     self.nrows = self.data.shape(self.CODE)[0]
                     self.nplots = self.nrows
                     if self.showSNR:
                         self.nrows += 1
                         self.nplots += 1
-                    self.ylabel = 'Height [Km]'
+                    self.ylabel = 'Height [km]'
                     if not self.titles:
                         self.titles = self.data.parameters \
                             if self.data.parameters else ['Param {}'.format(x) for x in xrange(self.nrows)]
                         if self.showSNR:
                             self.titles.append('SNR')
                 def plot(self):
                     self.data.normalize_heights()
                     self.x = self.times
                     self.y = self.data.heights
                     if self.showSNR:
                         self.z = numpy.concatenate(
                             (self.data[self.CODE], self.data['snr'])
                         )
                     else:
                         self.z = self.data[self.CODE]
                     self.z = numpy.ma.masked_invalid(self.z)
-                    for n, ax in enumerate(self.axes):
+                    if self.decimation is None:
+                        x, y, z = self.fill_gaps(self.x, self.y, self.z)
+                    else:
                         x, y, z = self.fill_gaps(*self.decimate())
+                    for n, ax in enumerate(self.axes):
                         self.zmax = self.zmax if self.zmax is not None else numpy.max(
                             self.z[n])
                         self.zmin = self.zmin if self.zmin is not None else numpy.min(
                             self.z[n])
                         if ax.firsttime:
                             if self.zlimits is not None:
                                 self.zmin, self.zmax = self.zlimits[n]
                             ax.plt = ax.pcolormesh(x, y, z[n].T * self.factors[n],
                                                    vmin=self.zmin,
                                                    vmax=self.zmax,
                                                    cmap=self.cmaps[n]
                                                    )
                         else:
                             if self.zlimits is not None:
                                 self.zmin, self.zmax = self.zlimits[n]
                             ax.collections.remove(ax.collections[0])
                             ax.plt = ax.pcolormesh(x, y, z[n].T * self.factors[n],
                                                    vmin=self.zmin,
                                                    vmax=self.zmax,
                                                    cmap=self.cmaps[n]
                                                    )
                     self.saveTime = self.min_time
             class PlotOutputData(PlotParamData):
                 '''
                 Plot data_output object
                 '''
                 CODE = 'output'
                 colormap = 'seismic'

schainpy/model/graphics/mpldriver.py +12 -6

-            import numpy
+            import os
-            import datetime
             import sys
+            import datetime
+            import numpy
             import matplotlib
-            if 'linux' in sys.platform:
+            if 'BACKEND' in os.environ:
+                matplotlib.use(os.environ['BACKEND'])
+            elif 'linux' in sys.platform:
                 matplotlib.use("TkAgg")
+            elif 'darwin' in sys.platform:
-            if 'darwin' in sys.platform:
+                matplotlib.use('TkAgg')
-                matplotlib.use('TKAgg')
+            else:
+                from schainpy.utils import log
+                log.warning('Using default Backend="Agg"', 'INFO')
+                matplotlib.use('Agg')
             # Qt4Agg', 'GTK', 'GTKAgg', 'ps', 'agg', 'cairo', 'MacOSX', 'GTKCairo', 'WXAgg', 'template', 'TkAgg', 'GTK3Cairo', 'GTK3Agg', 'svg', 'WebAgg', 'CocoaAgg', 'emf', 'gdk', 'WX'
             import matplotlib.pyplot
             from mpl_toolkits.axes_grid1 import make_axes_locatable
             from matplotlib.ticker import FuncFormatter, LinearLocator
             ###########################################
             # Actualizacion de las funciones del driver
             ###########################################
             # create jro colormap
             jet_values = matplotlib.pyplot.get_cmap("jet", 100)(numpy.arange(100))[10:90]
             blu_values = matplotlib.pyplot.get_cmap(
                 "seismic_r", 20)(numpy.arange(20))[10:15]
             ncmap = matplotlib.colors.LinearSegmentedColormap.from_list(
                 "jro", numpy.vstack((blu_values, jet_values)))
             matplotlib.pyplot.register_cmap(cmap=ncmap)
             def createFigure(id, wintitle, width, height, facecolor="w", show=True, dpi=80):
                 matplotlib.pyplot.ioff()
                 fig = matplotlib.pyplot.figure(num=id, facecolor=facecolor, figsize=(
 .0 * width / dpi, 1.0 * height / dpi))
                 fig.canvas.manager.set_window_title(wintitle)
             #     fig.canvas.manager.resize(width, height)
                 matplotlib.pyplot.ion()
                 if show:
                     matplotlib.pyplot.show()
                 return fig
             def closeFigure(show=False, fig=None):
                 #     matplotlib.pyplot.ioff()
                 #     matplotlib.pyplot.pause(0)
                 if show:
                     matplotlib.pyplot.show()
                 if fig != None:
                     matplotlib.pyplot.close(fig)
             #         matplotlib.pyplot.pause(0)
             #         matplotlib.pyplot.ion()
                     return
                 matplotlib.pyplot.close("all")
             #     matplotlib.pyplot.pause(0)
             #     matplotlib.pyplot.ion()
                 return
             def saveFigure(fig, filename):
                 #     matplotlib.pyplot.ioff()
                 fig.savefig(filename, dpi=matplotlib.pyplot.gcf().dpi)
             #     matplotlib.pyplot.ion()
             def clearFigure(fig):
                 fig.clf()
             def setWinTitle(fig, title):
                 fig.canvas.manager.set_window_title(title)
             def setTitle(fig, title):
                 fig.suptitle(title)
             def createAxes(fig, nrow, ncol, xpos, ypos, colspan, rowspan, polar=False):
                 matplotlib.pyplot.ioff()
                 matplotlib.pyplot.figure(fig.number)
                 axes = matplotlib.pyplot.subplot2grid((nrow, ncol),
                                                       (xpos, ypos),
                                                       colspan=colspan,
                                                       rowspan=rowspan,
                                                       polar=polar)
                 matplotlib.pyplot.ion()
                 return axes
             def setAxesText(ax, text):
                 ax.annotate(text,
                             xy=(.1, .99),
                             xycoords='figure fraction',
                             horizontalalignment='left',
                             verticalalignment='top',
                             fontsize=10)
             def printLabels(ax, xlabel, ylabel, title):
                 ax.set_xlabel(xlabel, size=11)
                 ax.set_ylabel(ylabel, size=11)
                 ax.set_title(title, size=8)
             def createPline(ax, x, y, xmin, xmax, ymin, ymax, xlabel='', ylabel='', title='',
                             ticksize=9, xtick_visible=True, ytick_visible=True,
                             nxticks=4, nyticks=10,
                             grid=None, color='blue'):
                 """
                 Input:
                     grid    :    None, 'both', 'x', 'y'
                 """
                 matplotlib.pyplot.ioff()
                 ax.set_xlim([xmin, xmax])
                 ax.set_ylim([ymin, ymax])
                 printLabels(ax, xlabel, ylabel, title)
                 ######################################################
                 if (xmax - xmin) <= 1:
                     xtickspos = numpy.linspace(xmin, xmax, nxticks)
                     xtickspos = numpy.array([float("%.1f" % i) for i in xtickspos])
                     ax.set_xticks(xtickspos)
                 else:
                     xtickspos = numpy.arange(nxticks) * \
                         int((xmax - xmin) / (nxticks)) + int(xmin)
             #         xtickspos = numpy.arange(nxticks)*float(xmax-xmin)/float(nxticks) + int(xmin)
                     ax.set_xticks(xtickspos)
                 for tick in ax.get_xticklabels():
                     tick.set_visible(xtick_visible)
                 for tick in ax.xaxis.get_major_ticks():
                     tick.label.set_fontsize(ticksize)
                 ######################################################
                 for tick in ax.get_yticklabels():
                     tick.set_visible(ytick_visible)
                 for tick in ax.yaxis.get_major_ticks():
                     tick.label.set_fontsize(ticksize)
                 ax.plot(x, y, color=color)
                 iplot = ax.lines[-1]
                 ######################################################
                 if '0.' in matplotlib.__version__[0:2]:
                     print "The matplotlib version has to be updated to 1.1 or newer"
                     return iplot
                 if '1.0.' in matplotlib.__version__[0:4]:
                     print "The matplotlib version has to be updated to 1.1 or newer"
                     return iplot
                 if grid != None:
                     ax.grid(b=True, which='major', axis=grid)
                 matplotlib.pyplot.tight_layout()
                 matplotlib.pyplot.ion()
                 return iplot
             def set_linedata(ax, x, y, idline):
                 ax.lines[idline].set_data(x, y)
             def pline(iplot, x, y, xlabel='', ylabel='', title=''):
                 ax = iplot.axes
                 printLabels(ax, xlabel, ylabel, title)
                 set_linedata(ax, x, y, idline=0)
             def addpline(ax, x, y, color, linestyle, lw):
                 ax.plot(x, y, color=color, linestyle=linestyle, lw=lw)
             def createPcolor(ax, x, y, z, xmin, xmax, ymin, ymax, zmin, zmax,
                              xlabel='', ylabel='', title='', ticksize=9,
                              colormap='jet', cblabel='', cbsize="5%",
                              XAxisAsTime=False):
                 matplotlib.pyplot.ioff()
                 divider = make_axes_locatable(ax)
                 ax_cb = divider.new_horizontal(size=cbsize, pad=0.05)
                 fig = ax.get_figure()
                 fig.add_axes(ax_cb)
                 ax.set_xlim([xmin, xmax])
                 ax.set_ylim([ymin, ymax])
                 printLabels(ax, xlabel, ylabel, title)
                 z = numpy.ma.masked_invalid(z)
                 cmap = matplotlib.pyplot.get_cmap(colormap)
                 cmap.set_bad('black', 1.)
                 imesh = ax.pcolormesh(x, y, z.T, vmin=zmin, vmax=zmax, cmap=cmap)
                 cb = matplotlib.pyplot.colorbar(imesh, cax=ax_cb)
                 cb.set_label(cblabel)
             #    for tl in ax_cb.get_yticklabels():
             #        tl.set_visible(True)
                 for tick in ax.yaxis.get_major_ticks():
                     tick.label.set_fontsize(ticksize)
                 for tick in ax.xaxis.get_major_ticks():
                     tick.label.set_fontsize(ticksize)
                 for tick in cb.ax.get_yticklabels():
                     tick.set_fontsize(ticksize)
                 ax_cb.yaxis.tick_right()
                 if '0.' in matplotlib.__version__[0:2]:
                     print "The matplotlib version has to be updated to 1.1 or newer"
                     return imesh
                 if '1.0.' in matplotlib.__version__[0:4]:
                     print "The matplotlib version has to be updated to 1.1 or newer"
                     return imesh
                 matplotlib.pyplot.tight_layout()
                 if XAxisAsTime:
                     def func(x, pos): return ('%s') % (
                         datetime.datetime.utcfromtimestamp(x).strftime("%H:%M:%S"))
                     ax.xaxis.set_major_formatter(FuncFormatter(func))
                     ax.xaxis.set_major_locator(LinearLocator(7))
                 matplotlib.pyplot.ion()
                 return imesh
             def pcolor(imesh, z, xlabel='', ylabel='', title=''):
                 z = z.T
                 ax = imesh.axes
                 printLabels(ax, xlabel, ylabel, title)
                 imesh.set_array(z.ravel())
             def addpcolor(ax, x, y, z, zmin, zmax, xlabel='', ylabel='', title='', colormap='jet'):
                 printLabels(ax, xlabel, ylabel, title)
                 ax.pcolormesh(x, y, z.T, vmin=zmin, vmax=zmax,
                               cmap=matplotlib.pyplot.get_cmap(colormap))
             def addpcolorbuffer(ax, x, y, z, zmin, zmax, xlabel='', ylabel='', title='', colormap='jet'):
                 printLabels(ax, xlabel, ylabel, title)
                 ax.collections.remove(ax.collections[0])
                 z = numpy.ma.masked_invalid(z)
                 cmap = matplotlib.pyplot.get_cmap(colormap)
                 cmap.set_bad('black', 1.)
                 ax.pcolormesh(x, y, z.T, vmin=zmin, vmax=zmax, cmap=cmap)
             def createPmultiline(ax, x, y, xmin, xmax, ymin, ymax, xlabel='', ylabel='', title='', legendlabels=None,
                                  ticksize=9, xtick_visible=True, ytick_visible=True,
                                  nxticks=4, nyticks=10,
                                  grid=None):
                 """
                 Input:
                     grid    :    None, 'both', 'x', 'y'
                 """
                 matplotlib.pyplot.ioff()
                 lines = ax.plot(x.T, y)
                 leg = ax.legend(lines, legendlabels, loc='upper right')
                 leg.get_frame().set_alpha(0.5)
                 ax.set_xlim([xmin, xmax])
                 ax.set_ylim([ymin, ymax])
                 printLabels(ax, xlabel, ylabel, title)
                 xtickspos = numpy.arange(nxticks) * \
                     int((xmax - xmin) / (nxticks)) + int(xmin)
                 ax.set_xticks(xtickspos)
                 for tick in ax.get_xticklabels():
                     tick.set_visible(xtick_visible)
                 for tick in ax.xaxis.get_major_ticks():
                     tick.label.set_fontsize(ticksize)
                 for tick in ax.get_yticklabels():
                     tick.set_visible(ytick_visible)
                 for tick in ax.yaxis.get_major_ticks():
                     tick.label.set_fontsize(ticksize)
                 iplot = ax.lines[-1]
                 if '0.' in matplotlib.__version__[0:2]:
                     print "The matplotlib version has to be updated to 1.1 or newer"
                     return iplot
                 if '1.0.' in matplotlib.__version__[0:4]:
                     print "The matplotlib version has to be updated to 1.1 or newer"
                     return iplot
                 if grid != None:
                     ax.grid(b=True, which='major', axis=grid)
                 matplotlib.pyplot.tight_layout()
                 matplotlib.pyplot.ion()
                 return iplot
             def pmultiline(iplot, x, y, xlabel='', ylabel='', title=''):
                 ax = iplot.axes
                 printLabels(ax, xlabel, ylabel, title)
                 for i in range(len(ax.lines)):
                     line = ax.lines[i]
                     line.set_data(x[i, :], y)
             def createPmultilineYAxis(ax, x, y, xmin, xmax, ymin, ymax, xlabel='', ylabel='', title='', legendlabels=None,
                                       ticksize=9, xtick_visible=True, ytick_visible=True,
                                       nxticks=4, nyticks=10, marker='.', markersize=10, linestyle="None",
                                       grid=None, XAxisAsTime=False):
                 """
                 Input:
                     grid    :    None, 'both', 'x', 'y'
                 """
                 matplotlib.pyplot.ioff()
             #    lines = ax.plot(x, y.T, marker=marker,markersize=markersize,linestyle=linestyle)
                 lines = ax.plot(x, y.T)
             #     leg = ax.legend(lines, legendlabels, loc=2, bbox_to_anchor=(1.01, 1.00), numpoints=1, handlelength=1.5, \
             #                     handletextpad=0.5, borderpad=0.5, labelspacing=0.5, borderaxespad=0.)
                 leg = ax.legend(lines, legendlabels,
                                 loc='upper right', bbox_to_anchor=(1.16, 1), borderaxespad=0)
                 for label in leg.get_texts():
                     label.set_fontsize(9)
                 ax.set_xlim([xmin, xmax])
                 ax.set_ylim([ymin, ymax])
                 printLabels(ax, xlabel, ylabel, title)
             #    xtickspos = numpy.arange(nxticks)*int((xmax-xmin)/(nxticks)) + int(xmin)
             #    ax.set_xticks(xtickspos)
                 for tick in ax.get_xticklabels():
                     tick.set_visible(xtick_visible)
                 for tick in ax.xaxis.get_major_ticks():
                     tick.label.set_fontsize(ticksize)
                 for tick in ax.get_yticklabels():
                     tick.set_visible(ytick_visible)
                 for tick in ax.yaxis.get_major_ticks():
                     tick.label.set_fontsize(ticksize)
                 iplot = ax.lines[-1]
                 if '0.' in matplotlib.__version__[0:2]:
                     print "The matplotlib version has to be updated to 1.1 or newer"
                     return iplot
                 if '1.0.' in matplotlib.__version__[0:4]:
                     print "The matplotlib version has to be updated to 1.1 or newer"
                     return iplot
                 if grid != None:
                     ax.grid(b=True, which='major', axis=grid)
                 matplotlib.pyplot.tight_layout()
                 if XAxisAsTime:
                     def func(x, pos): return ('%s') % (
                         datetime.datetime.utcfromtimestamp(x).strftime("%H:%M:%S"))
                     ax.xaxis.set_major_formatter(FuncFormatter(func))
                     ax.xaxis.set_major_locator(LinearLocator(7))
                 matplotlib.pyplot.ion()
                 return iplot
             def pmultilineyaxis(iplot, x, y, xlabel='', ylabel='', title=''):
                 ax = iplot.axes
                 printLabels(ax, xlabel, ylabel, title)
                 for i in range(len(ax.lines)):
                     line = ax.lines[i]
                     line.set_data(x, y[i, :])
             def createPolar(ax, x, y,
                             xlabel='', ylabel='', title='', ticksize=9,
                             colormap='jet', cblabel='', cbsize="5%",
                             XAxisAsTime=False):
                 matplotlib.pyplot.ioff()
                 ax.plot(x, y, 'bo', markersize=5)
             #     ax.set_rmax(90)
                 ax.set_ylim(0, 90)
                 ax.set_yticks(numpy.arange(0, 90, 20))
             #     ax.text(0, -110, ylabel, rotation='vertical', va ='center', ha = 'center' ,size='11')
             #     ax.text(0, 50, ylabel, rotation='vertical', va ='center', ha = 'left' ,size='11')
             #     ax.text(100, 100, 'example', ha='left', va='center', rotation='vertical')
                 ax.yaxis.labelpad = 40
                 printLabels(ax, xlabel, ylabel, title)
                 iplot = ax.lines[-1]
                 if '0.' in matplotlib.__version__[0:2]:
                     print "The matplotlib version has to be updated to 1.1 or newer"
                     return iplot
                 if '1.0.' in matplotlib.__version__[0:4]:
                     print "The matplotlib version has to be updated to 1.1 or newer"
                     return iplot
             #     if grid != None:
             #         ax.grid(b=True, which='major', axis=grid)
                 matplotlib.pyplot.tight_layout()
                 matplotlib.pyplot.ion()
                 return iplot
             def polar(iplot, x, y, xlabel='', ylabel='', title=''):
                 ax = iplot.axes
             #     ax.text(0, -110, ylabel, rotation='vertical', va ='center', ha = 'center',size='11')
                 printLabels(ax, xlabel, ylabel, title)
                 set_linedata(ax, x, y, idline=0)
             def draw(fig):
                 if type(fig) == 'int':
                     raise ValueError, "Error drawing: Fig parameter should be a matplotlib figure object figure"
                 fig.canvas.draw()
             def pause(interval=0.000001):
                 matplotlib.pyplot.pause(interval)

schainpy/model/io/jroIO_base.py +16 -25

             '''
             Created on Jul 2, 2014
             @author: roj-idl71
             '''
             import os
             import sys
             import glob
             import time
             import numpy
             import fnmatch
             import inspect
             import time
             import datetime
             import traceback
             import zmq
             try:
                 from gevent import sleep
             except:
                 from time import sleep
             from schainpy.model.data.jroheaderIO import PROCFLAG, BasicHeader, SystemHeader, RadarControllerHeader, ProcessingHeader
             from schainpy.model.data.jroheaderIO import get_dtype_index, get_numpy_dtype, get_procflag_dtype, get_dtype_width
+            from schainpy.utils import log
             LOCALTIME = True
             def isNumber(cad):
                 """
                 Chequea si el conjunto de caracteres que componen un string puede ser convertidos a un numero.
                 Excepciones:
                     Si un determinado string no puede ser convertido a numero
                 Input:
                     str, string al cual se le analiza para determinar si convertible a un numero o no
                 Return:
                     True    :    si el string es uno numerico
                     False   :    no es un string numerico
                 """
                 try:
                     float(cad)
                     return True
                 except:
                     return False
             def isFileInEpoch(filename, startUTSeconds, endUTSeconds):
                 """
                 Esta funcion determina si un archivo de datos se encuentra o no dentro del rango de fecha especificado.
                 Inputs:
                     filename            :    nombre completo del archivo de datos en formato Jicamarca (.r)
                     startUTSeconds      :    fecha inicial del rango seleccionado. La fecha esta dada en
                                              segundos contados desde 01/01/1970.
                     endUTSeconds        :    fecha final del rango seleccionado. La fecha esta dada en
                                              segundos contados desde 01/01/1970.
                 Return:
                     Boolean    :    Retorna True si el archivo de datos contiene datos en el rango de
                                     fecha especificado, de lo contrario retorna False.
                 Excepciones:
                     Si el archivo no existe o no puede ser abierto
                     Si la cabecera no puede ser leida.
                 """
                 basicHeaderObj = BasicHeader(LOCALTIME)
                 try:
                     fp = open(filename, 'rb')
                 except IOError:
                     print "The file %s can't be opened" % (filename)
                     return 0
                 sts = basicHeaderObj.read(fp)
                 fp.close()
                 if not(sts):
                     print "Skipping the file %s because it has not a valid header" % (filename)
                     return 0
                 if not ((startUTSeconds <= basicHeaderObj.utc) and (endUTSeconds > basicHeaderObj.utc)):
                     return 0
                 return 1
             def isTimeInRange(thisTime, startTime, endTime):
                 if endTime >= startTime:
                     if (thisTime < startTime) or (thisTime > endTime):
                         return 0
                     return 1
                 else:
                     if (thisTime < startTime) and (thisTime > endTime):
                         return 0
                     return 1
             def isFileInTimeRange(filename, startDate, endDate, startTime, endTime):
                 """
                 Retorna 1 si el archivo de datos se encuentra dentro del rango de horas especificado.
                 Inputs:
                     filename            :    nombre completo del archivo de datos en formato Jicamarca (.r)
                     startDate          :    fecha inicial del rango seleccionado en formato datetime.date
                     endDate            :    fecha final del rango seleccionado en formato datetime.date
                     startTime          :    tiempo inicial del rango seleccionado en formato datetime.time
                     endTime            :    tiempo final del rango seleccionado en formato datetime.time
                 Return:
                     Boolean    :    Retorna True si el archivo de datos contiene datos en el rango de
                                     fecha especificado, de lo contrario retorna False.
                 Excepciones:
                     Si el archivo no existe o no puede ser abierto
                     Si la cabecera no puede ser leida.
                 """
                 try:
                     fp = open(filename, 'rb')
                 except IOError:
                     print "The file %s can't be opened" % (filename)
                     return None
                 firstBasicHeaderObj = BasicHeader(LOCALTIME)
                 systemHeaderObj = SystemHeader()
                 radarControllerHeaderObj = RadarControllerHeader()
                 processingHeaderObj = ProcessingHeader()
                 lastBasicHeaderObj = BasicHeader(LOCALTIME)
                 sts = firstBasicHeaderObj.read(fp)
                 if not(sts):
                     print "[Reading] Skipping the file %s because it has not a valid header" % (filename)
                     return None
                 if not systemHeaderObj.read(fp):
                     return None
                 if not radarControllerHeaderObj.read(fp):
                     return None
                 if not processingHeaderObj.read(fp):
                     return None
                 filesize = os.path.getsize(filename)
                 offset = processingHeaderObj.blockSize + 24  # header size
                 if filesize <= offset:
                     print "[Reading] %s: This file has not enough data" % filename
                     return None
                 fp.seek(-offset, 2)
                 sts = lastBasicHeaderObj.read(fp)
                 fp.close()
                 thisDatetime = lastBasicHeaderObj.datatime
                 thisTime_last_block = thisDatetime.time()
                 thisDatetime = firstBasicHeaderObj.datatime
                 thisDate = thisDatetime.date()
                 thisTime_first_block = thisDatetime.time()
                 # General case
                 #           o>>>>>>>>>>>>>><<<<<<<<<<<<<<o
                 #-----------o----------------------------o-----------
                 #       startTime                     endTime
                 if endTime >= startTime:
                     if (thisTime_last_block < startTime) or (thisTime_first_block > endTime):
                         return None
                     return thisDatetime
                 # If endTime < startTime then endTime belongs to the next day
                 #<<<<<<<<<<<o                            o>>>>>>>>>>>
                 #-----------o----------------------------o-----------
                 #        endTime                    startTime
                 if (thisDate == startDate) and (thisTime_last_block < startTime):
                     return None
                 if (thisDate == endDate) and (thisTime_first_block > endTime):
                     return None
                 if (thisTime_last_block < startTime) and (thisTime_first_block > endTime):
                     return None
                 return thisDatetime
             def isFolderInDateRange(folder, startDate=None, endDate=None):
                 """
                 Retorna 1 si el archivo de datos se encuentra dentro del rango de horas especificado.
                 Inputs:
                     folder            :    nombre completo del directorio.
                                            Su formato deberia ser "/path_root/?YYYYDDD"
                                             siendo:
                                                 YYYY    :    Anio         (ejemplo 2015)
                                                 DDD     :    Dia del anio (ejemplo 305)
                     startDate          :    fecha inicial del rango seleccionado en formato datetime.date
                     endDate            :    fecha final del rango seleccionado en formato datetime.date
                 Return:
                     Boolean    :    Retorna True si el archivo de datos contiene datos en el rango de
                                     fecha especificado, de lo contrario retorna False.
                 Excepciones:
                     Si el directorio no tiene el formato adecuado
                 """
                 basename = os.path.basename(folder)
                 if not isRadarFolder(basename):
                     print "The folder %s has not the rigth format" % folder
                     return 0
                 if startDate and endDate:
                     thisDate = getDateFromRadarFolder(basename)
                     if thisDate < startDate:
                         return 0
                     if thisDate > endDate:
                         return 0
                 return 1
             def isFileInDateRange(filename, startDate=None, endDate=None):
                 """
                 Retorna 1 si el archivo de datos se encuentra dentro del rango de horas especificado.
                 Inputs:
                     filename            :    nombre completo del archivo de datos en formato Jicamarca (.r)
                                            Su formato deberia ser "?YYYYDDDsss"
                                             siendo:
                                                 YYYY    :    Anio         (ejemplo 2015)
                                                 DDD     :    Dia del anio (ejemplo 305)
                                                 sss     :    set
                     startDate          :    fecha inicial del rango seleccionado en formato datetime.date
                     endDate            :    fecha final del rango seleccionado en formato datetime.date
                 Return:
                     Boolean    :    Retorna True si el archivo de datos contiene datos en el rango de
                                     fecha especificado, de lo contrario retorna False.
                 Excepciones:
                     Si el archivo no tiene el formato adecuado
                 """
                 basename = os.path.basename(filename)
                 if not isRadarFile(basename):
                     print "The filename %s has not the rigth format" % filename
                     return 0
                 if startDate and endDate:
                     thisDate = getDateFromRadarFile(basename)
                     if thisDate < startDate:
                         return 0
                     if thisDate > endDate:
                         return 0
                 return 1
             def getFileFromSet(path, ext, set):
                 validFilelist = []
                 fileList = os.listdir(path)
                 # 0 1234 567 89A BCDE
                 # H YYYY DDD SSS .ext
                 for thisFile in fileList:
                     try:
                         year = int(thisFile[1:5])
                         doy = int(thisFile[5:8])
                     except:
                         continue
                     if (os.path.splitext(thisFile)[-1].lower() != ext.lower()):
                         continue
                     validFilelist.append(thisFile)
                 myfile = fnmatch.filter(
                     validFilelist, '*%4.4d%3.3d%3.3d*' % (year, doy, set))
                 if len(myfile) != 0:
                     return myfile[0]
                 else:
                     filename = '*%4.4d%3.3d%3.3d%s' % (year, doy, set, ext.lower())
                     print 'the filename %s does not exist' % filename
                     print '...going to the last file: '
                 if validFilelist:
                     validFilelist = sorted(validFilelist, key=str.lower)
                     return validFilelist[-1]
                 return None
             def getlastFileFromPath(path, ext):
                 """
                 Depura el fileList dejando solo los que cumplan el formato de "PYYYYDDDSSS.ext"
                 al final de la depuracion devuelve el ultimo file de la lista que quedo.
                 Input:
                     fileList    :    lista conteniendo todos los files (sin path) que componen una determinada carpeta
                     ext         :    extension de los files contenidos en una carpeta
                 Return:
                     El ultimo file de una determinada carpeta, no se considera el path.
                 """
                 validFilelist = []
                 fileList = os.listdir(path)
                 # 0 1234 567 89A BCDE
                 # H YYYY DDD SSS .ext
                 for thisFile in fileList:
                     year = thisFile[1:5]
                     if not isNumber(year):
                         continue
                     doy = thisFile[5:8]
                     if not isNumber(doy):
                         continue
                     year = int(year)
                     doy = int(doy)
                     if (os.path.splitext(thisFile)[-1].lower() != ext.lower()):
                         continue
                     validFilelist.append(thisFile)
                 if validFilelist:
                     validFilelist = sorted(validFilelist, key=str.lower)
                     return validFilelist[-1]
                 return None
             def checkForRealPath(path, foldercounter, year, doy, set, ext):
                 """
                 Por ser Linux Case Sensitive entonces checkForRealPath encuentra el nombre correcto de un path,
                 Prueba por varias combinaciones de nombres entre mayusculas y minusculas para determinar
                 el path exacto de un determinado file.
                 Example    :
                     nombre correcto del file es  .../.../D2009307/P2009307367.ext
                     Entonces la funcion prueba con las siguientes combinaciones
                         .../.../y2009307367.ext
                         .../.../Y2009307367.ext
                         .../.../x2009307/y2009307367.ext
                         .../.../x2009307/Y2009307367.ext
                         .../.../X2009307/y2009307367.ext
                         .../.../X2009307/Y2009307367.ext
                     siendo para este caso, la ultima combinacion de letras, identica al file buscado
                 Return:
                     Si encuentra la cobinacion adecuada devuelve el path completo y el nombre del file
                     caso contrario devuelve None como path y el la ultima combinacion de nombre en mayusculas
                     para el filename
                 """
                 fullfilename = None
                 find_flag = False
                 filename = None
                 prefixDirList = [None, 'd', 'D']
                 if ext.lower() == ".r":  # voltage
                     prefixFileList = ['d', 'D']
                 elif ext.lower() == ".pdata":  # spectra
                     prefixFileList = ['p', 'P']
                 else:
                     return None, filename
                 # barrido por las combinaciones posibles
                 for prefixDir in prefixDirList:
                     thispath = path
                     if prefixDir != None:
                         # formo el nombre del directorio xYYYYDDD (x=d o x=D)
                         if foldercounter == 0:
                             thispath = os.path.join(path, "%s%04d%03d" %
                                                     (prefixDir, year, doy))
                         else:
                             thispath = os.path.join(path, "%s%04d%03d_%02d" % (
                                 prefixDir, year, doy, foldercounter))
                     for prefixFile in prefixFileList:  # barrido por las dos combinaciones posibles de "D"
                         # formo el nombre del file xYYYYDDDSSS.ext
                         filename = "%s%04d%03d%03d%s" % (prefixFile, year, doy, set, ext)
                         fullfilename = os.path.join(
                             thispath, filename)  # formo el path completo
                         if os.path.exists(fullfilename):  # verifico que exista
                             find_flag = True
                             break
                     if find_flag:
                         break
                 if not(find_flag):
                     return None, filename
                 return fullfilename, filename
             def isRadarFolder(folder):
                 try:
                     year = int(folder[1:5])
                     doy = int(folder[5:8])
                 except:
                     return 0
                 return 1
             def isRadarFile(file):
                 try:
                     year = int(file[1:5])
                     doy = int(file[5:8])
                     set = int(file[8:11])
                 except:
                     return 0
                 return 1
             def getDateFromRadarFile(file):
                 try:
                     year = int(file[1:5])
                     doy = int(file[5:8])
                     set = int(file[8:11])
                 except:
                     return None
                 thisDate = datetime.date(year, 1, 1) + datetime.timedelta(doy - 1)
                 return thisDate
             def getDateFromRadarFolder(folder):
                 try:
                     year = int(folder[1:5])
                     doy = int(folder[5:8])
                 except:
                     return None
                 thisDate = datetime.date(year, 1, 1) + datetime.timedelta(doy - 1)
                 return thisDate
             class JRODataIO:
                 c = 3E8
                 isConfig = False
                 basicHeaderObj = None
                 systemHeaderObj = None
                 radarControllerHeaderObj = None
                 processingHeaderObj = None
                 dtype = None
                 pathList = []
                 filenameList = []
                 filename = None
                 ext = None
                 flagIsNewFile = 1
                 flagDiscontinuousBlock = 0
                 flagIsNewBlock = 0
                 fp = None
                 firstHeaderSize = 0
                 basicHeaderSize = 24
                 versionFile = 1103
                 fileSize = None
             #     ippSeconds = None
                 fileSizeByHeader = None
                 fileIndex = None
                 profileIndex = None
                 blockIndex = None
                 nTotalBlocks = None
                 maxTimeStep = 30
                 lastUTTime = None
                 datablock = None
                 dataOut = None
                 blocksize = None
                 getByBlock = False
                 def __init__(self):
                     raise NotImplementedError
                 def run(self):
                     raise NotImplementedError
                 def getDtypeWidth(self):
                     dtype_index = get_dtype_index(self.dtype)
                     dtype_width = get_dtype_width(dtype_index)
                     return dtype_width
                 def getAllowedArgs(self):
                     if hasattr(self, '__attrs__'):
                         return self.__attrs__
                     else:
                         return inspect.getargspec(self.run).args
             class JRODataReader(JRODataIO):
                 online = 0
                 realtime = 0
                 nReadBlocks = 0
                 delay = 10  # number of seconds waiting a new file
                 nTries = 3  # quantity tries
                 nFiles = 3  # number of files for searching
                 path = None
                 foldercounter = 0
                 flagNoMoreFiles = 0
                 datetimeList = []
                 __isFirstTimeOnline = 1
                 __printInfo = True
                 profileIndex = None
                 nTxs = 1
                 txIndex = None
                 # Added--------------------
                 selBlocksize = None
                 selBlocktime = None
                 def __init__(self):
                     """
                     This class is used to find data files
                     Example:
                         reader = JRODataReader()
                         fileList = reader.findDataFiles()
                     """
                     pass
                 def createObjByDefault(self):
                     """
                     """
                     raise NotImplementedError
                 def getBlockDimension(self):
                     raise NotImplementedError
                 def searchFilesOffLine(self,
                                        path,
                                        startDate=None,
                                        endDate=None,
                                        startTime=datetime.time(0, 0, 0),
                                        endTime=datetime.time(23, 59, 59),
                                        set=None,
                                        expLabel='',
                                        ext='.r',
                                        cursor=None,
                                        skip=None,
                                        walk=True):
                     self.filenameList = []
                     self.datetimeList = []
                     pathList = []
                     dateList, pathList = self.findDatafiles(
                         path, startDate, endDate, expLabel, ext, walk, include_path=True)
                     if dateList == []:
                         return [], []
                     if len(dateList) > 1:
                         print "[Reading] Data found for date range [%s - %s]: total days = %d" % (startDate, endDate, len(dateList))
                     else:
                         print "[Reading] Data found for date range [%s - %s]: date = %s" % (startDate, endDate, dateList[0])
                     filenameList = []
                     datetimeList = []
                     for thisPath in pathList:
                         fileList = glob.glob1(thisPath, "*%s" % ext)
                         fileList.sort()
-                        skippedFileList = []
+                        for file in fileList:
-                        if cursor is not None and skip is not None:
-                            if skip == 0:
-                                skippedFileList = []
-                            else:
-                                skippedFileList = fileList[cursor *
-                                                           skip: cursor * skip + skip]
-                        else:
-                            skippedFileList = fileList
-                        for file in skippedFileList:
                             filename = os.path.join(thisPath, file)
                             if not isFileInDateRange(filename, startDate, endDate):
                                 continue
                             thisDatetime = isFileInTimeRange(
                                 filename, startDate, endDate, startTime, endTime)
                             if not(thisDatetime):
                                 continue
                             filenameList.append(filename)
                             datetimeList.append(thisDatetime)
+                    if cursor is not None and skip is not None:
+                        filenameList = filenameList[cursor * skip:cursor * skip + skip]
+                        datetimeList = datetimeList[cursor * skip:cursor * skip + skip]
                     if not(filenameList):
                         print "[Reading] Time range selected invalid [%s - %s]: No *%s files in %s)" % (startTime, endTime, ext, path)
                         return [], []
                     print "[Reading] %d file(s) was(were) found in time range: %s - %s" % (len(filenameList), startTime, endTime)
-                    print
                     # for i in range(len(filenameList)):
                     #     print "[Reading] %s -> [%s]" %(filenameList[i], datetimeList[i].ctime())
                     self.filenameList = filenameList
                     self.datetimeList = datetimeList
                     return pathList, filenameList
                 def __searchFilesOnLine(self, path, expLabel="", ext=None, walk=True, set=None):
                     """
                     Busca el ultimo archivo de la ultima carpeta (determinada o no por startDateTime) y
                     devuelve el archivo encontrado ademas de otros datos.
                     Input:
                         path           :    carpeta donde estan contenidos los files que contiene data
                         expLabel        :     Nombre del subexperimento (subfolder)
                         ext              :    extension de los files
                         walk        :    Si es habilitado no realiza busquedas dentro de los ubdirectorios (doypath)
                     Return:
                         directory   :    eL directorio donde esta el file encontrado
                         filename    :    el ultimo file de una determinada carpeta
                         year        :    el anho
                         doy         :    el numero de dia del anho
                         set         :    el set del archivo
                     """
                     if not os.path.isdir(path):
                         return None, None, None, None, None, None
                     dirList = []
                     if not walk:
                         fullpath = path
                         foldercounter = 0
                     else:
                         # Filtra solo los directorios
                         for thisPath in os.listdir(path):
                             if not os.path.isdir(os.path.join(path, thisPath)):
                                 continue
                             if not isRadarFolder(thisPath):
                                 continue
                             dirList.append(thisPath)
                         if not(dirList):
                             return None, None, None, None, None, None
                         dirList = sorted(dirList, key=str.lower)
                         doypath = dirList[-1]
                         foldercounter = int(doypath.split('_')[1]) if len(
                             doypath.split('_')) > 1 else 0
                         fullpath = os.path.join(path, doypath, expLabel)
                     print "[Reading] %s folder was found: " % (fullpath)
                     if set == None:
                         filename = getlastFileFromPath(fullpath, ext)
                     else:
                         filename = getFileFromSet(fullpath, ext, set)
                     if not(filename):
                         return None, None, None, None, None, None
                     print "[Reading] %s file was found" % (filename)
                     if not(self.__verifyFile(os.path.join(fullpath, filename))):
                         return None, None, None, None, None, None
                     year = int(filename[1:5])
                     doy = int(filename[5:8])
                     set = int(filename[8:11])
                     return fullpath, foldercounter, filename, year, doy, set
                 def __setNextFileOffline(self):
                     idFile = self.fileIndex
                     while (True):
                         idFile += 1
                         if not(idFile < len(self.filenameList)):
                             self.flagNoMoreFiles = 1
             #                 print "[Reading] No more Files"
                             return 0
                         filename = self.filenameList[idFile]
                         if not(self.__verifyFile(filename)):
                             continue
                         fileSize = os.path.getsize(filename)
                         fp = open(filename, 'rb')
                         break
                     self.flagIsNewFile = 1
                     self.fileIndex = idFile
                     self.filename = filename
                     self.fileSize = fileSize
                     self.fp = fp
             #         print "[Reading] Setting the file: %s"%self.filename
                     return 1
                 def __setNextFileOnline(self):
                     """
                     Busca el siguiente file que tenga suficiente data para ser leida, dentro de un folder especifico, si
                     no encuentra un file valido espera un tiempo determinado y luego busca en los posibles n files
                     siguientes.
                     Affected:
                         self.flagIsNewFile
                         self.filename
                         self.fileSize
                         self.fp
                         self.set
                         self.flagNoMoreFiles
                     Return:
 : si luego de una busqueda del siguiente file valido este no pudo ser encontrado
 : si el file fue abierto con exito y esta listo a ser leido
                     Excepciones:
                         Si un determinado file no puede ser abierto
                     """
                     nFiles = 0
                     fileOk_flag = False
                     firstTime_flag = True
                     self.set += 1
                     if self.set > 999:
                         self.set = 0
                         self.foldercounter += 1
                     # busca el 1er file disponible
                     fullfilename, filename = checkForRealPath(
                         self.path, self.foldercounter, self.year, self.doy, self.set, self.ext)
                     if fullfilename:
                         if self.__verifyFile(fullfilename, False):
                             fileOk_flag = True
                     # si no encuentra un file entonces espera y vuelve a buscar
                     if not(fileOk_flag):
                         # busco en los siguientes self.nFiles+1 files posibles
                         for nFiles in range(self.nFiles + 1):
                             if firstTime_flag:  # si es la 1era vez entonces hace el for self.nTries veces
                                 tries = self.nTries
                             else:
                                 tries = 1  # si no es la 1era vez entonces solo lo hace una vez
                             for nTries in range(tries):
                                 if firstTime_flag:
                                     print "\t[Reading] Waiting %0.2f sec for the next file: \"%s\" , try %03d ..." % (self.delay, filename, nTries + 1)
                                     sleep(self.delay)
                                 else:
                                     print "\t[Reading] Searching the next \"%s%04d%03d%03d%s\" file ..." % (self.optchar, self.year, self.doy, self.set, self.ext)
                                 fullfilename, filename = checkForRealPath(
                                     self.path, self.foldercounter, self.year, self.doy, self.set, self.ext)
                                 if fullfilename:
                                     if self.__verifyFile(fullfilename):
                                         fileOk_flag = True
                                         break
                             if fileOk_flag:
                                 break
                             firstTime_flag = False
                             print "\t[Reading] Skipping the file \"%s\" due to this file doesn't exist" % filename
                             self.set += 1
                             # si no encuentro el file buscado cambio de carpeta y busco en la siguiente carpeta
                             if nFiles == (self.nFiles - 1):
                                 self.set = 0
                                 self.doy += 1
                                 self.foldercounter = 0
                     if fileOk_flag:
                         self.fileSize = os.path.getsize(fullfilename)
                         self.filename = fullfilename
                         self.flagIsNewFile = 1
                         if self.fp != None:
                             self.fp.close()
                         self.fp = open(fullfilename, 'rb')
                         self.flagNoMoreFiles = 0
             #             print '[Reading] Setting the file: %s' % fullfilename
                     else:
                         self.fileSize = 0
                         self.filename = None
                         self.flagIsNewFile = 0
                         self.fp = None
                         self.flagNoMoreFiles = 1
             #             print '[Reading] No more files to read'
                     return fileOk_flag
                 def setNextFile(self):
                     if self.fp != None:
                         self.fp.close()
                     if self.online:
                         newFile = self.__setNextFileOnline()
                     else:
                         newFile = self.__setNextFileOffline()
                     if not(newFile):
                         print '[Reading] No more files to read'
                         return 0
                     if self.verbose:
                         print '[Reading] Setting the file: %s' % self.filename
                     self.__readFirstHeader()
                     self.nReadBlocks = 0
                     return 1
                 def __waitNewBlock(self):
                     """
                     Return 1 si se encontro un nuevo bloque de datos, 0 de otra forma.
                     Si el modo de lectura es OffLine siempre retorn 0
                     """
                     if not self.online:
                         return 0
                     if (self.nReadBlocks >= self.processingHeaderObj.dataBlocksPerFile):
                         return 0
                     currentPointer = self.fp.tell()
                     neededSize = self.processingHeaderObj.blockSize + self.basicHeaderSize
                     for nTries in range(self.nTries):
                         self.fp.close()
                         self.fp = open(self.filename, 'rb')
                         self.fp.seek(currentPointer)
                         self.fileSize = os.path.getsize(self.filename)
                         currentSize = self.fileSize - currentPointer
                         if (currentSize >= neededSize):
                             self.basicHeaderObj.read(self.fp)
                             return 1
                         if self.fileSize == self.fileSizeByHeader:
                             #                self.flagEoF = True
                             return 0
                         print "[Reading] Waiting %0.2f seconds for the next block, try %03d ..." % (self.delay, nTries + 1)
                         sleep(self.delay)
                     return 0
                 def waitDataBlock(self, pointer_location):
                     currentPointer = pointer_location
                     neededSize = self.processingHeaderObj.blockSize  # + self.basicHeaderSize
                     for nTries in range(self.nTries):
                         self.fp.close()
                         self.fp = open(self.filename, 'rb')
                         self.fp.seek(currentPointer)
                         self.fileSize = os.path.getsize(self.filename)
                         currentSize = self.fileSize - currentPointer
                         if (currentSize >= neededSize):
                             return 1
                         print "[Reading] Waiting %0.2f seconds for the next block, try %03d ..." % (self.delay, nTries + 1)
                         sleep(self.delay)
                     return 0
                 def __jumpToLastBlock(self):
                     if not(self.__isFirstTimeOnline):
                         return
                     csize = self.fileSize - self.fp.tell()
                     blocksize = self.processingHeaderObj.blockSize
                     # salta el primer bloque de datos
                     if csize > self.processingHeaderObj.blockSize:
                         self.fp.seek(self.fp.tell() + blocksize)
                     else:
                         return
                     csize = self.fileSize - self.fp.tell()
                     neededsize = self.processingHeaderObj.blockSize + self.basicHeaderSize
                     while True:
                         if self.fp.tell() < self.fileSize:
                             self.fp.seek(self.fp.tell() + neededsize)
                         else:
                             self.fp.seek(self.fp.tell() - neededsize)
                             break
             #        csize = self.fileSize - self.fp.tell()
             #        neededsize = self.processingHeaderObj.blockSize + self.basicHeaderSize
             #        factor = int(csize/neededsize)
             #        if factor > 0:
             #            self.fp.seek(self.fp.tell() + factor*neededsize)
                     self.flagIsNewFile = 0
                     self.__isFirstTimeOnline = 0
                 def __setNewBlock(self):
                     # if self.server is None:
                     if self.fp == None:
                         return 0
             #         if self.online:
             #             self.__jumpToLastBlock()
                     if self.flagIsNewFile:
                         self.lastUTTime = self.basicHeaderObj.utc
                         return 1
                     if self.realtime:
                         self.flagDiscontinuousBlock = 1
                         if not(self.setNextFile()):
                             return 0
                         else:
                             return 1
                     # if self.server is None:
                     currentSize = self.fileSize - self.fp.tell()
                     neededSize = self.processingHeaderObj.blockSize + self.basicHeaderSize
                     if (currentSize >= neededSize):
                         self.basicHeaderObj.read(self.fp)
                         self.lastUTTime = self.basicHeaderObj.utc
                         return 1
                     # else:
                     #     self.basicHeaderObj.read(self.zHeader)
                     #     self.lastUTTime = self.basicHeaderObj.utc
                     #     return 1
                     if self.__waitNewBlock():
                         self.lastUTTime = self.basicHeaderObj.utc
                         return 1
                     # if self.server is None:
                     if not(self.setNextFile()):
                         return 0
                     deltaTime = self.basicHeaderObj.utc - self.lastUTTime
                     self.lastUTTime = self.basicHeaderObj.utc
                     self.flagDiscontinuousBlock = 0
                     if deltaTime > self.maxTimeStep:
                         self.flagDiscontinuousBlock = 1
                     return 1
                 def readNextBlock(self):
                     # Skip block out of startTime and endTime
                     while True:
                         if not(self.__setNewBlock()):
                             return 0
                         if not(self.readBlock()):
                             return 0
                         self.getBasicHeader()
                         if (self.dataOut.datatime < datetime.datetime.combine(self.startDate, self.startTime)) or (self.dataOut.datatime > datetime.datetime.combine(self.endDate, self.endTime)):
                             print "[Reading] Block No. %d/%d -> %s [Skipping]" % (self.nReadBlocks,
                                                                                   self.processingHeaderObj.dataBlocksPerFile,
                                                                                   self.dataOut.datatime.ctime())
                             continue
                         break
                     if self.verbose:
                         print "[Reading] Block No. %d/%d -> %s" % (self.nReadBlocks,
                                                                    self.processingHeaderObj.dataBlocksPerFile,
                                                                    self.dataOut.datatime.ctime())
                     return 1
                 def __readFirstHeader(self):
                     self.basicHeaderObj.read(self.fp)
                     self.systemHeaderObj.read(self.fp)
                     self.radarControllerHeaderObj.read(self.fp)
                     self.processingHeaderObj.read(self.fp)
                     self.firstHeaderSize = self.basicHeaderObj.size
                     datatype = int(numpy.log2((self.processingHeaderObj.processFlags &
                                                PROCFLAG.DATATYPE_MASK)) - numpy.log2(PROCFLAG.DATATYPE_CHAR))
                     if datatype == 0:
                         datatype_str = numpy.dtype([('real', '<i1'), ('imag', '<i1')])
                     elif datatype == 1:
                         datatype_str = numpy.dtype([('real', '<i2'), ('imag', '<i2')])
                     elif datatype == 2:
                         datatype_str = numpy.dtype([('real', '<i4'), ('imag', '<i4')])
                     elif datatype == 3:
                         datatype_str = numpy.dtype([('real', '<i8'), ('imag', '<i8')])
                     elif datatype == 4:
                         datatype_str = numpy.dtype([('real', '<f4'), ('imag', '<f4')])
                     elif datatype == 5:
                         datatype_str = numpy.dtype([('real', '<f8'), ('imag', '<f8')])
                     else:
                         raise ValueError, 'Data type was not defined'
                     self.dtype = datatype_str
                     #self.ippSeconds = 2 * 1000 * self.radarControllerHeaderObj.ipp / self.c
                     self.fileSizeByHeader = self.processingHeaderObj.dataBlocksPerFile * self.processingHeaderObj.blockSize + \
                         self.firstHeaderSize + self.basicHeaderSize * \
                         (self.processingHeaderObj.dataBlocksPerFile - 1)
                     #        self.dataOut.channelList = numpy.arange(self.systemHeaderObj.numChannels)
                     #        self.dataOut.channelIndexList = numpy.arange(self.systemHeaderObj.numChannels)
                     self.getBlockDimension()
                 def __verifyFile(self, filename, msgFlag=True):
                     msg = None
                     try:
                         fp = open(filename, 'rb')
                     except IOError:
                         if msgFlag:
                             print "[Reading] File %s can't be opened" % (filename)
                         return False
                     currentPosition = fp.tell()
                     neededSize = self.processingHeaderObj.blockSize + self.firstHeaderSize
                     if neededSize == 0:
                         basicHeaderObj = BasicHeader(LOCALTIME)
                         systemHeaderObj = SystemHeader()
                         radarControllerHeaderObj = RadarControllerHeader()
                         processingHeaderObj = ProcessingHeader()
                         if not(basicHeaderObj.read(fp)):
                             fp.close()
                             return False
                         if not(systemHeaderObj.read(fp)):
                             fp.close()
                             return False
                         if not(radarControllerHeaderObj.read(fp)):
                             fp.close()
                             return False
                         if not(processingHeaderObj.read(fp)):
                             fp.close()
                             return False
                         neededSize = processingHeaderObj.blockSize + basicHeaderObj.size
                     else:
                         msg = "[Reading] Skipping the file %s due to it hasn't enough data" % filename
                     fp.close()
                     fileSize = os.path.getsize(filename)
                     currentSize = fileSize - currentPosition
                     if currentSize < neededSize:
                         if msgFlag and (msg != None):
                             print msg
                         return False
                     return True
                 def findDatafiles(self, path, startDate=None, endDate=None, expLabel='', ext='.r', walk=True, include_path=False):
                     path_empty = True
                     dateList = []
                     pathList = []
                     multi_path = path.split(',')
                     if not walk:
                         for single_path in multi_path:
                             if not os.path.isdir(single_path):
                                 continue
                             fileList = glob.glob1(single_path, "*" + ext)
                             if not fileList:
                                 continue
                             path_empty = False
                             fileList.sort()
                             for thisFile in fileList:
                                 if not os.path.isfile(os.path.join(single_path, thisFile)):
                                     continue
                                 if not isRadarFile(thisFile):
                                     continue
                                 if not isFileInDateRange(thisFile, startDate, endDate):
                                     continue
                                 thisDate = getDateFromRadarFile(thisFile)
                                 if thisDate in dateList:
                                     continue
                                 dateList.append(thisDate)
                                 pathList.append(single_path)
                     else:
                         for single_path in multi_path:
                             if not os.path.isdir(single_path):
                                 continue
                             dirList = []
                             for thisPath in os.listdir(single_path):
                                 if not os.path.isdir(os.path.join(single_path, thisPath)):
                                     continue
                                 if not isRadarFolder(thisPath):
                                     continue
                                 if not isFolderInDateRange(thisPath, startDate, endDate):
                                     continue
                                 dirList.append(thisPath)
                             if not dirList:
                                 continue
                             dirList.sort()
                             for thisDir in dirList:
                                 datapath = os.path.join(single_path, thisDir, expLabel)
                                 fileList = glob.glob1(datapath, "*" + ext)
                                 if not fileList:
                                     continue
                                 path_empty = False
                                 thisDate = getDateFromRadarFolder(thisDir)
                                 pathList.append(datapath)
                                 dateList.append(thisDate)
                     dateList.sort()
                     if walk:
                         pattern_path = os.path.join(multi_path[0], "[dYYYYDDD]", expLabel)
                     else:
                         pattern_path = multi_path[0]
                     if path_empty:
                         print "[Reading] No *%s files in %s for %s to %s" % (ext, pattern_path, startDate, endDate)
                     else:
                         if not dateList:
                             print "[Reading] Date range selected invalid [%s - %s]: No *%s files in %s)" % (startDate, endDate, ext, path)
                     if include_path:
                         return dateList, pathList
                     return dateList
                 def setup(self,
                           path=None,
                           startDate=None,
                           endDate=None,
                           startTime=datetime.time(0, 0, 0),
                           endTime=datetime.time(23, 59, 59),
                           set=None,
                           expLabel="",
                           ext=None,
                           online=False,
                           delay=60,
                           walk=True,
                           getblock=False,
                           nTxs=1,
                           realtime=False,
                           blocksize=None,
                           blocktime=None,
                           skip=None,
                           cursor=None,
                           warnings=True,
                           verbose=True,
                           server=None,
                           format=None,
                           oneDDict=None,
                           twoDDict=None,
                           ind2DList=None):
                     if server is not None:
                         if 'tcp://' in server:
                             address = server
                         else:
                             address = 'ipc:///tmp/%s' % server
                         self.server = address
                         self.context = zmq.Context()
                         self.receiver = self.context.socket(zmq.PULL)
                         self.receiver.connect(self.server)
                         time.sleep(0.5)
                         print '[Starting] ReceiverData from {}'.format(self.server)
                     else:
                         self.server = None
                         if path == None:
                             raise ValueError, "[Reading] The path is not valid"
                         if ext == None:
                             ext = self.ext
                         if online:
                             print "[Reading] Searching files in online mode..."
                             for nTries in range(self.nTries):
                                 fullpath, foldercounter, file, year, doy, set = self.__searchFilesOnLine(
                                     path=path, expLabel=expLabel, ext=ext, walk=walk, set=set)
                                 if fullpath:
                                     break
                                 print '[Reading] Waiting %0.2f sec for an valid file in %s: try %02d ...' % (self.delay, path, nTries + 1)
                                 sleep(self.delay)
                             if not(fullpath):
                                 print "[Reading] There 'isn't any valid file in %s" % path
                                 return
                             self.year = year
                             self.doy = doy
                             self.set = set - 1
                             self.path = path
                             self.foldercounter = foldercounter
                             last_set = None
                         else:
                             print "[Reading] Searching files in offline mode ..."
                             pathList, filenameList = self.searchFilesOffLine(path, startDate=startDate, endDate=endDate,
                                                                              startTime=startTime, endTime=endTime,
                                                                              set=set, expLabel=expLabel, ext=ext,
                                                                              walk=walk, cursor=cursor,
                                                                              skip=skip)
                             if not(pathList):
                                 self.fileIndex = -1
                                 self.pathList = []
                                 self.filenameList = []
                                 return
                             self.fileIndex = -1
                             self.pathList = pathList
                             self.filenameList = filenameList
                             file_name = os.path.basename(filenameList[-1])
                             basename, ext = os.path.splitext(file_name)
                             last_set = int(basename[-3:])
                         self.online = online
                         self.realtime = realtime
                         self.delay = delay
                         ext = ext.lower()
                         self.ext = ext
                         self.getByBlock = getblock
                         self.nTxs = nTxs
                         self.startTime = startTime
                         self.endTime = endTime
                         self.endDate = endDate
                         self.startDate = startDate
                         # Added-----------------
                         self.selBlocksize = blocksize
                         self.selBlocktime = blocktime
                         # Verbose-----------
                         self.verbose = verbose
                         self.warnings = warnings
                         if not(self.setNextFile()):
                             if (startDate != None) and (endDate != None):
                                 print "[Reading] No files in range: %s - %s" % (datetime.datetime.combine(startDate, startTime).ctime(), datetime.datetime.combine(endDate, endTime).ctime())
                             elif startDate != None:
                                 print "[Reading] No files in range: %s" % (datetime.datetime.combine(startDate, startTime).ctime())
                             else:
                                 print "[Reading] No files"
                                 self.fileIndex = -1
                                 self.pathList = []
                                 self.filenameList = []
                                 return
                 #         self.getBasicHeader()
                         if last_set != None:
                             self.dataOut.last_block = last_set * \
                                 self.processingHeaderObj.dataBlocksPerFile + self.basicHeaderObj.dataBlock
                     return
                 def getBasicHeader(self):
                     self.dataOut.utctime = self.basicHeaderObj.utc + self.basicHeaderObj.miliSecond / \
 . + self.profileIndex * self.radarControllerHeaderObj.ippSeconds
                     self.dataOut.flagDiscontinuousBlock = self.flagDiscontinuousBlock
                     self.dataOut.timeZone = self.basicHeaderObj.timeZone
                     self.dataOut.dstFlag = self.basicHeaderObj.dstFlag
                     self.dataOut.errorCount = self.basicHeaderObj.errorCount
                     self.dataOut.useLocalTime = self.basicHeaderObj.useLocalTime
                     self.dataOut.ippSeconds = self.radarControllerHeaderObj.ippSeconds / self.nTxs
             #         self.dataOut.nProfiles = self.processingHeaderObj.profilesPerBlock*self.nTxs
                 def getFirstHeader(self):
                     raise NotImplementedError
                 def getData(self):
                     raise NotImplementedError
                 def hasNotDataInBuffer(self):
                     raise NotImplementedError
                 def readBlock(self):
                     raise NotImplementedError
                 def isEndProcess(self):
                     return self.flagNoMoreFiles
                 def printReadBlocks(self):
                     print "[Reading] Number of read blocks per file %04d" % self.nReadBlocks
                 def printTotalBlocks(self):
                     print "[Reading] Number of read blocks %04d" % self.nTotalBlocks
                 def printNumberOfBlock(self):
                     'SPAM!'
             #         if self.flagIsNewBlock:
             #             print "[Reading] Block No. %d/%d -> %s" %(self.nReadBlocks,
             #                                                       self.processingHeaderObj.dataBlocksPerFile,
             #                                                       self.dataOut.datatime.ctime())
                 def printInfo(self):
                     if self.__printInfo == False:
                         return
                     self.basicHeaderObj.printInfo()
                     self.systemHeaderObj.printInfo()
                     self.radarControllerHeaderObj.printInfo()
                     self.processingHeaderObj.printInfo()
                     self.__printInfo = False
                 def run(self,
                         path=None,
                         startDate=None,
                         endDate=None,
                         startTime=datetime.time(0, 0, 0),
                         endTime=datetime.time(23, 59, 59),
                         set=None,
                         expLabel="",
                         ext=None,
                         online=False,
                         delay=60,
                         walk=True,
                         getblock=False,
                         nTxs=1,
                         realtime=False,
                         blocksize=None,
                         blocktime=None,
                         skip=None,
                         cursor=None,
                         warnings=True,
                         server=None,
                         verbose=True,
                         format=None,
                         oneDDict=None,
                         twoDDict=None,
                         ind2DList=None, **kwargs):
                     if not(self.isConfig):
                         self.setup(path=path,
                                    startDate=startDate,
                                    endDate=endDate,
                                    startTime=startTime,
                                    endTime=endTime,
                                    set=set,
                                    expLabel=expLabel,
                                    ext=ext,
                                    online=online,
                                    delay=delay,
                                    walk=walk,
                                    getblock=getblock,
                                    nTxs=nTxs,
                                    realtime=realtime,
                                    blocksize=blocksize,
                                    blocktime=blocktime,
                                    skip=skip,
                                    cursor=cursor,
                                    warnings=warnings,
                                    server=server,
                                    verbose=verbose,
                                    format=format,
                                    oneDDict=oneDDict,
                                    twoDDict=twoDDict,
                                    ind2DList=ind2DList)
                         self.isConfig = True
                     if server is None:
                         self.getData()
                     else:
                         self.getFromServer()
             class JRODataWriter(JRODataIO):
                 """
                 Esta clase permite escribir datos a archivos procesados (.r o ,pdata). La escritura
                 de los datos siempre se realiza por bloques.
                 """
                 blockIndex = 0
                 path = None
                 setFile = None
                 profilesPerBlock = None
                 blocksPerFile = None
                 nWriteBlocks = 0
                 fileDate = None
                 def __init__(self, dataOut=None):
                     raise NotImplementedError
                 def hasAllDataInBuffer(self):
                     raise NotImplementedError
                 def setBlockDimension(self):
                     raise NotImplementedError
                 def writeBlock(self):
                     raise NotImplementedError
                 def putData(self):
                     raise NotImplementedError
                 def getProcessFlags(self):
                     processFlags = 0
                     dtype_index = get_dtype_index(self.dtype)
                     procflag_dtype = get_procflag_dtype(dtype_index)
                     processFlags += procflag_dtype
                     if self.dataOut.flagDecodeData:
                         processFlags += PROCFLAG.DECODE_DATA
                     if self.dataOut.flagDeflipData:
                         processFlags += PROCFLAG.DEFLIP_DATA
                     if self.dataOut.code is not None:
                         processFlags += PROCFLAG.DEFINE_PROCESS_CODE
                     if self.dataOut.nCohInt > 1:
                         processFlags += PROCFLAG.COHERENT_INTEGRATION
                     if self.dataOut.type == "Spectra":
                         if self.dataOut.nIncohInt > 1:
                             processFlags += PROCFLAG.INCOHERENT_INTEGRATION
                         if self.dataOut.data_dc is not None:
                             processFlags += PROCFLAG.SAVE_CHANNELS_DC
                         if self.dataOut.flagShiftFFT:
                             processFlags += PROCFLAG.SHIFT_FFT_DATA
                     return processFlags
                 def setBasicHeader(self):
                     self.basicHeaderObj.size = self.basicHeaderSize  # bytes
                     self.basicHeaderObj.version = self.versionFile
                     self.basicHeaderObj.dataBlock = self.nTotalBlocks
                     utc = numpy.floor(self.dataOut.utctime)
                     milisecond = (self.dataOut.utctime - utc) * 1000.0
                     self.basicHeaderObj.utc = utc
                     self.basicHeaderObj.miliSecond = milisecond
                     self.basicHeaderObj.timeZone = self.dataOut.timeZone
                     self.basicHeaderObj.dstFlag = self.dataOut.dstFlag
                     self.basicHeaderObj.errorCount = self.dataOut.errorCount
                 def setFirstHeader(self):
                     """
                     Obtiene una copia del First Header
                     Affected:
                         self.basicHeaderObj
                         self.systemHeaderObj
                         self.radarControllerHeaderObj
                         self.processingHeaderObj self.
                     Return:
                         None
                     """
                     raise NotImplementedError
                 def __writeFirstHeader(self):
                     """
                     Escribe el primer header del file es decir el Basic header y el Long header (SystemHeader, RadarControllerHeader, ProcessingHeader)
                     Affected:
                         __dataType
                     Return:
                         None
                     """
             #        CALCULAR PARAMETROS
                     sizeLongHeader = self.systemHeaderObj.size + \
                         self.radarControllerHeaderObj.size + self.processingHeaderObj.size
                     self.basicHeaderObj.size = self.basicHeaderSize + sizeLongHeader
                     self.basicHeaderObj.write(self.fp)
                     self.systemHeaderObj.write(self.fp)
                     self.radarControllerHeaderObj.write(self.fp)
                     self.processingHeaderObj.write(self.fp)
                 def __setNewBlock(self):
                     """
                     Si es un nuevo file escribe el First Header caso contrario escribe solo el Basic Header
                     Return:
 :    si no pudo escribir nada
 :    Si escribio el Basic el First Header
                     """
                     if self.fp == None:
                         self.setNextFile()
                     if self.flagIsNewFile:
                         return 1
                     if self.blockIndex < self.processingHeaderObj.dataBlocksPerFile:
                         self.basicHeaderObj.write(self.fp)
                         return 1
                     if not(self.setNextFile()):
                         return 0
                     return 1
                 def writeNextBlock(self):
                     """
                     Selecciona el bloque siguiente de datos y los escribe en un file
                     Return:
 :    Si no hizo pudo escribir el bloque de datos
 :    Si no pudo escribir el bloque de datos
                     """
                     if not(self.__setNewBlock()):
                         return 0
                     self.writeBlock()
                     print "[Writing] Block No. %d/%d" % (self.blockIndex,
                                                          self.processingHeaderObj.dataBlocksPerFile)
                     return 1
                 def setNextFile(self):
                     """
                     Determina el siguiente file que sera escrito
                     Affected:
                         self.filename
                         self.subfolder
                         self.fp
                         self.setFile
                         self.flagIsNewFile
                     Return:
 :    Si el archivo no puede ser escrito
 :    Si el archivo esta listo para ser escrito
                     """
                     ext = self.ext
                     path = self.path
                     if self.fp != None:
                         self.fp.close()
                     timeTuple = time.localtime(self.dataOut.utctime)
                     subfolder = 'd%4.4d%3.3d' % (timeTuple.tm_year, timeTuple.tm_yday)
                     fullpath = os.path.join(path, subfolder)
                     setFile = self.setFile
                     if not(os.path.exists(fullpath)):
                         os.mkdir(fullpath)
                         setFile = -1  # inicializo mi contador de seteo
                     else:
                         filesList = os.listdir(fullpath)
                         if len(filesList) > 0:
                             filesList = sorted(filesList, key=str.lower)
                             filen = filesList[-1]
                             # el filename debera tener el siguiente formato
                             # 0 1234 567 89A BCDE (hex)
                             # x YYYY DDD SSS .ext
                             if isNumber(filen[8:11]):
                                 # inicializo mi contador de seteo al seteo del ultimo file
                                 setFile = int(filen[8:11])
                             else:
                                 setFile = -1
                         else:
                             setFile = -1  # inicializo mi contador de seteo
                     setFile += 1
                     # If this is a new day it resets some values
                     if self.dataOut.datatime.date() > self.fileDate:
                         setFile = 0
                         self.nTotalBlocks = 0
-                    filen = '%s%4.4d%3.3d%3.3d%s' % (
+                    filen = '{}{:04d}{:03d}{:03d}{}'.format(
-                        self.optchar, timeTuple.tm_year, timeTuple.tm_yday, setFile, ext)
+                            self.optchar, timeTuple.tm_year, timeTuple.tm_yday, setFile, ext)
                     filename = os.path.join(path, subfolder, filen)
                     fp = open(filename, 'wb')
                     self.blockIndex = 0
                     # guardando atributos
                     self.filename = filename
                     self.subfolder = subfolder
                     self.fp = fp
                     self.setFile = setFile
                     self.flagIsNewFile = 1
                     self.fileDate = self.dataOut.datatime.date()
                     self.setFirstHeader()
                     print '[Writing] Opening file: %s' % self.filename
                     self.__writeFirstHeader()
                     return 1
                 def setup(self, dataOut, path, blocksPerFile, profilesPerBlock=64, set=None, ext=None, datatype=4):
                     """
                     Setea el tipo de formato en la cual sera guardada la data y escribe el First Header
                     Inputs:
                         path                :    directory where data will be saved
                         profilesPerBlock    :    number of profiles per block
                         set                 :    initial file set
                         datatype            :    An integer number that defines data type:
 : int8  (1 byte)
 : int16 (2 bytes)
 : int32 (4 bytes)
 : int64 (8 bytes)
 : float32 (4 bytes)
 : double64 (8 bytes)
                     Return:
 :    Si no realizo un buen seteo
 :    Si realizo un buen seteo
                     """
                     if ext == None:
                         ext = self.ext
                     self.ext = ext.lower()
                     self.path = path
                     if set is None:
                         self.setFile = -1
                     else:
                         self.setFile = set - 1
                     self.blocksPerFile = blocksPerFile
                     self.profilesPerBlock = profilesPerBlock
                     self.dataOut = dataOut
                     self.fileDate = self.dataOut.datatime.date()
                     # By default
                     self.dtype = self.dataOut.dtype
                     if datatype is not None:
                         self.dtype = get_numpy_dtype(datatype)
                     if not(self.setNextFile()):
                         print "[Writing] There isn't a next file"
                         return 0
                     self.setBlockDimension()
                     return 1
                 def run(self, dataOut, path, blocksPerFile, profilesPerBlock=64, set=None, ext=None, datatype=4, **kwargs):
                     if not(self.isConfig):
                         self.setup(dataOut, path, blocksPerFile, profilesPerBlock=profilesPerBlock,
                                    set=set, ext=ext, datatype=datatype, **kwargs)
                         self.isConfig = True
                     self.putData()

schainpy/model/utils/jroutils_publish.py +35 -39

             '''
             @author: Juan C. Espinoza
             '''
             import time
             import json
             import numpy
             import paho.mqtt.client as mqtt
             import zmq
             import datetime
             from zmq.utils.monitor import recv_monitor_message
             from functools import wraps
             from threading import Thread
             from multiprocessing import Process
             from schainpy.model.proc.jroproc_base import Operation, ProcessingUnit
             from schainpy.model.data.jrodata import JROData
             from schainpy.utils import log
             MAXNUMX = 100
             MAXNUMY = 100
             class PrettyFloat(float):
                 def __repr__(self):
                     return '%.2f' % self
             def roundFloats(obj):
                 if isinstance(obj, list):
                     return map(roundFloats, obj)
                 elif isinstance(obj, float):
                     return round(obj, 2)
             def decimate(z, MAXNUMY):
                 dy = int(len(z[0])/MAXNUMY) + 1
                 return z[::, ::dy]
             class throttle(object):
                 '''
                 Decorator that prevents a function from being called more than once every
                 time period.
                 To create a function that cannot be called more than once a minute, but
                 will sleep until it can be called:
                 @throttle(minutes=1)
                 def foo():
                   pass
                 for i in range(10):
                   foo()
                   print "This function has run %s times." % i
                 '''
                 def __init__(self, seconds=0, minutes=0, hours=0):
                     self.throttle_period = datetime.timedelta(
                         seconds=seconds, minutes=minutes, hours=hours
                     )
                     self.time_of_last_call = datetime.datetime.min
                 def __call__(self, fn):
                     @wraps(fn)
                     def wrapper(*args, **kwargs):
                         coerce = kwargs.pop('coerce', None)
                         if coerce:
                             self.time_of_last_call = datetime.datetime.now()
                             return fn(*args, **kwargs)
                         else:
                             now = datetime.datetime.now()
                             time_since_last_call = now - self.time_of_last_call
                             time_left = self.throttle_period - time_since_last_call
                             if time_left > datetime.timedelta(seconds=0):
                                 return
                         self.time_of_last_call = datetime.datetime.now()
                         return fn(*args, **kwargs)
                     return wrapper
             class Data(object):
                 '''
                 Object to hold data to be plotted
                 '''
-                def __init__(self, plottypes, throttle_value):
+                def __init__(self, plottypes, throttle_value, exp_code):
                     self.plottypes = plottypes
                     self.throttle = throttle_value
+                    self.exp_code = exp_code
                     self.ended = False
                     self.localtime = False
                     self.__times = []
                     self.__heights = []
                 def __str__(self):
                     dum = ['{}{}'.format(key, self.shape(key)) for key in self.data]
                     return 'Data[{}][{}]'.format(';'.join(dum), len(self.__times))
                 def __len__(self):
                     return len(self.__times)
                 def __getitem__(self, key):
                     if key not in self.data:
                         raise KeyError(log.error('Missing key: {}'.format(key)))
                     if 'spc' in key:
                         ret = self.data[key]
                     else:
                         ret = numpy.array([self.data[key][x] for x in self.times])
                         if ret.ndim > 1:
                             ret = numpy.swapaxes(ret, 0, 1)
                     return ret
                 def __contains__(self, key):
                     return key in self.data
                 def setup(self):
                     '''
                     Configure object
                     '''
                     self.ended = False
                     self.data = {}
                     self.__times = []
                     self.__heights = []
                     self.__all_heights = set()
                     for plot in self.plottypes:
                         if 'snr' in plot:
                             plot = 'snr'
                         self.data[plot] = {}
                 def shape(self, key):
                     '''
                     Get the shape of the one-element data for the given key
                     '''
                     if len(self.data[key]):
                         if 'spc' in key:
                             return self.data[key].shape
                         return self.data[key][self.__times[0]].shape
                     return (0,)
                 def update(self, dataOut, tm):
                     '''
                     Update data object with new dataOut
                     '''
                     if tm in self.__times:
                         return
                     self.parameters = getattr(dataOut, 'parameters', [])
-                    self.pairs = dataOut.pairsList
+                    if hasattr(dataOut, 'pairsList'):
+                        self.pairs = dataOut.pairsList
                     self.channels = dataOut.channelList
                     self.interval = dataOut.getTimeInterval()
                     self.localtime = dataOut.useLocalTime
                     if 'spc' in self.plottypes or 'cspc' in self.plottypes:
                         self.xrange = (dataOut.getFreqRange(1)/1000., dataOut.getAcfRange(1), dataOut.getVelRange(1))
                     self.__heights.append(dataOut.heightList)
                     self.__all_heights.update(dataOut.heightList)
                     self.__times.append(tm)
                     for plot in self.plottypes:
                         if plot == 'spc':
                             z = dataOut.data_spc/dataOut.normFactor
                             self.data[plot] = 10*numpy.log10(z)
                         if plot == 'cspc':
                             self.data[plot] = dataOut.data_cspc
                         if plot == 'noise':
                             self.data[plot][tm] = 10*numpy.log10(dataOut.getNoise()/dataOut.normFactor)
                         if plot == 'rti':
                             self.data[plot][tm] = dataOut.getPower()
                         if plot == 'snr_db':
                             self.data['snr'][tm] = dataOut.data_SNR
                         if plot == 'snr':
                             self.data[plot][tm] = 10*numpy.log10(dataOut.data_SNR)
                         if plot == 'dop':
                             self.data[plot][tm] = 10*numpy.log10(dataOut.data_DOP)
                         if plot == 'mean':
                             self.data[plot][tm] = dataOut.data_MEAN
                         if plot == 'std':
                             self.data[plot][tm] = dataOut.data_STD
                         if plot == 'coh':
                             self.data[plot][tm] = dataOut.getCoherence()
                         if plot == 'phase':
                             self.data[plot][tm] = dataOut.getCoherence(phase=True)
                         if plot == 'output':
                             self.data[plot][tm] = dataOut.data_output
                         if plot == 'param':
                             self.data[plot][tm] = dataOut.data_param
                 def normalize_heights(self):
                     '''
                     Ensure same-dimension of the data for different heighList
                     '''
                     H = numpy.array(list(self.__all_heights))
                     H.sort()
                     for key in self.data:
                         shape = self.shape(key)[:-1] + H.shape
                         for tm, obj in self.data[key].items():
                             h = self.__heights[self.__times.index(tm)]
                             if H.size == h.size:
                                 continue
                             index = numpy.where(numpy.in1d(H, h))[0]
                             dummy = numpy.zeros(shape) + numpy.nan
                             if len(shape) == 2:
                                 dummy[:, index] = obj
                             else:
                                 dummy[index] = obj
                             self.data[key][tm] = dummy
                     self.__heights = [H for tm in self.__times]
                 def jsonify(self, decimate=False):
                     '''
                     Convert data to json
                     '''
-                    ret = {}
+                    data = {}
                     tm = self.times[-1]
-                    for key, value in self.data:
+                    for key in self.data:
                         if key in ('spc', 'cspc'):
-                            ret[key] = roundFloats(self.data[key].to_list())
+                            dx = int(self.data[key].shape[1]/MAXNUMX) + 1
+                            dy = int(self.data[key].shape[2]/MAXNUMY) + 1
+                            data[key] = roundFloats(self.data[key][::, ::dx, ::dy].tolist())
                         else:
-                            ret[key] = roundFloats(self.data[key][tm].to_list())
+                            data[key] = roundFloats(self.data[key][tm].tolist())
-                    ret['timestamp'] = tm
+                    ret = {'data': data}
+                    ret['exp_code'] = self.exp_code
+                    ret['time'] = tm
                     ret['interval'] = self.interval
+                    ret['localtime'] = self.localtime
+                    ret['yrange'] = roundFloats(self.heights.tolist())
+                    if key in ('spc', 'cspc'):
+                        ret['xrange'] = roundFloats(self.xrange[2][::dx].tolist())
+                    if hasattr(self, 'pairs'):
+                        ret['pairs'] = self.pairs
+                    return json.dumps(ret)
                 @property
                 def times(self):
                     '''
                     Return the list of times of the current data
                     '''
                     ret = numpy.array(self.__times)
                     ret.sort()
                     return ret
                 @property
                 def heights(self):
                     '''
                     Return the list of heights of the current data
                     '''
                     return numpy.array(self.__heights[-1])
             class PublishData(Operation):
                 '''
                 Operation to send data over zmq.
                 '''
                 __attrs__ = ['host', 'port', 'delay', 'zeromq', 'mqtt', 'verbose']
                 def __init__(self, **kwargs):
                     """Inicio."""
                     Operation.__init__(self, **kwargs)
                     self.isConfig = False
                     self.client = None
                     self.zeromq = None
                     self.mqtt = None
                 def on_disconnect(self, client, userdata, rc):
                     if rc != 0:
                         log.warning('Unexpected disconnection.')
                     self.connect()
                 def connect(self):
                     log.warning('trying to connect')
                     try:
                         self.client.connect(
                             host=self.host,
                             port=self.port,
                             keepalive=60*10,
                             bind_address='')
                         self.client.loop_start()
                         # self.client.publish(
                         #     self.topic + 'SETUP',
                         #     json.dumps(setup),
                         #     retain=True
                         #     )
                     except:
                         log.error('MQTT Conection error.')
                         self.client = False
                 def setup(self, port=1883, username=None, password=None, clientId="user", zeromq=1, verbose=True, **kwargs):
                     self.counter = 0
                     self.topic = kwargs.get('topic', 'schain')
                     self.delay = kwargs.get('delay', 0)
                     self.plottype = kwargs.get('plottype', 'spectra')
                     self.host = kwargs.get('host', "10.10.10.82")
                     self.port = kwargs.get('port', 3000)
                     self.clientId = clientId
                     self.cnt = 0
                     self.zeromq = zeromq
                     self.mqtt = kwargs.get('plottype', 0)
                     self.client = None
                     self.verbose = verbose
                     setup = []
                     if mqtt is 1:
                         self.client = mqtt.Client(
                             client_id=self.clientId + self.topic + 'SCHAIN',
                             clean_session=True)
                         self.client.on_disconnect = self.on_disconnect
                         self.connect()
                         for plot in self.plottype:
                             setup.append({
                                 'plot': plot,
                                 'topic': self.topic + plot,
                                 'title': getattr(self, plot + '_' + 'title', False),
                                 'xlabel': getattr(self, plot + '_' + 'xlabel', False),
                                 'ylabel': getattr(self, plot + '_' + 'ylabel', False),
                                 'xrange': getattr(self, plot + '_' + 'xrange', False),
                                 'yrange': getattr(self, plot + '_' + 'yrange', False),
                                 'zrange': getattr(self, plot + '_' + 'zrange', False),
                             })
                     if zeromq is 1:
                         context = zmq.Context()
                         self.zmq_socket = context.socket(zmq.PUSH)
                         server = kwargs.get('server', 'zmq.pipe')
                         if 'tcp://' in server:
                             address = server
                         else:
                             address = 'ipc:///tmp/%s' % server
                         self.zmq_socket.connect(address)
                         time.sleep(1)
                 def publish_data(self):
                     self.dataOut.finished = False
                     if self.mqtt is 1:
                         yData = self.dataOut.heightList[:2].tolist()
                         if self.plottype == 'spectra':
                             data = getattr(self.dataOut, 'data_spc')
                             z = data/self.dataOut.normFactor
                             zdB = 10*numpy.log10(z)
                             xlen, ylen = zdB[0].shape
                             dx = int(xlen/MAXNUMX) + 1
                             dy = int(ylen/MAXNUMY) + 1
                             Z = [0 for i in self.dataOut.channelList]
                             for i in self.dataOut.channelList:
                                 Z[i] = zdB[i][::dx, ::dy].tolist()
                             payload = {
                                 'timestamp': self.dataOut.utctime,
                                 'data': roundFloats(Z),
                                 'channels': ['Ch %s' % ch for ch in self.dataOut.channelList],
                                 'interval': self.dataOut.getTimeInterval(),
                                 'type': self.plottype,
                                 'yData': yData
                             }
                         elif self.plottype in ('rti', 'power'):
                             data = getattr(self.dataOut, 'data_spc')
                             z = data/self.dataOut.normFactor
                             avg = numpy.average(z, axis=1)
                             avgdB = 10*numpy.log10(avg)
                             xlen, ylen = z[0].shape
                             dy = numpy.floor(ylen/self.__MAXNUMY) + 1
                             AVG = [0 for i in self.dataOut.channelList]
                             for i in self.dataOut.channelList:
                                 AVG[i] = avgdB[i][::dy].tolist()
                             payload = {
                                 'timestamp': self.dataOut.utctime,
                                 'data': roundFloats(AVG),
                                 'channels': ['Ch %s' % ch for ch in self.dataOut.channelList],
                                 'interval': self.dataOut.getTimeInterval(),
                                 'type': self.plottype,
                                 'yData': yData
                             }
                         elif self.plottype == 'noise':
                             noise = self.dataOut.getNoise()/self.dataOut.normFactor
                             noisedB = 10*numpy.log10(noise)
                             payload = {
                                 'timestamp': self.dataOut.utctime,
                                 'data': roundFloats(noisedB.reshape(-1, 1).tolist()),
                                 'channels': ['Ch %s' % ch for ch in self.dataOut.channelList],
                                 'interval': self.dataOut.getTimeInterval(),
                                 'type': self.plottype,
                                 'yData': yData
                             }
                         elif self.plottype == 'snr':
                             data = getattr(self.dataOut, 'data_SNR')
                             avgdB = 10*numpy.log10(data)
                             ylen = data[0].size
                             dy = numpy.floor(ylen/self.__MAXNUMY) + 1
                             AVG = [0 for i in self.dataOut.channelList]
                             for i in self.dataOut.channelList:
                                 AVG[i] = avgdB[i][::dy].tolist()
                             payload = {
                                 'timestamp': self.dataOut.utctime,
                                 'data': roundFloats(AVG),
                                 'channels': ['Ch %s' % ch for ch in self.dataOut.channelList],
                                 'type': self.plottype,
                                 'yData': yData
                             }
                         else:
                             print "Tipo de grafico invalido"
                             payload = {
                                 'data': 'None',
                                 'timestamp': 'None',
                                 'type': None
                             }
                         self.client.publish(self.topic + self.plottype, json.dumps(payload), qos=0)
                     if self.zeromq is 1:
                         if self.verbose:
                             log.log(
                                 'Sending {} - {}'.format(self.dataOut.type, self.dataOut.datatime),
                                 self.name
                             )
                         self.zmq_socket.send_pyobj(self.dataOut)
                 def run(self, dataOut, **kwargs):
                     self.dataOut = dataOut
                     if not self.isConfig:
                         self.setup(**kwargs)
                         self.isConfig = True
                     self.publish_data()
                     time.sleep(self.delay)
                 def close(self):
                     if self.zeromq is 1:
                         self.dataOut.finished = True
                         self.zmq_socket.send_pyobj(self.dataOut)
                         time.sleep(0.1)
                         self.zmq_socket.close()
                     if self.client:
                         self.client.loop_stop()
                         self.client.disconnect()
             class ReceiverData(ProcessingUnit):
                 __attrs__ = ['server']
                 def __init__(self, **kwargs):
                     ProcessingUnit.__init__(self, **kwargs)
                     self.isConfig = False
                     server = kwargs.get('server', 'zmq.pipe')
                     if 'tcp://' in server:
                         address = server
                     else:
                         address = 'ipc:///tmp/%s' % server
                     self.address = address
                     self.dataOut = JROData()
                 def setup(self):
                     self.context = zmq.Context()
                     self.receiver = self.context.socket(zmq.PULL)
                     self.receiver.bind(self.address)
                     time.sleep(0.5)
                     log.success('ReceiverData from {}'.format(self.address))
                 def run(self):
                     if not self.isConfig:
                         self.setup()
                         self.isConfig = True
                     self.dataOut = self.receiver.recv_pyobj()
                     log.log('{} - {}'.format(self.dataOut.type,
                                              self.dataOut.datatime.ctime(),),
                             'Receiving')
             class PlotterReceiver(ProcessingUnit, Process):
                 throttle_value = 5
-                __attrs__ = ['server', 'plottypes', 'realtime', 'localtime', 'throttle']
+                __attrs__ = ['server', 'plottypes', 'realtime', 'localtime', 'throttle',
+                    'exp_code', 'web_server']
                 def __init__(self, **kwargs):
                     ProcessingUnit.__init__(self, **kwargs)
                     Process.__init__(self)
                     self.mp = False
                     self.isConfig = False
                     self.isWebConfig = False
                     self.connections = 0
                     server = kwargs.get('server', 'zmq.pipe')
-                    plot_server = kwargs.get('plot_server', 'zmq.web')
+                    web_server = kwargs.get('web_server', None)
                     if 'tcp://' in server:
                         address = server
                     else:
                         address = 'ipc:///tmp/%s' % server
-                    if 'tcp://' in plot_server:
-                        plot_address = plot_server
-                    else:
-                        plot_address = 'ipc:///tmp/%s' % plot_server
                     self.address = address
-                    self.plot_address = plot_address
+                    self.web_address = web_server
                     self.plottypes = [s.strip() for s in kwargs.get('plottypes', 'rti').split(',')]
                     self.realtime = kwargs.get('realtime', False)
                     self.localtime = kwargs.get('localtime', True)
                     self.throttle_value = kwargs.get('throttle', 5)
+                    self.exp_code = kwargs.get('exp_code', None)
                     self.sendData = self.initThrottle(self.throttle_value)
                     self.dates = []
                     self.setup()
                 def setup(self):
-                    self.data = Data(self.plottypes, self.throttle_value)
+                    self.data = Data(self.plottypes, self.throttle_value, self.exp_code)
                     self.isConfig = True
                 def event_monitor(self, monitor):
                     events = {}
                     for name in dir(zmq):
                         if name.startswith('EVENT_'):
                             value = getattr(zmq, name)
                             events[value] = name
                     while monitor.poll():
                         evt = recv_monitor_message(monitor)
                         if evt['event'] == 32:
                             self.connections += 1
                         if evt['event'] == 512:
                             pass
                         evt.update({'description': events[evt['event']]})
                         if evt['event'] == zmq.EVENT_MONITOR_STOPPED:
                             break
                     monitor.close()
                     print('event monitor thread done!')
                 def initThrottle(self, throttle_value):
                     @throttle(seconds=throttle_value)
                     def sendDataThrottled(fn_sender, data):
                         fn_sender(data)
                     return sendDataThrottled
                 def send(self, data):
                     log.success('Sending {}'.format(data), self.name)
                     self.sender.send_pyobj(data)
                 def run(self):
                     log.success(
                         'Starting from {}'.format(self.address),
                         self.name
                     )
                     self.context = zmq.Context()
                     self.receiver = self.context.socket(zmq.PULL)
                     self.receiver.bind(self.address)
                     monitor = self.receiver.get_monitor_socket()
                     self.sender = self.context.socket(zmq.PUB)
-                    if self.realtime:
+                    if self.web_address:
+                        log.success(
+                            'Sending to web: {}'.format(self.web_address),
+                            self.name
+                        )
                         self.sender_web = self.context.socket(zmq.PUB)
-                        self.sender_web.connect(self.plot_address)
+                        self.sender_web.connect(self.web_address)
                         time.sleep(1)
                     if 'server' in self.kwargs:
                         self.sender.bind("ipc:///tmp/{}.plots".format(self.kwargs['server']))
                     else:
                         self.sender.bind("ipc:///tmp/zmq.plots")
                     time.sleep(2)
                     t = Thread(target=self.event_monitor, args=(monitor,))
                     t.start()
                     while True:
                         dataOut = self.receiver.recv_pyobj()
                         if not dataOut.flagNoData:
                             if dataOut.type == 'Parameters':
                                 tm = dataOut.utctimeInit
                             else:
                                 tm = dataOut.utctime
                             if dataOut.useLocalTime:
                                 if not self.localtime:
                                     tm += time.timezone
                                 dt = datetime.datetime.fromtimestamp(tm).date()
                             else:
                                 if self.localtime:
                                     tm -= time.timezone
                                 dt = datetime.datetime.utcfromtimestamp(tm).date()
                             coerce = False
                             if dt not in self.dates:
                                 if self.data:
                                     self.data.ended = True
                                     self.send(self.data)
                                     coerce = True
                                 self.data.setup()
                                 self.dates.append(dt)
                             self.data.update(dataOut, tm)
                         if dataOut.finished is True:
                             self.connections -= 1
                             if self.connections == 0 and dt in self.dates:
                                 self.data.ended = True
                                 self.send(self.data)
                                 self.data.setup()
                         else:
                             if self.realtime:
                                 self.send(self.data)
-                                # self.sender_web.send_string(self.data.jsonify())
+                                if self.web_address:
+                                    self.sender_web.send(self.data.jsonify())
                             else:
                                 self.sendData(self.send, self.data, coerce=coerce)
                                 coerce = False
                     return
-                def sendToWeb(self):
-                    if not self.isWebConfig:
-                        context = zmq.Context()
-                        sender_web_config = context.socket(zmq.PUB)
-                        if 'tcp://' in self.plot_address:
-                            dum, address, port = self.plot_address.split(':')
-                            conf_address = '{}:{}:{}'.format(dum, address, int(port)+1)
-                        else:
-                            conf_address = self.plot_address + '.config'
-                        sender_web_config.bind(conf_address)
-                        time.sleep(1)
-                        for kwargs in self.operationKwargs.values():
-                            if 'plot' in kwargs:
-                                log.success('[Sending] Config data to web for {}'.format(kwargs['code'].upper()))
-                                sender_web_config.send_string(json.dumps(kwargs))
-                        self.isWebConfig = True

setup.py +27 -37

             '''
             Created on Jul 16, 2014
             @author: Miguel Urco
             '''
             import os
             from setuptools import setup, Extension
             from setuptools.command.build_ext import build_ext as _build_ext
             from schainpy import __version__
-            from schainpy.utils import log
             class build_ext(_build_ext):
                 def finalize_options(self):
                     _build_ext.finalize_options(self)
                     # Prevent numpy from thinking it is still in its setup process:
                     __builtins__.__NUMPY_SETUP__ = False
                     import numpy
                     self.include_dirs.append(numpy.get_include())
+            setup(name = "schainpy",
-            try:
+                  version = __version__,
-                from PyQt4 import QtCore, QtGui
+                  description = "Python tools to read, write and process Jicamarca data",
-                from PyQt4.QtGui import QApplication
+                  author = "Miguel Urco",
-            except:
+                  author_email = "miguel.urco@jro.igp.gob.pe",
-                log.warning(
+                  url = "http://jro.igp.gob.pe",
-                    'You should install PyQt4 module in order to run the GUI. See the README.')
+                  packages = {'schainpy',
-            setup(name="schainpy",
-                  version=__version__,
-                  description="Python tools to read, write and process Jicamarca data",
-                  author="Miguel Urco",
-                  author_email="miguel.urco@jro.igp.gob.pe",
-                  url="http://jro.igp.gob.pe",
-                  packages={'schainpy',
                               'schainpy.model',
                               'schainpy.model.data',
                               'schainpy.model.graphics',
                               'schainpy.model.io',
                               'schainpy.model.proc',
                               'schainpy.model.serializer',
                               'schainpy.model.utils',
+                              'schainpy.utils',
                               'schainpy.gui',
                               'schainpy.gui.figures',
                               'schainpy.gui.viewcontroller',
                               'schainpy.gui.viewer',
-                              'schainpy.gui.viewer.windows'},
+                              'schainpy.gui.viewer.windows',
-                  ext_package='schainpy',
+                              'schainpy.cli'},
-                  py_modules=[''],
+                  ext_package = 'schainpy',
-                  package_data={'': ['schain.conf.template'],
+                  package_data = {'': ['schain.conf.template'],
-                                'schainpy.gui.figures': ['*.png', '*.jpg'],
+                                  'schainpy.gui.figures': ['*.png', '*.jpg'],
-                                },
+                                 },
-                  include_package_data=False,
+                  include_package_data = False,
-                  scripts=['schainpy/gui/schainGUI'],
+                  scripts = ['schainpy/gui/schainGUI'],
-                  ext_modules=[
+                  ext_modules = [
-                      Extension("cSchain", ["schainpy/model/proc/extensions.c"]
+                      Extension("cSchain", ["schainpy/model/proc/extensions.c"])
-                                )],
+                      ],
-                  entry_points={
+                  entry_points = {
                       'console_scripts': [
-                          'schain = schaincli.cli:main',
+                          'schain = schainpy.cli.cli:main',
                       ],
                   },
-                  cmdclass={'build_ext': build_ext},
+                  cmdclass = {'build_ext': build_ext},
-                  setup_requires=["numpy >= 1.11.2"],
+                  setup_requires = ["numpy >= 1.11.2"],
-                  install_requires=[
+                  install_requires = [
                       "scipy >= 0.14.0",
                       "h5py >= 2.2.1",
-                      "matplotlib >= 1.4.2",
+                      "matplotlib >= 2.0.0",
                       "pyfits >= 3.4",
                       "paramiko >= 2.1.2",
                       "paho-mqtt >= 1.2",
                       "zmq",
                       "fuzzywuzzy",
                       "click",
                       "python-Levenshtein"
-                  ],
+                      ],
+            )

General Comments 0

Write
Preview

You need to be logged in to leave comments. Login now

No TODOs yet

	Site-wide shortcuts
/	Use quick search box
g h	Goto home page
g g	Goto my private gists page
g G	Goto my public gists page
g 0-9	Goto bookmarked items from 0-9
n r	New repository page
n g	New gist page

	Repositories
g s	Goto summary page
g c	Goto changelog page
g f	Goto files page
g F	Goto files page with file search activated
g p	Goto pull requests page
g o	Goto repository settings
g O	Goto repository access permissions settings
t s	Toggle sidebar on some pages