schain-jc Commit - r1054:10ce564971bb

merge from master

José Chávez -

r1054:10ce564971bb

parent child

Context file:

r1054:10ce564971bb

Expand all files

The requested changes are too big and content was truncated. Show full diff

schainpy/CHANGELOG.md created 644 +108 0

@@ -0,0 +1,108
	1	## CHANGELOG:
	2
	3	### 2.3
	4	* Added high order function `multiSchain` for multiprocessing scripts.
	5	* Added two new Processing Units `PublishData` and `ReceiverData` for receiving and sending dataOut through multiple ways (tcp, ipc, inproc).
	6	* Added a new graphics Processing Unit `PlotterReceiver`. It is decoupled from normal processing sequence with support for data generated by multiprocessing scripts.
	7	* Added support for sending realtime graphic to web server.
	8	* XML command `schain` is now `schain --xml`.
	9	* Added a CLI tool named `schain`.
	10	* Scripts templates can be now generated with `schain generate`.
	11	* Now it is possible to search Processing Units and Operations with `schain search [module]` to get the right name and its allowed parameters.
	12	* Added suggestions when parameters are poorly written.
	13	* `Controller.start()` now runs in a different process than the process calling it.
	14	* Added `schainpy.utils.log` for log standarization.
	15	* Running script on online mode no longer ignores date and hour. Issue #1109.
	16	* Added support for receving voltage data directly from JARS (tcp, ipc).
	17	* Updated README for compatibility with MAC OS GUI installation.
	18	* Setup now installs numpy.
	19
	20	### 2.2.6
	21	* Graphics generated by the GUI are now the same as generated by scripts. Issue #1074.
	22	* Added support for C extensions.
	23	* Function `hildebrand_sehkon` optimized with a C wrapper.
	24	* Numpy version updated.
	25	* Migration to GIT.
	26
	27	### 2.2.5:
	28	* splitProfiles and combineProfiles modules were added to VoltageProc and Signal Chain GUI.
	29	* nProfiles of USRP data (hdf5) is the number of profiles thera are in one second.
	30	* jroPlotter works directly with data objects instead of dictionaries
	31	* script "schain" was added to Signal Chain installer
	32
	33	### 2.2.4.1:
	34	* jroIO_usrp.py is update to read Sandra's data
	35	* decimation in Spectra and RTI plots is always enabled.
	36	* time* window option added to GUI
	37
	38	### 2.2.4:
	39	* jroproc_spectra_lags.py added to schainpy
	40	* Bug fixed in schainGUI: ProcUnit was created with the same id in some cases.
	41	* Bug fixed in jroHeaderIO: Header size validation.
	42
	43	### 2.2.3.1:
	44	* Filtering block by time has been added.
	45	* Bug fixed plotting RTI, CoherenceMap and others using xmin and xmax parameters. The first day worked
	46	properly but the next days did not.
	47
	48	### 2.2.3:
	49	* Bug fixed in GUI: Error getting(reading) Code value
	50	* Bug fixed in GUI: Flip option always needs channelList field
	51	* Bug fixed in jrodata: when one branch modified a value in "dataOut" (example: dataOut.code) this value
	52	was modified for every branch (because this was a reference). It was modified in data.copy()
	53	* Bug fixed in jroproc_voltage.profileSelector(): rangeList replaces to profileRangeList.
	54
	55	### 2.2.2:
	56	* VoltageProc: ProfileSelector, Reshape, Decoder with nTxs!=1 and getblock=True was tested
	57	* Rawdata and testRawdata.py added to Signal Chain project
	58
	59	### 2.2.1:
	60	* Bugs fixed in GUI
	61	* Views were improved in GUI
	62	* Support to MST* ISR experiments
	63	* Bug fixed getting noise using hyldebrant. (minimum number of points > 20%)
	64	* handleError added to jroplotter.py
	65
	66	### 2.2.0:
	67	* GUI: use of external plotter
	68	* Compatible with matplotlib 1.5.0
	69
	70	### 2.1.5:
	71	* serializer module added to Signal Chain
	72	* jroplotter.py added to Signal Chain
	73
	74	### 2.1.4.2:
	75	* A new Plotter Class was added
	76	* Project.start() does not accept filename as a parameter anymore
	77
	78	### 2.1.4.1:
	79	* Send notifications when an error different to ValueError is detected
	80
	81	### 2.1.4:
	82	* Sending error notifications to signal chain administrator
	83	* Login to email server added
	84
	85	### 2.1.3.3:
	86	* Colored Button Icons were added to GUI
	87
	88	### 2.1.3.2:
	89	* GUI: user interaction enhanced
	90	* controller_api.py: Safe access to ControllerThead
	91
	92	### 2.1.3.1:
	93	* GUI: every icon were resized
	94	* jroproc_voltage.py: Print a message when "Read from code" option is selected and the code is not defined inside data file
	95
	96	### 2.1.3:
	97	* jroplot_heispectra.py: SpectraHeisScope was not showing the right channels
	98	* jroproc_voltage.py: Bug fixed selecting profiles (self.nProfiles took a wrong value),
	99	Bug fixed selecting heights by block (selecting profiles instead heights)
	100	* jroproc_voltage.py: New feature added: decoding data by block using FFT.
	101	* jroIO_heispectra.py: Bug fixed in FitsReader. Using local Fits instance instead schainpy.mode.data.jrodata.Fits.
	102	* jroIO_heispectra.py: Channel index list does not exist.
	103
	104	### 2.1.2:
	105	* jroutils_ftp.py: Bug fixed, Any error sending file stopped the Server Thread
	106	Server thread opens and closes remote server each time file list is sent
	107	* jroplot_spectra.py: Noise path was not being created when noise data is saved.
	108	* jroIO_base.py: startTime can be greater than endTime. Example: SpreadF [18:00 * 07:00] No newline at end of file

.gitignore +11 -1

             # Byte-compiled / optimized / DLL files
             __pycache__/
             *.py[cod]
             *$py.class
             # C extensions
             *.so
             # Distribution / packaging
             .Python
             env/
             build/
             develop-eggs/
             dist/
             downloads/
             eggs/
             .eggs/
             lib/
             lib64/
             parts/
             sdist/
             var/
             wheels/
             *.egg-info/
             .installed.cfg
             *.egg
             # PyInstaller
             #  Usually these files are written by a python script from a template
             #  before PyInstaller builds the exe, so as to inject date/other infos into it.
             *.manifest
             *.spec
             # Installer logs
             pip-log.txt
             pip-delete-this-directory.txt
             # Unit test / coverage reports
             htmlcov/
             .tox/
             .coverage
             .coverage.*
             .cache
             nosetests.xml
             coverage.xml
             *,cover
             .hypothesis/
             # Translations
             *.mo
             *.pot
             # Django stuff:
             *.log
             local_settings.py
             # Flask stuff:
             instance/
             .webassets-cache
             # Scrapy stuff:
             .scrapy
             # Sphinx documentation
             docs/_build/
             # PyBuilder
             target/
             # Jupyter Notebook
             .ipynb_checkpoints
             # pyenv
             .python-version
             # celery beat schedule file
             celerybeat-schedule
             # SageMath parsed files
             *.sage.py
             # dotenv
             .env
             # virtualenv
             .venv
             venv/
             ENV/
             # Spyder project settings
             .spyderproject
             .spyproject
             # Rope project settings
             .ropeproject
             # mkdocs documentation
             /site
             # eclipse
             .project
             .pydevproject
+            <<<<<<< HEAD
             # vscode
             .vscode
             schaingui/node_modules/
-            schainpy/scripts/
  No newline at end of file
+            schainpy/scripts/
+            =======
+            .svn/
+            *.png
+            *.pyc
+            schainpy/scripts
+            .vscode
+            schaingui/node_modules
+            trash
+            >>>>>>> master

README.md +24 -8

             # Signal Chain
             ## Introduction
             Signal Chain (SCh) is a radar data processing library developed using [Python](www.python.org) at JRO. SCh provides modules to read, write, process and plot data.
             ## Installation
-            Install system dependencies, download the latest stable release from [svn](http://jro-dev.igp.gob.pe/svn/jro_soft/schain/Releases/) e.g. schainpy-2.2.5.tar.gz. and install it as a normal python package.
+            Install system dependencies, clone the latest version from [git](http://jro-dev.igp.gob.pe/rhodecode/schain/) and install it as a normal python package.
+            ### Linux based system
             ```
             $ sudo apt-get install python-pip python-dev gfortran libpng-dev freetype* libblas-dev liblapack-dev libatlas-base-dev python-qt4 python-tk libssl-dev libhdf5-dev
-            $ tar xvzf schainpy-2.2.5.tar.gz
+            $ sudo pip install numpy
-            $ cd schainpy-2.2.5
+            $ git clone http://jro-dev.igp.gob.pe/rhodecode/schain/
+            $ cd schain
             $ sudo pip install ./
-            ```
-            **Its recommended to install schain in a virtual environment**
             ```
+            **It is recommended to install schain in a virtual environment**
+            ```
             $ sudo pip install virtualenv
             $ virtualenv /path/to/virtual --system-site-packages
             $ source /path/to/virtual/bin/activate
-            (virtual) $ cd schainpy-2.2.5
+            (virtual) $ cd schain
             (virtual) $ pip install ./
+            ```
+            ### MAC Os
             ```
+            $ brew install cartr/qt4/pyqt
+            $ git clone http://jro-dev.igp.gob.pe/rhodecode/schain/
+            $ cd schain
+            $ pip install ./
+            ```
+            if ```pip install ./``` does not work, install a proper python enviroment, and repeat the steps.
+            ```
+            $ brew install python
+            ```
             ## First Script
             Read Spectra data (.pdata) - remove dc - plot spectra & RTI
             Import SCh and creating a project
             ```python
             #!/usr/bin/python
             from schainpy.controller import Project
             controller = Project()
             controller.setup(id = '100',
                              name='test',
                              description='Basic experiment')
             ```
             Adding read unit and operations
             ```python
             read_unit = controller.addReadUnit(datatype='Spectra',
                                                path='/path/to/pdata/',
                                                startDate='2014/01/31',
                                                endDate='2014/03/31',
                                                startTime='00:00:00',
                                                endTime='23:59:59',
                                                online=0,
                                                walk=0)
             proc_unit = controller.addProcUnit(datatype='Spectra',
                                                inputId=read_unit.getId())
             op = proc_unit.addOperation(name='selectChannels')
             op.addParameter(name='channelList', value='0,1', format='intlist')
             op = proc_unit.addOperation(name='selectHeights')
             op.addParameter(name='minHei', value='80', format='float')
             op.addParameter(name='maxHei', value='200', format='float')
             op = proc_unit.addOperation(name='removeDC')
             ```
             Plotting data & start project
             ```python
             op = proc_unit.addOperation(name='SpectraPlot', optype='other')
             op.addParameter(name='id', value='1', format='int')
             op.addParameter(name='wintitle', value='Spectra', format='str')
             op = procUnitConfObj1.addOperation(name='RTIPlot', optype='other')
             op.addParameter(name='id', value='2', format='int')
             op.addParameter(name='wintitle', value='RTI', format='str')
             controller.start()
             ```
             Full script
             ```python
             #!/usr/bin/python
             from schainpy.controller import Project
             controller = Project()
             controller.setup(id = '100',
                              name='test',
                              description='Basic experiment')
             read_unit = controller.addReadUnit(datatype='Spectra',
                                                path='/path/to/pdata/',
                                                startDate='2014/01/31',
                                                endDate='2014/03/31',
                                                startTime='00:00:00',
                                                endTime='23:59:59',
                                                online=0,
                                                walk=0)
             proc_unit = controller.addProcUnit(datatype='Spectra',
                                                inputId=read_unit.getId())
             op = proc_unit.addOperation(name='selectChannels')
             op.addParameter(name='channelList', value='0,1', format='intlist')
             op = proc_unit.addOperation(name='selectHeights')
             op.addParameter(name='minHei', value='80', format='float')
             op.addParameter(name='maxHei', value='200', format='float')
             op = proc_unit.addOperation(name='removeDC')
             op = proc_unit.addOperation(name='SpectraPlot', optype='other')
             op.addParameter(name='id', value='6', format='int')
             op.addParameter(name='wintitle', value='Spectra', format='str')
             op = procUnitConfObj1.addOperation(name='RTIPlot', optype='other')
             op.addParameter(name='id', value='2', format='int')
             op.addParameter(name='wintitle', value='RTI', format='str')
             controller.start()
-            ```
+            ```
  No newline at end of file

schainpy/__init__.py +2 -1

             '''
             Created on Feb 7, 2012
             @author $Author$
             @version $Id$
             '''
-            __version__ = "2.3"
  No newline at end of file
+            __version__ = "2.3"

schainpy/controller.py +39 -39

             '''
             Created on September , 2012
             @author:
             '''
             import sys
             import ast
             import datetime
             import traceback
             import math
             import time
             from multiprocessing import Process, Queue, cpu_count
             import schainpy
             import schainpy.admin
             from xml.etree.ElementTree import ElementTree, Element, SubElement, tostring
             from xml.dom import minidom
             from schainpy.model import *
             from time import sleep
             def prettify(elem):
                 """Return a pretty-printed XML string for the Element.
                 """
                 rough_string = tostring(elem, 'utf-8')
                 reparsed = minidom.parseString(rough_string)
                 return reparsed.toprettyxml(indent="  ")
             def multiSchain(child, nProcess=cpu_count(), startDate=None, endDate=None, by_day=False):
                 skip = 0
                 cursor = 0
                 nFiles = None
                 processes = []
                 dt1 = datetime.datetime.strptime(startDate, '%Y/%m/%d')
                 dt2 = datetime.datetime.strptime(endDate, '%Y/%m/%d')
                 days = (dt2 - dt1).days
                 for day in range(days+1):
                     skip = 0
                     cursor = 0
                     q = Queue()
                     processes = []
                     dt = (dt1 + datetime.timedelta(day)).strftime('%Y/%m/%d')
                     firstProcess = Process(target=child, args=(cursor, skip, q, dt))
                     firstProcess.start()
                     if by_day:
                         continue
                     nFiles = q.get()
                     firstProcess.terminate()
                     skip = int(math.ceil(nFiles/nProcess))
                     while True:
                         processes.append(Process(target=child, args=(cursor, skip, q, dt)))
                         processes[cursor].start()
                         if nFiles < cursor*skip:
                             break
                         cursor += 1
                     def beforeExit(exctype, value, trace):
                         for process in processes:
                             process.terminate()
                             process.join()
                         print traceback.print_tb(trace)
                     sys.excepthook = beforeExit
                     for process in processes:
                         process.join()
                         process.terminate()
                     time.sleep(3)
             class ParameterConf():
                 id = None
                 name = None
                 value = None
                 format = None
                 __formated_value = None
                 ELEMENTNAME = 'Parameter'
                 def __init__(self):
                     self.format = 'str'
                 def getElementName(self):
                     return self.ELEMENTNAME
                 def getValue(self):
                     value = self.value
                     format = self.format
                     if self.__formated_value != None:
                         return self.__formated_value
                     if format == 'obj':
                         return value
                     if format == 'str':
                         self.__formated_value = str(value)
                         return self.__formated_value
                     if value == '':
                         raise ValueError, "%s: This parameter value is empty" %self.name
                     if format == 'list':
                         strList = value.split(',')
                         self.__formated_value = strList
                         return self.__formated_value
                     if format == 'intlist':
                         """
                         Example:
                             value = (0,1,2)
                         """
                         new_value = ast.literal_eval(value)
                         if type(new_value) not in (tuple, list):
                             new_value = [int(new_value)]
                         self.__formated_value = new_value
                         return self.__formated_value
                     if format == 'floatlist':
                         """
                         Example:
                             value = (0.5, 1.4, 2.7)
                         """
                         new_value = ast.literal_eval(value)
                         if type(new_value) not in (tuple, list):
                             new_value = [float(new_value)]
                         self.__formated_value = new_value
                         return self.__formated_value
                     if format == 'date':
                         strList = value.split('/')
                         intList = [int(x) for x in strList]
                         date = datetime.date(intList[0], intList[1], intList[2])
                         self.__formated_value = date
                         return self.__formated_value
                     if format == 'time':
                         strList = value.split(':')
                         intList = [int(x) for x in strList]
                         time = datetime.time(intList[0], intList[1], intList[2])
                         self.__formated_value = time
                         return self.__formated_value
                     if format == 'pairslist':
                         """
                         Example:
                             value = (0,1),(1,2)
                         """
                         new_value = ast.literal_eval(value)
                         if type(new_value) not in (tuple, list):
                             raise ValueError, "%s has to be a tuple or list of pairs" %value
                         if type(new_value[0]) not in (tuple, list):
                             if len(new_value) != 2:
                                 raise ValueError, "%s has to be a tuple or list of pairs" %value
                             new_value = [new_value]
                         for thisPair in new_value:
                             if len(thisPair) != 2:
                                 raise ValueError, "%s has to be a tuple or list of pairs" %value
                         self.__formated_value = new_value
                         return self.__formated_value
                     if format == 'multilist':
                         """
                         Example:
                             value = (0,1,2),(3,4,5)
                         """
                         multiList = ast.literal_eval(value)
                         if type(multiList[0]) == int:
                             multiList = ast.literal_eval("(" + value + ")")
                         self.__formated_value = multiList
                         return self.__formated_value
                     if format == 'bool':
                         value = int(value)
                     if format == 'int':
                         value = float(value)
                     format_func = eval(format)
                     self.__formated_value = format_func(value)
                     return self.__formated_value
                 def updateId(self, new_id):
                     self.id = str(new_id)
                 def setup(self, id, name, value, format='str'):
                     self.id = str(id)
                     self.name = name
                     if format == 'obj':
                         self.value = value
                     else:
                         self.value = str(value)
                     self.format = str.lower(format)
                     self.getValue()
                     return 1
                 def update(self, name, value, format='str'):
                     self.name = name
                     self.value = str(value)
                     self.format = format
                 def makeXml(self, opElement):
                     if self.name not in ('queue',):
                         parmElement = SubElement(opElement, self.ELEMENTNAME)
                         parmElement.set('id', str(self.id))
                         parmElement.set('name', self.name)
                         parmElement.set('value', self.value)
                         parmElement.set('format', self.format)
                 def readXml(self, parmElement):
                     self.id = parmElement.get('id')
                     self.name = parmElement.get('name')
                     self.value = parmElement.get('value')
                     self.format = str.lower(parmElement.get('format'))
                     #Compatible with old signal chain version
                     if self.format == 'int' and self.name == 'idfigure':
                         self.name = 'id'
                 def printattr(self):
                     print "Parameter[%s]: name = %s, value = %s, format = %s" %(self.id, self.name, self.value, self.format)
             class OperationConf():
                 id = None
                 name = None
                 priority = None
                 type = None
                 parmConfObjList = []
                 ELEMENTNAME = 'Operation'
                 def __init__(self):
                     self.id = '0'
                     self.name = None
                     self.priority = None
                     self.type = 'self'
                 def __getNewId(self):
                     return int(self.id)*10 + len(self.parmConfObjList) + 1
                 def updateId(self, new_id):
                     self.id = str(new_id)
                     n = 1
                     for parmObj in self.parmConfObjList:
                         idParm = str(int(new_id)*10 + n)
                         parmObj.updateId(idParm)
                         n += 1
                 def getElementName(self):
                     return self.ELEMENTNAME
                 def getParameterObjList(self):
                     return self.parmConfObjList
                 def getParameterObj(self, parameterName):
                     for parmConfObj in self.parmConfObjList:
                         if parmConfObj.name != parameterName:
                             continue
                         return parmConfObj
                     return None
                 def getParameterObjfromValue(self, parameterValue):
                     for parmConfObj in self.parmConfObjList:
                         if parmConfObj.getValue() != parameterValue:
                             continue
                         return parmConfObj.getValue()
                     return None
                 def getParameterValue(self, parameterName):
                     parameterObj = self.getParameterObj(parameterName)
-            #         if not parameterObj:
+                #         if not parameterObj:
-            #             return None
+                #             return None
                     value = parameterObj.getValue()
                     return value
                 def getKwargs(self):
                     kwargs = {}
                     for parmConfObj in self.parmConfObjList:
                         if self.name == 'run' and parmConfObj.name == 'datatype':
                             continue
                         kwargs[parmConfObj.name] = parmConfObj.getValue()
                     return kwargs
                 def setup(self, id, name, priority, type):
                     self.id = str(id)
                     self.name = name
                     self.type = type
                     self.priority = priority
                     self.parmConfObjList = []
                 def removeParameters(self):
                     for obj in self.parmConfObjList:
                         del obj
                     self.parmConfObjList = []
                 def addParameter(self, name, value, format='str'):
                     id = self.__getNewId()
                     parmConfObj = ParameterConf()
                     if not parmConfObj.setup(id, name, value, format):
                         return None
                     self.parmConfObjList.append(parmConfObj)
                     return parmConfObj
                 def changeParameter(self, name, value, format='str'):
                     parmConfObj = self.getParameterObj(name)
                     parmConfObj.update(name, value, format)
                     return parmConfObj
                 def makeXml(self, procUnitElement):
                     opElement = SubElement(procUnitElement, self.ELEMENTNAME)
                     opElement.set('id', str(self.id))
                     opElement.set('name', self.name)
                     opElement.set('type', self.type)
                     opElement.set('priority', str(self.priority))
                     for parmConfObj in self.parmConfObjList:
                         parmConfObj.makeXml(opElement)
                 def readXml(self, opElement):
                     self.id = opElement.get('id')
                     self.name = opElement.get('name')
                     self.type = opElement.get('type')
                     self.priority = opElement.get('priority')
                     #Compatible with old signal chain version
                     #Use of 'run' method instead 'init'
                     if self.type == 'self' and self.name == 'init':
                         self.name = 'run'
                     self.parmConfObjList = []
                     parmElementList = opElement.iter(ParameterConf().getElementName())
                     for parmElement in parmElementList:
                         parmConfObj = ParameterConf()
                         parmConfObj.readXml(parmElement)
                         #Compatible with old signal chain version
                         #If an 'plot' OPERATION is found, changes name operation by the value of its type PARAMETER
                         if self.type != 'self' and self.name == 'Plot':
                             if parmConfObj.format == 'str' and parmConfObj.name == 'type':
                                 self.name = parmConfObj.value
                                 continue
                         self.parmConfObjList.append(parmConfObj)
                 def printattr(self):
                     print "%s[%s]: name = %s, type = %s, priority = %s" %(self.ELEMENTNAME,
                                                                           self.id,
                                                                           self.name,
                                                                           self.type,
                                                                           self.priority)
                     for parmConfObj in self.parmConfObjList:
                         parmConfObj.printattr()
                 def createObject(self, plotter_queue=None):
                     if self.type == 'self':
                         raise ValueError, "This operation type cannot be created"
                     if self.type == 'plotter':
                         #Plotter(plotter_name)
                         if not plotter_queue:
                             raise ValueError, "plotter_queue is not defined. Use:\nmyProject = Project()\nmyProject.setPlotterQueue(plotter_queue)"
                         opObj = Plotter(self.name, plotter_queue)
                     if self.type == 'external' or self.type == 'other':
                         className = eval(self.name)
                         kwargs = self.getKwargs()
                         opObj = className(**kwargs)
                     return opObj
             class ProcUnitConf():
                 id = None
                 name = None
                 datatype = None
                 inputId = None
                 parentId = None
                 opConfObjList = []
                 procUnitObj = None
                 opObjList = []
                 ELEMENTNAME = 'ProcUnit'
                 def __init__(self):
                     self.id = None
                     self.datatype = None
                     self.name = None
                     self.inputId = None
                     self.opConfObjList = []
                     self.procUnitObj = None
                     self.opObjDict = {}
                 def __getPriority(self):
                     return len(self.opConfObjList)+1
                 def __getNewId(self):
                     return int(self.id)*10 + len(self.opConfObjList) + 1
                 def getElementName(self):
                     return self.ELEMENTNAME
                 def getId(self):
                     return self.id
                 def updateId(self, new_id, parentId=parentId):
                     new_id = int(parentId)*10 + (int(self.id) % 10)
                     new_inputId = int(parentId)*10 + (int(self.inputId) % 10)
                     #If this proc unit has not inputs
                     if self.inputId == '0':
                         new_inputId = 0
                     n = 1
                     for opConfObj in self.opConfObjList:
                         idOp = str(int(new_id)*10 + n)
                         opConfObj.updateId(idOp)
                         n += 1
                     self.parentId = str(parentId)
                     self.id = str(new_id)
                     self.inputId = str(new_inputId)
                 def getInputId(self):
                     return self.inputId
                 def getOperationObjList(self):
                     return self.opConfObjList
                 def getOperationObj(self, name=None):
                     for opConfObj in self.opConfObjList:
                         if opConfObj.name != name:
                             continue
                         return opConfObj
                     return None
                 def getOpObjfromParamValue(self, value=None):
                     for opConfObj in self.opConfObjList:
                         if opConfObj.getParameterObjfromValue(parameterValue=value) != value:
                             continue
                         return opConfObj
                     return None
                 def getProcUnitObj(self):
                     return self.procUnitObj
                 def setup(self, id, name, datatype, inputId, parentId=None):
                     #Compatible with old signal chain version
                     if datatype==None and name==None:
                         raise ValueError, "datatype or name should be defined"
                     if name==None:
                         if 'Proc' in datatype:
                             name = datatype
                         else:
                             name = '%sProc' %(datatype)
                     if datatype==None:
                         datatype = name.replace('Proc','')
                     self.id = str(id)
                     self.name = name
                     self.datatype = datatype
                     self.inputId = inputId
                     self.parentId = parentId
                     self.opConfObjList = []
                     self.addOperation(name='run', optype='self')
                 def removeOperations(self):
                     for obj in self.opConfObjList:
                         del obj
                     self.opConfObjList = []
                     self.addOperation(name='run')
                 def addParameter(self, **kwargs):
                     '''
                     Add parameters to "run" operation
                     '''
                     opObj = self.opConfObjList[0]
                     opObj.addParameter(**kwargs)
                     return opObj
                 def addOperation(self, name, optype='self'):
                     id = self.__getNewId()
                     priority = self.__getPriority()
                     opConfObj = OperationConf()
                     opConfObj.setup(id, name=name, priority=priority, type=optype)
                     self.opConfObjList.append(opConfObj)
                     return opConfObj
                 def makeXml(self, projectElement):
                     procUnitElement = SubElement(projectElement, self.ELEMENTNAME)
                     procUnitElement.set('id', str(self.id))
                     procUnitElement.set('name', self.name)
                     procUnitElement.set('datatype', self.datatype)
                     procUnitElement.set('inputId', str(self.inputId))
                     for opConfObj in self.opConfObjList:
                         opConfObj.makeXml(procUnitElement)
                 def readXml(self, upElement):
                     self.id = upElement.get('id')
                     self.name = upElement.get('name')
                     self.datatype = upElement.get('datatype')
                     self.inputId = upElement.get('inputId')
                     if self.ELEMENTNAME == "ReadUnit":
                         self.datatype = self.datatype.replace("Reader", "")
                     if self.ELEMENTNAME == "ProcUnit":
                         self.datatype = self.datatype.replace("Proc", "")
                     if self.inputId == 'None':
                         self.inputId = '0'
                     self.opConfObjList = []
                     opElementList = upElement.iter(OperationConf().getElementName())
                     for opElement in opElementList:
                         opConfObj = OperationConf()
                         opConfObj.readXml(opElement)
                         self.opConfObjList.append(opConfObj)
                 def printattr(self):
                     print "%s[%s]: name = %s, datatype = %s, inputId = %s" %(self.ELEMENTNAME,
-                                                                         self.id,
+                                                                            self.id,
-                                                                         self.name,
+                                                                            self.name,
-                                                                         self.datatype,
+                                                                            self.datatype,
-                                                                         self.inputId)
+                                                                            self.inputId)
                     for opConfObj in self.opConfObjList:
                         opConfObj.printattr()
                 def getKwargs(self):
                     opObj = self.opConfObjList[0]
                     kwargs = opObj.getKwargs()
                     return kwargs
                 def createObjects(self, plotter_queue=None):
                     className = eval(self.name)
                     kwargs = self.getKwargs()
                     procUnitObj = className(**kwargs)
                     for opConfObj in self.opConfObjList:
                         if opConfObj.type=='self' and self.name=='run':
                             continue
                         elif opConfObj.type=='self':
                             procUnitObj.addOperationKwargs(opConfObj.id, **opConfObj.getKwargs())
                             continue
                         opObj = opConfObj.createObject(plotter_queue)
                         self.opObjDict[opConfObj.id] = opObj
                         procUnitObj.addOperation(opObj, opConfObj.id)
                     self.procUnitObj = procUnitObj
                     return procUnitObj
                 def run(self):
                     is_ok = False
                     for opConfObj in self.opConfObjList:
                         kwargs = {}
                         for parmConfObj in opConfObj.getParameterObjList():
                             if opConfObj.name == 'run' and parmConfObj.name == 'datatype':
                                 continue
                             kwargs[parmConfObj.name] = parmConfObj.getValue()
                         #ini = time.time()
                         #print "\tRunning the '%s' operation with %s" %(opConfObj.name, opConfObj.id)
                         sts = self.procUnitObj.call(opType = opConfObj.type,
                                                     opName = opConfObj.name,
                                                     opId = opConfObj.id,
+                                                    **kwargs)
-            #             total_time = time.time() - ini
+                        #             total_time = time.time() - ini
+                        #
-            #             if total_time > 0.002:
+                        #             if total_time > 0.002:
-            #                 print "%s::%s took %f seconds" %(self.name, opConfObj.name, total_time)
+                        #                 print "%s::%s took %f seconds" %(self.name, opConfObj.name, total_time)
                         is_ok = is_ok or sts
                     return is_ok
                 def close(self):
                     for opConfObj in self.opConfObjList:
                         if opConfObj.type == 'self':
                             continue
                         opObj = self.procUnitObj.getOperationObj(opConfObj.id)
                         opObj.close()
                     self.procUnitObj.close()
                     return
             class ReadUnitConf(ProcUnitConf):
                 path = None
                 startDate = None
                 endDate = None
                 startTime = None
                 endTime = None
                 ELEMENTNAME = 'ReadUnit'
                 def __init__(self):
                     self.id = None
                     self.datatype = None
                     self.name = None
                     self.inputId = None
                     self.parentId = None
                     self.opConfObjList = []
                     self.opObjList = []
                 def getElementName(self):
                     return self.ELEMENTNAME
                 def setup(self, id, name, datatype, path='', startDate="", endDate="", startTime="",
                           endTime="", parentId=None, queue=None, server=None, **kwargs):
                     #Compatible with old signal chain version
                     if datatype==None and name==None:
                         raise ValueError, "datatype or name should be defined"
                     if name==None:
                         if 'Reader' in datatype:
                             name = datatype
                         else:
                             name = '%sReader' %(datatype)
                     if datatype==None:
                         datatype = name.replace('Reader','')
                     self.id = id
                     self.name = name
                     self.datatype = datatype
                     if path != '':
                         self.path = os.path.abspath(path)
                     self.startDate = startDate
                     self.endDate = endDate
                     self.startTime = startTime
                     self.endTime = endTime
                     self.inputId = '0'
                     self.parentId = parentId
                     self.queue = queue
                     self.server = server
                     self.addRunOperation(**kwargs)
                 def update(self, datatype, path, startDate, endDate, startTime, endTime, parentId=None, name=None, **kwargs):
                     #Compatible with old signal chain version
                     if datatype==None and name==None:
                         raise ValueError, "datatype or name should be defined"
                     if name==None:
                         if 'Reader' in datatype:
                             name = datatype
                         else:
                             name = '%sReader' %(datatype)
                     if datatype==None:
                         datatype = name.replace('Reader','')
                     self.datatype = datatype
                     self.name = name
                     self.path = path
                     self.startDate = startDate
                     self.endDate = endDate
                     self.startTime = startTime
                     self.endTime = endTime
                     self.inputId = '0'
                     self.parentId = parentId
                     self.updateRunOperation(**kwargs)
                 def removeOperations(self):
                     for obj in self.opConfObjList:
                         del obj
                     self.opConfObjList = []
                 def addRunOperation(self, **kwargs):
                     opObj = self.addOperation(name = 'run', optype = 'self')
                     if self.server is None:
                         opObj.addParameter(name='datatype' , value=self.datatype, format='str')
                         opObj.addParameter(name='path'     , value=self.path, format='str')
                         opObj.addParameter(name='startDate' , value=self.startDate, format='date')
                         opObj.addParameter(name='endDate'   , value=self.endDate, format='date')
                         opObj.addParameter(name='startTime' , value=self.startTime, format='time')
                         opObj.addParameter(name='endTime'   , value=self.endTime, format='time')
                         opObj.addParameter(name='queue'   , value=self.queue, format='obj')
                         for key, value in kwargs.items():
                             opObj.addParameter(name=key, value=value, format=type(value).__name__)
                     else:
                         opObj.addParameter(name='server' , value=self.server, format='str')
                     return opObj
                 def updateRunOperation(self, **kwargs):
                     opObj = self.getOperationObj(name = 'run')
                     opObj.removeParameters()
                     opObj.addParameter(name='datatype' , value=self.datatype, format='str')
                     opObj.addParameter(name='path'     , value=self.path, format='str')
                     opObj.addParameter(name='startDate' , value=self.startDate, format='date')
                     opObj.addParameter(name='endDate'   , value=self.endDate, format='date')
                     opObj.addParameter(name='startTime' , value=self.startTime, format='time')
                     opObj.addParameter(name='endTime'   , value=self.endTime, format='time')
                     for key, value in kwargs.items():
                         opObj.addParameter(name=key, value=value, format=type(value).__name__)
                     return opObj
-            #     def makeXml(self, projectElement):
+                    #     def makeXml(self, projectElement):
+                    #
-            #         procUnitElement = SubElement(projectElement, self.ELEMENTNAME)
+                    #         procUnitElement = SubElement(projectElement, self.ELEMENTNAME)
-            #         procUnitElement.set('id', str(self.id))
+                    #         procUnitElement.set('id', str(self.id))
-            #         procUnitElement.set('name', self.name)
+                    #         procUnitElement.set('name', self.name)
-            #         procUnitElement.set('datatype', self.datatype)
+                    #         procUnitElement.set('datatype', self.datatype)
-            #         procUnitElement.set('inputId', str(self.inputId))
+                    #         procUnitElement.set('inputId', str(self.inputId))
+                    #
-            #         for opConfObj in self.opConfObjList:
+                    #         for opConfObj in self.opConfObjList:
-            #             opConfObj.makeXml(procUnitElement)
+                    #             opConfObj.makeXml(procUnitElement)
                 def readXml(self, upElement):
                     self.id = upElement.get('id')
                     self.name = upElement.get('name')
                     self.datatype = upElement.get('datatype')
                     self.inputId = upElement.get('inputId')
                     if self.ELEMENTNAME == "ReadUnit":
                         self.datatype = self.datatype.replace("Reader", "")
                     if self.inputId == 'None':
                         self.inputId = '0'
                     self.opConfObjList = []
                     opElementList = upElement.iter(OperationConf().getElementName())
                     for opElement in opElementList:
                         opConfObj = OperationConf()
                         opConfObj.readXml(opElement)
                         self.opConfObjList.append(opConfObj)
                         if opConfObj.name == 'run':
                             self.path = opConfObj.getParameterValue('path')
                             self.startDate = opConfObj.getParameterValue('startDate')
                             self.endDate = opConfObj.getParameterValue('endDate')
                             self.startTime = opConfObj.getParameterValue('startTime')
                             self.endTime = opConfObj.getParameterValue('endTime')
             class Project():
                 id = None
                 name = None
                 description = None
                 filename = None
                 procUnitConfObjDict = None
                 ELEMENTNAME = 'Project'
                 plotterQueue = None
                 def __init__(self, plotter_queue=None):
                     self.id = None
                     self.name = None
                     self.description = None
                     self.plotterQueue = plotter_queue
                     self.procUnitConfObjDict = {}
                 def __getNewId(self):
                     idList = self.procUnitConfObjDict.keys()
                     id = int(self.id)*10
                     while True:
                         id += 1
                         if str(id) in idList:
                             continue
                         break
                     return str(id)
                 def getElementName(self):
                     return self.ELEMENTNAME
                 def getId(self):
                     return self.id
                 def updateId(self, new_id):
                     self.id = str(new_id)
                     keyList = self.procUnitConfObjDict.keys()
                     keyList.sort()
                     n = 1
                     newProcUnitConfObjDict = {}
                     for procKey in keyList:
                         procUnitConfObj = self.procUnitConfObjDict[procKey]
                         idProcUnit = str(int(self.id)*10 + n)
                         procUnitConfObj.updateId(idProcUnit, parentId = self.id)
                         newProcUnitConfObjDict[idProcUnit] = procUnitConfObj
                         n += 1
                     self.procUnitConfObjDict = newProcUnitConfObjDict
                 def setup(self, id, name, description):
                     self.id = str(id)
                     self.name = name
                     self.description = description
                 def update(self, name, description):
                     self.name = name
                     self.description = description
                 def addReadUnit(self, id=None, datatype=None, name=None, **kwargs):
                     if id is None:
                         idReadUnit = self.__getNewId()
                     else:
                         idReadUnit = str(id)
                     readUnitConfObj = ReadUnitConf()
                     readUnitConfObj.setup(idReadUnit, name, datatype, parentId=self.id, **kwargs)
                     self.procUnitConfObjDict[readUnitConfObj.getId()] = readUnitConfObj
                     return readUnitConfObj
                 def addProcUnit(self, inputId='0', datatype=None, name=None):
                     idProcUnit = self.__getNewId()
                     procUnitConfObj = ProcUnitConf()
                     procUnitConfObj.setup(idProcUnit, name, datatype, inputId, parentId=self.id)
                     self.procUnitConfObjDict[procUnitConfObj.getId()] = procUnitConfObj
                     return procUnitConfObj
                 def removeProcUnit(self, id):
                     if id in self.procUnitConfObjDict.keys():
                         self.procUnitConfObjDict.pop(id)
                 def getReadUnitId(self):
                     readUnitConfObj = self.getReadUnitObj()
                     return readUnitConfObj.id
                 def getReadUnitObj(self):
                     for obj in self.procUnitConfObjDict.values():
                         if obj.getElementName() == "ReadUnit":
                             return obj
                     return None
                 def getProcUnitObj(self, id=None, name=None):
                     if id != None:
                         return self.procUnitConfObjDict[id]
                     if name != None:
                         return self.getProcUnitObjByName(name)
                     return None
                 def getProcUnitObjByName(self, name):
                     for obj in self.procUnitConfObjDict.values():
                         if obj.name == name:
                             return obj
                     return None
                 def procUnitItems(self):
                     return self.procUnitConfObjDict.items()
                 def makeXml(self):
                     projectElement = Element('Project')
                     projectElement.set('id', str(self.id))
                     projectElement.set('name', self.name)
                     projectElement.set('description', self.description)
                     for procUnitConfObj in self.procUnitConfObjDict.values():
                         procUnitConfObj.makeXml(projectElement)
                     self.projectElement = projectElement
                 def writeXml(self, filename=None):
                     if filename == None:
                         if self.filename:
                             filename = self.filename
                         else:
                             filename = "schain.xml"
                     if not filename:
                         print "filename has not been defined. Use setFilename(filename) for do it."
                         return 0
                     abs_file = os.path.abspath(filename)
                     if not os.access(os.path.dirname(abs_file), os.W_OK):
                         print "No write permission on %s" %os.path.dirname(abs_file)
                         return 0
                     if os.path.isfile(abs_file) and not(os.access(abs_file, os.W_OK)):
                         print "File %s already exists and it could not be overwriten" %abs_file
                         return 0
                     self.makeXml()
                     ElementTree(self.projectElement).write(abs_file, method='xml')
                     self.filename = abs_file
                     return 1
                 def readXml(self, filename = None):
                     if not filename:
                         print "filename is not defined"
                         return 0
                     abs_file = os.path.abspath(filename)
                     if not os.path.isfile(abs_file):
                         print "%s file does not exist" %abs_file
                         return 0
                     self.projectElement = None
                     self.procUnitConfObjDict = {}
                     try:
                         self.projectElement = ElementTree().parse(abs_file)
                     except:
                         print "Error reading %s, verify file format" %filename
                         return 0
                     self.project = self.projectElement.tag
                     self.id = self.projectElement.get('id')
                     self.name = self.projectElement.get('name')
                     self.description = self.projectElement.get('description')
                     readUnitElementList = self.projectElement.iter(ReadUnitConf().getElementName())
                     for readUnitElement in readUnitElementList:
                         readUnitConfObj = ReadUnitConf()
                         readUnitConfObj.readXml(readUnitElement)
                         if readUnitConfObj.parentId == None:
                             readUnitConfObj.parentId = self.id
                         self.procUnitConfObjDict[readUnitConfObj.getId()] = readUnitConfObj
                     procUnitElementList = self.projectElement.iter(ProcUnitConf().getElementName())
                     for procUnitElement in procUnitElementList:
                         procUnitConfObj = ProcUnitConf()
                         procUnitConfObj.readXml(procUnitElement)
                         if procUnitConfObj.parentId == None:
                             procUnitConfObj.parentId = self.id
                         self.procUnitConfObjDict[procUnitConfObj.getId()] = procUnitConfObj
                     self.filename = abs_file
                     return 1
                 def printattr(self):
                     print "Project[%s]: name = %s, description = %s" %(self.id,
-                                                                          self.name,
+                                                                        self.name,
-                                                                          self.description)
+                                                                        self.description)
                     for procUnitConfObj in self.procUnitConfObjDict.values():
                         procUnitConfObj.printattr()
                 def createObjects(self):
                     for procUnitConfObj in self.procUnitConfObjDict.values():
                         procUnitConfObj.createObjects(self.plotterQueue)
                 def __connect(self, objIN, thisObj):
                     thisObj.setInput(objIN.getOutputObj())
                 def connectObjects(self):
                     for thisPUConfObj in self.procUnitConfObjDict.values():
                         inputId = thisPUConfObj.getInputId()
                         if int(inputId) == 0:
                             continue
                         #Get input object
                         puConfINObj = self.procUnitConfObjDict[inputId]
                         puObjIN = puConfINObj.getProcUnitObj()
                         #Get current object
                         thisPUObj = thisPUConfObj.getProcUnitObj()
                         self.__connect(puObjIN, thisPUObj)
                 def __handleError(self, procUnitConfObj, send_email=True):
                     import socket
                     err = traceback.format_exception(sys.exc_info()[0],
-                                                     sys.exc_info()[1],
+                                                    sys.exc_info()[1],
-                                                     sys.exc_info()[2])
+                                                    sys.exc_info()[2])
                     print "***** Error occurred in %s *****" %(procUnitConfObj.name)
                     print "***** %s" %err[-1]
                     message = "".join(err)
                     sys.stderr.write(message)
                     if not send_email:
                         return
                     subject =  "SChain v%s: Error running %s\n" %(schainpy.__version__, procUnitConfObj.name)
                     subtitle = "%s: %s\n" %(procUnitConfObj.getElementName() ,procUnitConfObj.name)
                     subtitle += "Hostname: %s\n" %socket.gethostbyname(socket.gethostname())
                     subtitle += "Working directory: %s\n" %os.path.abspath("./")
                     subtitle += "Configuration file: %s\n" %self.filename
                     subtitle += "Time: %s\n" %str(datetime.datetime.now())
                     readUnitConfObj = self.getReadUnitObj()
                     if readUnitConfObj:
                         subtitle += "\nInput parameters:\n"
                         subtitle += "[Data path = %s]\n" %readUnitConfObj.path
                         subtitle += "[Data type = %s]\n" %readUnitConfObj.datatype
                         subtitle += "[Start date = %s]\n" %readUnitConfObj.startDate
                         subtitle += "[End date = %s]\n" %readUnitConfObj.endDate
                         subtitle += "[Start time = %s]\n" %readUnitConfObj.startTime
                         subtitle += "[End time = %s]\n" %readUnitConfObj.endTime
                     adminObj = schainpy.admin.SchainNotify()
                     adminObj.sendAlert(message=message,
-                                       subject=subject,
+                                        subject=subject,
-                                       subtitle=subtitle,
+                                        subtitle=subtitle,
-                                       filename=self.filename)
+                                        filename=self.filename)
                 def isPaused(self):
                     return 0
                 def isStopped(self):
                     return 0
                 def runController(self):
                     """
                     returns 0 when this process has been stopped, 1 otherwise
                     """
                     if self.isPaused():
                         print "Process suspended"
                         while True:
                             sleep(0.1)
                             if not self.isPaused():
                                 break
                             if self.isStopped():
                                 break
                         print "Process reinitialized"
                     if self.isStopped():
                         print "Process stopped"
                         return 0
                     return 1
                 def setFilename(self, filename):
                     self.filename = filename
                 def setPlotterQueue(self, plotter_queue):
                     raise NotImplementedError, "Use schainpy.controller_api.ControllerThread instead Project class"
                 def getPlotterQueue(self):
                     raise NotImplementedError, "Use schainpy.controller_api.ControllerThread instead Project class"
                 def useExternalPlotter(self):
                     raise NotImplementedError, "Use schainpy.controller_api.ControllerThread instead Project class"
                 def run(self):
                     print
                     print "*"*60
                     print "   Starting SIGNAL CHAIN PROCESSING v%s " %schainpy.__version__
                     print "*"*60
                     print
                     keyList = self.procUnitConfObjDict.keys()
                     keyList.sort()
                     while(True):
                         is_ok = False
                         for procKey in keyList:
             #                 print "Running the '%s' process with %s" %(procUnitConfObj.name, procUnitConfObj.id)
                             procUnitConfObj = self.procUnitConfObjDict[procKey]
                             try:
                                 sts = procUnitConfObj.run()
                                 is_ok = is_ok or sts
                             except KeyboardInterrupt:
                                 is_ok = False
                                 break
                             except ValueError, e:
                                 sleep(0.5)
                                 self.__handleError(procUnitConfObj, send_email=True)
                                 is_ok = False
                                 break
                             except:
                                 sleep(0.5)
                                 self.__handleError(procUnitConfObj)
                                 is_ok = False
                                 break
                         #If every process unit finished so end process
                         if not(is_ok):
             #                 print "Every process unit have finished"
                             break
                         if not self.runController():
                             break
                     #Closing every process
                     for procKey in keyList:
                         procUnitConfObj = self.procUnitConfObjDict[procKey]
                         procUnitConfObj.close()
                     print "Process finished"
                 def start(self, filename=None):
                     self.writeXml(filename)
                     self.createObjects()
                     self.connectObjects()
                     self.run()

schainpy/controller_api.py +3 -3

             import threading
             from Queue import Queue
             from schainpy.controller import Project
             from schainpy.model.graphics.jroplotter import PlotManager
             class ControllerThread(threading.Thread, Project):
                 def __init__(self, plotter_queue=None):
                     threading.Thread.__init__(self)
                     Project.__init__(self, plotter_queue)
                     self.setDaemon(True)
                     self.lock = threading.Lock()
-                    self.control = {'stop':False, 'pause':False}
+                    self.control = { 'stop':False, 'pause':False }
                 def __del__(self):
                     self.control['stop'] = True
                 def stop(self):
                     self.lock.acquire()
                     self.control['stop'] = True
                     self.lock.release()
                 def pause(self):
                     self.lock.acquire()
                     self.control['pause'] = not(self.control['pause'])
                     paused = self.control['pause']
                     self.lock.release()
                     return paused
                 def isPaused(self):
                     self.lock.acquire()
                     paused = self.control['pause']
                     self.lock.release()
                     return paused
                 def isStopped(self):
                     self.lock.acquire()
                     stopped = self.control['stop']
                     self.lock.release()
                     return stopped
                 def run(self):
                     self.control['stop'] = False
                     self.control['pause'] = False
                     self.writeXml()
                     self.createObjects()
                     self.connectObjects()
                     Project.run(self)
                 def isRunning(self):
                     return self.is_alive()
                 def isFinished(self):
                     return not self.is_alive()
                 def setPlotters(self):
                     plotterList = PlotManager.plotterList
                     for thisPUConfObj in self.procUnitConfObjDict.values():
                         inputId = thisPUConfObj.getInputId()
                         if int(inputId) == 0:
                             continue
                         for thisOpObj in thisPUConfObj.getOperationObjList():
                             if thisOpObj.type == "self":
                                 continue
                             if thisOpObj.name in plotterList:
-                                thisOpObj.type = "plotter"
+                                thisOpObj.type = "other"
                 def setPlotterQueue(self, plotter_queue):
                     self.plotterQueue = plotter_queue
                 def getPlotterQueue(self):
                     return self.plotterQueue
                 def useExternalPlotter(self):
                     self.plotterQueue = Queue(10)
                     self.setPlotters()
                     plotManagerObj = PlotManager(self.plotterQueue)
                     plotManagerObj.setController(self)
                     return plotManagerObj
             # from PyQt4 import QtCore
             # from PyQt4.QtCore import SIGNAL
             #
             # class ControllerQThread(QtCore.QThread, Project):
             #
             #     def __init__(self, filename):
             #
             #         QtCore.QThread.__init__(self)
             #         Project.__init__(self)
             #
             #         self.filename = filename
             #
             #         self.lock = threading.Lock()
             #         self.control = {'stop':False, 'pause':False}
             #
             #     def __del__(self):
             #
             #         self.control['stop'] = True
             #         self.wait()
             #
             #     def stop(self):
             #
             #         self.lock.acquire()
             #
             #         self.control['stop'] = True
             #
             #         self.lock.release()
             #
             #     def pause(self):
             #
             #         self.lock.acquire()
             #
             #         self.control['pause'] = not(self.control['pause'])
             #         paused = self.control['pause']
             #
             #         self.lock.release()
             #
             #         return paused
             #
             #     def isPaused(self):
             #
             #         self.lock.acquire()
             #         paused = self.control['pause']
             #         self.lock.release()
             #
             #         return paused
             #
             #     def isStopped(self):
             #
             #         self.lock.acquire()
             #         stopped = self.control['stop']
             #         self.lock.release()
             #
             #         return stopped
             #
             #     def run(self):
             #
             #         self.control['stop'] = False
             #         self.control['pause'] = False
             #
             #         self.readXml(self.filename)
             #         self.createObjects()
             #         self.connectObjects()
             #         self.emit( SIGNAL( "jobStarted( PyQt_PyObject )" ), 1)
             #         Project.run(self)
             #         self.emit( SIGNAL( "jobFinished( PyQt_PyObject )" ), 1)
             #

schainpy/model/data/jrodata.py 0 -2

             '''
             $Author: murco $
             $Id: JROData.py 173 2012-11-20 15:06:21Z murco $
             '''
             import copy
             import numpy
             import datetime
             from jroheaderIO import SystemHeader, RadarControllerHeader
             from schainpy import cSchain
             def getNumpyDtype(dataTypeCode):
                 if dataTypeCode == 0:
                     numpyDtype = numpy.dtype([('real','<i1'),('imag','<i1')])
                 elif dataTypeCode == 1:
                     numpyDtype = numpy.dtype([('real','<i2'),('imag','<i2')])
                 elif dataTypeCode == 2:
                     numpyDtype = numpy.dtype([('real','<i4'),('imag','<i4')])
                 elif dataTypeCode == 3:
                     numpyDtype = numpy.dtype([('real','<i8'),('imag','<i8')])
                 elif dataTypeCode == 4:
                     numpyDtype = numpy.dtype([('real','<f4'),('imag','<f4')])
                 elif dataTypeCode == 5:
                     numpyDtype = numpy.dtype([('real','<f8'),('imag','<f8')])
                 else:
                     raise ValueError, 'dataTypeCode was not defined'
                 return numpyDtype
             def getDataTypeCode(numpyDtype):
                 if numpyDtype == numpy.dtype([('real','<i1'),('imag','<i1')]):
                     datatype = 0
                 elif numpyDtype == numpy.dtype([('real','<i2'),('imag','<i2')]):
                     datatype = 1
                 elif numpyDtype == numpy.dtype([('real','<i4'),('imag','<i4')]):
                     datatype = 2
                 elif numpyDtype == numpy.dtype([('real','<i8'),('imag','<i8')]):
                     datatype = 3
                 elif numpyDtype == numpy.dtype([('real','<f4'),('imag','<f4')]):
                     datatype = 4
                 elif numpyDtype == numpy.dtype([('real','<f8'),('imag','<f8')]):
                     datatype = 5
                 else:
                     datatype = None
                 return datatype
             def hildebrand_sekhon(data, navg):
                 """
                 This method is for the objective determination of the noise level in Doppler spectra. This
                 implementation technique is based on the fact that the standard deviation of the spectral
                 densities is equal to the mean spectral density for white Gaussian noise
                 Inputs:
                     Data    :    heights
                     navg    :    numbers of averages
                 Return:
                     -1        :    any error
                     anoise    :    noise's level
                 """
                 sortdata = numpy.sort(data, axis=None)
             #     lenOfData = len(sortdata)
             #     nums_min = lenOfData*0.2
             #
             #     if nums_min <= 5:
             #         nums_min = 5
             #
             #     sump = 0.
             #
             #     sumq = 0.
             #
             #     j = 0
             #
             #     cont = 1
             #
             #     while((cont==1)and(j<lenOfData)):
             #
             #         sump += sortdata[j]
             #
             #         sumq += sortdata[j]**2
             #
             #         if j > nums_min:
             #             rtest = float(j)/(j-1) + 1.0/navg
             #             if ((sumq*j) > (rtest*sump**2)):
             #                 j = j - 1
             #                 sump  = sump - sortdata[j]
             #                 sumq =  sumq - sortdata[j]**2
             #                 cont = 0
             #
             #         j += 1
             #
             #     lnoise = sump /j
             #
             #     return lnoise
                 return cSchain.hildebrand_sekhon(sortdata, navg)
             class Beam:
                 def __init__(self):
                     self.codeList = []
                     self.azimuthList = []
                     self.zenithList = []
             class GenericData(object):
                 flagNoData = True
                 def copy(self, inputObj=None):
                     if inputObj == None:
                         return copy.deepcopy(self)
                     for key in inputObj.__dict__.keys():
                         attribute = inputObj.__dict__[key]
                         #If this attribute is a tuple or list
                         if type(inputObj.__dict__[key]) in (tuple, list):
                             self.__dict__[key] = attribute[:]
                             continue
                         #If this attribute is another object or instance
                         if hasattr(attribute, '__dict__'):
                             self.__dict__[key] = attribute.copy()
                             continue
                         self.__dict__[key] = inputObj.__dict__[key]
                 def deepcopy(self):
                     return copy.deepcopy(self)
                 def isEmpty(self):
                     return self.flagNoData
             class JROData(GenericData):
             #    m_BasicHeader = BasicHeader()
             #    m_ProcessingHeader = ProcessingHeader()
                 systemHeaderObj = SystemHeader()
                 radarControllerHeaderObj = RadarControllerHeader()
             #    data = None
                 type = None
                 datatype = None #dtype but in string
             #     dtype = None
             #    nChannels = None
             #    nHeights = None
                 nProfiles = None
                 heightList = None
                 channelList = None
                 flagDiscontinuousBlock = False
                 useLocalTime = False
                 utctime = None
                 timeZone = None
                 dstFlag = None
                 errorCount = None
                 blocksize = None
             #     nCode = None
             #
             #     nBaud = None
             #
             #     code = None
                 flagDecodeData = False #asumo q la data no esta decodificada
                 flagDeflipData = False #asumo q la data no esta sin flip
                 flagShiftFFT = False
             #     ippSeconds = None
             #     timeInterval = None
                 nCohInt = None
             #     noise = None
                 windowOfFilter = 1
                 #Speed of ligth
                 C = 3e8
                 frequency = 49.92e6
                 realtime = False
                 beacon_heiIndexList = None
                 last_block = None
                 blocknow = None
                 azimuth = None
                 zenith = None
                 beam = Beam()
                 profileIndex = None
                 def getNoise(self):
                     raise NotImplementedError
                 def getNChannels(self):
                     return len(self.channelList)
                 def getChannelIndexList(self):
                     return range(self.nChannels)
                 def getNHeights(self):
                     return len(self.heightList)
                 def getHeiRange(self, extrapoints=0):
                     heis = self.heightList
             #        deltah = self.heightList[1] - self.heightList[0]
             #
             #        heis.append(self.heightList[-1])
                     return heis
                 def getDeltaH(self):
                     delta = self.heightList[1] - self.heightList[0]
                     return delta
                 def getltctime(self):
                     if self.useLocalTime:
                         return self.utctime - self.timeZone*60
                     return self.utctime
                 def getDatatime(self):
                     datatimeValue = datetime.datetime.utcfromtimestamp(self.ltctime)
                     return datatimeValue
                 def getTimeRange(self):
                     datatime = []
                     datatime.append(self.ltctime)
                     datatime.append(self.ltctime + self.timeInterval+1)
                     datatime = numpy.array(datatime)
                     return datatime
                 def getFmaxTimeResponse(self):
                     period = (10**-6)*self.getDeltaH()/(0.15)
                     PRF = 1./(period * self.nCohInt)
                     fmax = PRF
                     return fmax
                 def getFmax(self):
                     PRF = 1./(self.ippSeconds * self.nCohInt)
                     fmax = PRF
                     return fmax
                 def getVmax(self):
                     _lambda = self.C/self.frequency
                     vmax = self.getFmax() * _lambda/2
                     return vmax
                 def get_ippSeconds(self):
                     '''
                     '''
                     return self.radarControllerHeaderObj.ippSeconds
                 def set_ippSeconds(self, ippSeconds):
                     '''
                     '''
                     self.radarControllerHeaderObj.ippSeconds = ippSeconds
                     return
                 def get_dtype(self):
                     '''
                     '''
                     return getNumpyDtype(self.datatype)
                 def set_dtype(self, numpyDtype):
                     '''
                     '''
                     self.datatype = getDataTypeCode(numpyDtype)
                 def get_code(self):
                     '''
                     '''
                     return self.radarControllerHeaderObj.code
                 def set_code(self, code):
                     '''
                     '''
                     self.radarControllerHeaderObj.code = code
                     return
                 def get_ncode(self):
                     '''
                     '''
                     return self.radarControllerHeaderObj.nCode
                 def set_ncode(self, nCode):
                     '''
                     '''
                     self.radarControllerHeaderObj.nCode = nCode
                     return
                 def get_nbaud(self):
                     '''
                     '''
                     return self.radarControllerHeaderObj.nBaud
                 def set_nbaud(self, nBaud):
                     '''
                     '''
                     self.radarControllerHeaderObj.nBaud = nBaud
                     return
                 nChannels = property(getNChannels, "I'm the 'nChannel' property.")
                 channelIndexList = property(getChannelIndexList, "I'm the 'channelIndexList' property.")
                 nHeights = property(getNHeights, "I'm the 'nHeights' property.")
                 #noise = property(getNoise, "I'm the 'nHeights' property.")
                 datatime = property(getDatatime, "I'm the 'datatime' property")
                 ltctime = property(getltctime, "I'm the 'ltctime' property")
                 ippSeconds = property(get_ippSeconds, set_ippSeconds)
                 dtype = property(get_dtype, set_dtype)
             #     timeInterval = property(getTimeInterval, "I'm the 'timeInterval' property")
                 code = property(get_code, set_code)
                 nCode = property(get_ncode, set_ncode)
                 nBaud = property(get_nbaud, set_nbaud)
             class Voltage(JROData):
                 #data es un numpy array de 2 dmensiones (canales, alturas)
                 data = None
                 def __init__(self):
                     '''
                     Constructor
                     '''
                     self.useLocalTime = True
                     self.radarControllerHeaderObj = RadarControllerHeader()
                     self.systemHeaderObj = SystemHeader()
                     self.type = "Voltage"
                     self.data = None
             #         self.dtype = None
             #        self.nChannels = 0
             #        self.nHeights = 0
                     self.nProfiles = None
                     self.heightList = None
                     self.channelList = None
             #        self.channelIndexList = None
                     self.flagNoData = True
                     self.flagDiscontinuousBlock = False
                     self.utctime = None
                     self.timeZone = None
                     self.dstFlag = None
                     self.errorCount = None
                     self.nCohInt = None
                     self.blocksize = None
                     self.flagDecodeData = False #asumo q la data no esta decodificada
                     self.flagDeflipData = False #asumo q la data no esta sin flip
                     self.flagShiftFFT = False
                     self.flagDataAsBlock = False    #Asumo que la data es leida perfil a perfil
                     self.profileIndex = 0
                 def getNoisebyHildebrand(self, channel = None):
                     """
                     Determino el nivel de ruido usando el metodo Hildebrand-Sekhon
                     Return:
                         noiselevel
                     """
                     if channel != None:
                         data = self.data[channel]
                         nChannels = 1
                     else:
                         data = self.data
                         nChannels = self.nChannels
                     noise = numpy.zeros(nChannels)
                     power = data * numpy.conjugate(data)
                     for thisChannel in range(nChannels):
                         if nChannels == 1:
                             daux = power[:].real
                         else:
                             daux = power[thisChannel,:].real
                         noise[thisChannel] = hildebrand_sekhon(daux, self.nCohInt)
                     return noise
                 def getNoise(self, type = 1, channel = None):
                     if type == 1:
                         noise = self.getNoisebyHildebrand(channel)
                     return noise
                 def getPower(self, channel = None):
                     if channel != None:
                         data = self.data[channel]
                     else:
                         data = self.data
                     power = data * numpy.conjugate(data)
                     powerdB = 10*numpy.log10(power.real)
                     powerdB = numpy.squeeze(powerdB)
                     return powerdB
                 def getTimeInterval(self):
                     timeInterval = self.ippSeconds * self.nCohInt
                     return timeInterval
                 noise = property(getNoise, "I'm the 'nHeights' property.")
                 timeInterval = property(getTimeInterval, "I'm the 'timeInterval' property")
             class Spectra(JROData):
                 #data spc es un numpy array de 2 dmensiones (canales, perfiles, alturas)
                 data_spc = None
                 #data cspc es un numpy array de 2 dmensiones (canales, pares, alturas)
                 data_cspc = None
                 #data dc es un numpy array de 2 dmensiones (canales, alturas)
                 data_dc = None
                 #data power
                 data_pwr = None
                 nFFTPoints = None
             #     nPairs = None
                 pairsList = None
                 nIncohInt = None
                 wavelength = None #Necesario para cacular el rango de velocidad desde la frecuencia
                 nCohInt = None #se requiere para determinar el valor de timeInterval
                 ippFactor = None
                 profileIndex = 0
                 plotting = "spectra"
                 def __init__(self):
                     '''
                     Constructor
                     '''
                     self.useLocalTime = True
                     self.radarControllerHeaderObj = RadarControllerHeader()
                     self.systemHeaderObj = SystemHeader()
                     self.type = "Spectra"
             #        self.data = None
             #         self.dtype = None
             #        self.nChannels = 0
             #        self.nHeights = 0
                     self.nProfiles = None
                     self.heightList = None
                     self.channelList = None
             #        self.channelIndexList = None
                     self.pairsList = None
                     self.flagNoData = True
                     self.flagDiscontinuousBlock = False
                     self.utctime = None
                     self.nCohInt = None
                     self.nIncohInt = None
                     self.blocksize = None
                     self.nFFTPoints = None
                     self.wavelength = None
                     self.flagDecodeData = False #asumo q la data no esta decodificada
                     self.flagDeflipData = False #asumo q la data no esta sin flip
                     self.flagShiftFFT = False
                     self.ippFactor = 1
                     #self.noise = None
                     self.beacon_heiIndexList = []
                     self.noise_estimation = None
                 def getNoisebyHildebrand(self, xmin_index=None, xmax_index=None, ymin_index=None, ymax_index=None):
                     """
                     Determino el nivel de ruido usando el metodo Hildebrand-Sekhon
                     Return:
                         noiselevel
                     """
                     noise = numpy.zeros(self.nChannels)
                     for channel in range(self.nChannels):
                         daux = self.data_spc[channel,xmin_index:xmax_index,ymin_index:ymax_index]
                         noise[channel] = hildebrand_sekhon(daux, self.nIncohInt)
                     return noise
                 def getNoise(self, xmin_index=None, xmax_index=None, ymin_index=None, ymax_index=None):
                     if self.noise_estimation is not None:
                         return self.noise_estimation #this was estimated by getNoise Operation defined in jroproc_spectra.py
                     else:
                         noise = self.getNoisebyHildebrand(xmin_index, xmax_index, ymin_index, ymax_index)
                         return noise
                 def getFreqRangeTimeResponse(self, extrapoints=0):
                     deltafreq = self.getFmaxTimeResponse() / (self.nFFTPoints*self.ippFactor)
                     freqrange = deltafreq*(numpy.arange(self.nFFTPoints+extrapoints)-self.nFFTPoints/2.) - deltafreq/2
                     return freqrange
                 def getAcfRange(self, extrapoints=0):
                     deltafreq = 10./(self.getFmax() / (self.nFFTPoints*self.ippFactor))
                     freqrange = deltafreq*(numpy.arange(self.nFFTPoints+extrapoints)-self.nFFTPoints/2.) - deltafreq/2
                     return freqrange
                 def getFreqRange(self, extrapoints=0):
                     deltafreq = self.getFmax() / (self.nFFTPoints*self.ippFactor)
                     freqrange = deltafreq*(numpy.arange(self.nFFTPoints+extrapoints)-self.nFFTPoints/2.) - deltafreq/2
                     return freqrange
                 def getVelRange(self, extrapoints=0):
                     deltav = self.getVmax() / (self.nFFTPoints*self.ippFactor)
                     velrange = deltav*(numpy.arange(self.nFFTPoints+extrapoints)-self.nFFTPoints/2.) #- deltav/2
                     return velrange
                 def getNPairs(self):
                     return len(self.pairsList)
                 def getPairsIndexList(self):
                     return range(self.nPairs)
                 def getNormFactor(self):
                     pwcode = 1
                     if self.flagDecodeData:
                         pwcode = numpy.sum(self.code[0]**2)
                     #normFactor = min(self.nFFTPoints,self.nProfiles)*self.nIncohInt*self.nCohInt*pwcode*self.windowOfFilter
                     normFactor = self.nProfiles*self.nIncohInt*self.nCohInt*pwcode*self.windowOfFilter
                     return normFactor
                 def getFlagCspc(self):
                     if self.data_cspc is None:
                         return True
                     return False
                 def getFlagDc(self):
                     if self.data_dc is None:
                         return True
                     return False
                 def getTimeInterval(self):
                     timeInterval = self.ippSeconds * self.nCohInt * self.nIncohInt * self.nProfiles
                     return timeInterval
                 def getPower(self):
                     factor = self.normFactor
                     z = self.data_spc/factor
                     z = numpy.where(numpy.isfinite(z), z, numpy.NAN)
                     avg = numpy.average(z, axis=1)
                     return 10*numpy.log10(avg)
                 def getCoherence(self, pairsList=None, phase=False):
                     z = []
                     if pairsList is None:
                         pairsIndexList = self.pairsIndexList
                     else:
                         pairsIndexList = []
                         for pair in pairsList:
                             if pair not in self.pairsList:
                                 raise ValueError, "Pair %s is not in dataOut.pairsList" %(pair)
                             pairsIndexList.append(self.pairsList.index(pair))
                     for i in range(len(pairsIndexList)):
                         pair = self.pairsList[pairsIndexList[i]]
                         ccf = numpy.average(self.data_cspc[pairsIndexList[i], :, :], axis=0)
                         powa = numpy.average(self.data_spc[pair[0], :, :], axis=0)
                         powb = numpy.average(self.data_spc[pair[1], :, :], axis=0)
                         avgcoherenceComplex = ccf/numpy.sqrt(powa*powb)
                         if phase:
                             data = numpy.arctan2(avgcoherenceComplex.imag,
                                                  avgcoherenceComplex.real)*180/numpy.pi
                         else:
                             data = numpy.abs(avgcoherenceComplex)
                         z.append(data)
                     return numpy.array(z)
                 def setValue(self, value):
                     print "This property should not be initialized"
                     return
                 nPairs = property(getNPairs, setValue, "I'm the 'nPairs' property.")
                 pairsIndexList = property(getPairsIndexList, setValue, "I'm the 'pairsIndexList' property.")
                 normFactor = property(getNormFactor, setValue, "I'm the 'getNormFactor' property.")
                 flag_cspc = property(getFlagCspc, setValue)
                 flag_dc = property(getFlagDc, setValue)
                 noise = property(getNoise, setValue, "I'm the 'nHeights' property.")
                 timeInterval = property(getTimeInterval, setValue, "I'm the 'timeInterval' property")
             class SpectraHeis(Spectra):
                 data_spc = None
                 data_cspc = None
                 data_dc = None
                 nFFTPoints = None
             #     nPairs = None
                 pairsList = None
                 nCohInt = None
                 nIncohInt = None
                 def __init__(self):
                     self.radarControllerHeaderObj = RadarControllerHeader()
                     self.systemHeaderObj = SystemHeader()
                     self.type = "SpectraHeis"
             #         self.dtype = None
             #        self.nChannels = 0
             #        self.nHeights = 0
                     self.nProfiles = None
                     self.heightList = None
                     self.channelList = None
             #        self.channelIndexList = None
                     self.flagNoData = True
                     self.flagDiscontinuousBlock = False
             #         self.nPairs = 0
                     self.utctime = None
                     self.blocksize = None
                     self.profileIndex = 0
                     self.nCohInt = 1
                     self.nIncohInt = 1
                 def getNormFactor(self):
                     pwcode = 1
                     if self.flagDecodeData:
                         pwcode = numpy.sum(self.code[0]**2)
                     normFactor = self.nIncohInt*self.nCohInt*pwcode
                     return normFactor
                 def getTimeInterval(self):
                     timeInterval = self.ippSeconds * self.nCohInt * self.nIncohInt
                     return timeInterval
                 normFactor = property(getNormFactor, "I'm the 'getNormFactor' property.")
                 timeInterval = property(getTimeInterval, "I'm the 'timeInterval' property")
             class Fits(JROData):
                 heightList = None
                 channelList = None
                 flagNoData = True
                 flagDiscontinuousBlock = False
                 useLocalTime = False
                 utctime = None
                 timeZone = None
             #     ippSeconds = None
             #     timeInterval = None
                 nCohInt = None
                 nIncohInt = None
                 noise = None
                 windowOfFilter = 1
                 #Speed of ligth
                 C = 3e8
                 frequency = 49.92e6
                 realtime = False
                 def __init__(self):
                     self.type = "Fits"
                     self.nProfiles = None
                     self.heightList = None
                     self.channelList = None
             #         self.channelIndexList = None
                     self.flagNoData = True
                     self.utctime = None
                     self.nCohInt = 1
                     self.nIncohInt = 1
                     self.useLocalTime = True
                     self.profileIndex = 0
             #         self.utctime = None
             #         self.timeZone = None
             #         self.ltctime = None
             #         self.timeInterval = None
             #         self.header = None
             #         self.data_header = None
             #         self.data = None
             #         self.datatime = None
             #         self.flagNoData = False
             #         self.expName = ''
             #         self.nChannels = None
             #         self.nSamples = None
             #         self.dataBlocksPerFile = None
             #         self.comments = ''
             #
                 def getltctime(self):
                     if self.useLocalTime:
                         return self.utctime - self.timeZone*60
                     return self.utctime
                 def getDatatime(self):
                     datatime = datetime.datetime.utcfromtimestamp(self.ltctime)
                     return datatime
                 def getTimeRange(self):
                     datatime = []
                     datatime.append(self.ltctime)
                     datatime.append(self.ltctime + self.timeInterval)
                     datatime = numpy.array(datatime)
                     return datatime
                 def getHeiRange(self):
                     heis = self.heightList
                     return heis
                 def getNHeights(self):
                     return len(self.heightList)
                 def getNChannels(self):
                     return len(self.channelList)
                 def getChannelIndexList(self):
                     return range(self.nChannels)
                 def getNoise(self, type = 1):
                     #noise = numpy.zeros(self.nChannels)
                     if type == 1:
                         noise = self.getNoisebyHildebrand()
                     if type == 2:
                         noise = self.getNoisebySort()
                     if type == 3:
                         noise = self.getNoisebyWindow()
                     return noise
                 def getTimeInterval(self):
                     timeInterval = self.ippSeconds * self.nCohInt * self.nIncohInt
                     return timeInterval
                 datatime = property(getDatatime, "I'm the 'datatime' property")
                 nHeights = property(getNHeights, "I'm the 'nHeights' property.")
                 nChannels = property(getNChannels, "I'm the 'nChannel' property.")
                 channelIndexList = property(getChannelIndexList, "I'm the 'channelIndexList' property.")
                 noise = property(getNoise, "I'm the 'nHeights' property.")
                 ltctime = property(getltctime, "I'm the 'ltctime' property")
                 timeInterval = property(getTimeInterval, "I'm the 'timeInterval' property")
             class Correlation(JROData):
                 noise = None
                 SNR = None
                 #--------------------------------------------------
                 mode = None
                 split = False
                 data_cf = None
                 lags = None
                 lagRange = None
                 pairsList = None
                 normFactor = None
                 #--------------------------------------------------
             #     calculateVelocity = None
                 nLags = None
                 nPairs = None
                 nAvg = None
                 def __init__(self):
                     '''
                     Constructor
                     '''
                     self.radarControllerHeaderObj = RadarControllerHeader()
                     self.systemHeaderObj = SystemHeader()
                     self.type = "Correlation"
                     self.data = None
                     self.dtype = None
                     self.nProfiles = None
                     self.heightList = None
                     self.channelList = None
                     self.flagNoData = True
                     self.flagDiscontinuousBlock = False
                     self.utctime = None
                     self.timeZone = None
                     self.dstFlag = None
                     self.errorCount = None
                     self.blocksize = None
                     self.flagDecodeData = False #asumo q la data no esta decodificada
                     self.flagDeflipData = False #asumo q la data no esta sin flip
                     self.pairsList = None
                     self.nPoints = None
                 def getPairsList(self):
                     return self.pairsList
                 def getNoise(self, mode = 2):
                     indR = numpy.where(self.lagR == 0)[0][0]
                     indT = numpy.where(self.lagT == 0)[0][0]
                     jspectra0 = self.data_corr[:,:,indR,:]
                     jspectra = copy.copy(jspectra0)
                     num_chan = jspectra.shape[0]
                     num_hei = jspectra.shape[2]
                     freq_dc = jspectra.shape[1]/2
                     ind_vel = numpy.array([-2,-1,1,2]) + freq_dc
                     if ind_vel[0]<0:
                         ind_vel[range(0,1)] = ind_vel[range(0,1)] + self.num_prof
                     if mode == 1:
                         jspectra[:,freq_dc,:] = (jspectra[:,ind_vel[1],:] + jspectra[:,ind_vel[2],:])/2 #CORRECCION
                     if mode == 2:
                         vel = numpy.array([-2,-1,1,2])
                         xx = numpy.zeros([4,4])
                         for fil in range(4):
                             xx[fil,:] = vel[fil]**numpy.asarray(range(4))
                         xx_inv = numpy.linalg.inv(xx)
                         xx_aux = xx_inv[0,:]
                         for ich in range(num_chan):
                             yy = jspectra[ich,ind_vel,:]
                             jspectra[ich,freq_dc,:] = numpy.dot(xx_aux,yy)
                             junkid = jspectra[ich,freq_dc,:]<=0
                             cjunkid = sum(junkid)
                             if cjunkid.any():
                                 jspectra[ich,freq_dc,junkid.nonzero()] = (jspectra[ich,ind_vel[1],junkid] + jspectra[ich,ind_vel[2],junkid])/2
                     noise = jspectra0[:,freq_dc,:] - jspectra[:,freq_dc,:]
                     return noise
                 def getTimeInterval(self):
                     timeInterval = self.ippSeconds * self.nCohInt * self.nProfiles
                     return timeInterval
                 def splitFunctions(self):
                     pairsList = self.pairsList
                     ccf_pairs = []
                     acf_pairs = []
                     ccf_ind = []
                     acf_ind = []
                     for l in range(len(pairsList)):
                         chan0 = pairsList[l][0]
                         chan1 = pairsList[l][1]
                         #Obteniendo pares de Autocorrelacion
                         if chan0 == chan1:
                             acf_pairs.append(chan0)
                             acf_ind.append(l)
                         else:
                             ccf_pairs.append(pairsList[l])
                             ccf_ind.append(l)
                     data_acf = self.data_cf[acf_ind]
                     data_ccf = self.data_cf[ccf_ind]
                     return acf_ind, ccf_ind, acf_pairs, ccf_pairs, data_acf, data_ccf
                 def getNormFactor(self):
                     acf_ind, ccf_ind, acf_pairs, ccf_pairs, data_acf, data_ccf = self.splitFunctions()
                     acf_pairs = numpy.array(acf_pairs)
                     normFactor = numpy.zeros((self.nPairs,self.nHeights))
                     for p in range(self.nPairs):
                         pair = self.pairsList[p]
                         ch0 = pair[0]
                         ch1 = pair[1]
                         ch0_max = numpy.max(data_acf[acf_pairs==ch0,:,:], axis=1)
                         ch1_max = numpy.max(data_acf[acf_pairs==ch1,:,:], axis=1)
                         normFactor[p,:] = numpy.sqrt(ch0_max*ch1_max)
                     return normFactor
                 timeInterval = property(getTimeInterval, "I'm the 'timeInterval' property")
                 normFactor = property(getNormFactor, "I'm the 'normFactor property'")
             class Parameters(Spectra):
                 experimentInfo = None   #Information about the experiment
                 #Information from previous data
                 inputUnit = None        #Type of data to be processed
                 operation = None        #Type of operation to parametrize
                 #normFactor = None       #Normalization Factor
                 groupList = None        #List of Pairs, Groups, etc
                 #Parameters
                 data_param = None       #Parameters obtained
                 data_pre = None         #Data Pre Parametrization
                 data_SNR = None         #Signal to Noise Ratio
             #     heightRange = None      #Heights
                 abscissaList = None    #Abscissa, can be velocities, lags or time
             #    noise = None            #Noise Potency
                 utctimeInit = None      #Initial UTC time
                 paramInterval = None    #Time interval to calculate Parameters in seconds
                 useLocalTime = True
                 #Fitting
                 data_error = None       #Error of the estimation
                 constants = None
                 library = None
                 #Output signal
                 outputInterval = None     #Time interval to calculate output signal in seconds
                 data_output = None       #Out signal
                 nAvg = None
                 noise_estimation = None
                 def __init__(self):
                     '''
                     Constructor
                     '''
                     self.radarControllerHeaderObj = RadarControllerHeader()
                     self.systemHeaderObj = SystemHeader()
                     self.type = "Parameters"
                 def getTimeRange1(self, interval):
                     datatime = []
                     if self.useLocalTime:
                         time1 = self.utctimeInit - self.timeZone*60
                     else:
                         time1 = self.utctimeInit
                     datatime.append(time1)
                     datatime.append(time1 + interval)
                     datatime = numpy.array(datatime)
                     return datatime
                 def getTimeInterval(self):
                     if hasattr(self, 'timeInterval1'):
                         return self.timeInterval1
                     else:
                         return self.paramInterval
                 def getNoise(self):
                     return self.spc_noise
                 timeInterval = property(getTimeInterval)

schainpy/model/graphics/jroplot_spectra.py +2 -2

             '''
             Created on Jul 9, 2014
             @author: roj-idl71
             '''
             import os
             import datetime
             import numpy
             from figure import Figure, isRealtime, isTimeInHourRange
             from plotting_codes import *
             class SpectraPlot(Figure):
                 isConfig = None
                 __nsubplots = None
                 WIDTHPROF = None
                 HEIGHTPROF = None
                 PREFIX = 'spc'
                 def __init__(self, **kwargs):
                     Figure.__init__(self, **kwargs)
                     self.isConfig = False
                     self.__nsubplots = 1
-                    self.WIDTH = 250
+                    self.WIDTH = 1000
-                    self.HEIGHT = 250
+                    self.HEIGHT = 1000
                     self.WIDTHPROF = 120
                     self.HEIGHTPROF = 0
                     self.counter_imagwr = 0
                     self.PLOT_CODE = SPEC_CODE
                     self.FTP_WEI = None
                     self.EXP_CODE = None
                     self.SUB_EXP_CODE = None
                     self.PLOT_POS = None
                     self.__xfilter_ena = False
                     self.__yfilter_ena = False
                 def getSubplots(self):
                     ncol = int(numpy.sqrt(self.nplots)+0.9)
                     nrow = int(self.nplots*1./ncol + 0.9)
                     return nrow, ncol
                 def setup(self, id, nplots, wintitle, showprofile=True, show=True):
                     self.__showprofile = showprofile
                     self.nplots = nplots
                     ncolspan = 1
                     colspan = 1
                     if showprofile:
                         ncolspan = 3
                         colspan = 2
                         self.__nsubplots = 2
                     self.createFigure(id = id,
                                       wintitle = wintitle,
                                       widthplot = self.WIDTH + self.WIDTHPROF,
                                       heightplot = self.HEIGHT + self.HEIGHTPROF,
                                       show=show)
                     nrow, ncol = self.getSubplots()
                     counter = 0
                     for y in range(nrow):
                         for x in range(ncol):
                             if counter >= self.nplots:
                                 break
                             self.addAxes(nrow, ncol*ncolspan, y, x*ncolspan, colspan, 1)
                             if showprofile:
                                 self.addAxes(nrow, ncol*ncolspan, y, x*ncolspan+colspan, 1, 1)
                             counter += 1
                 def run(self, dataOut, id, wintitle="", channelList=None, showprofile=True,
                         xmin=None, xmax=None, ymin=None, ymax=None, zmin=None, zmax=None,
                         save=False, figpath='./', figfile=None, show=True, ftp=False, wr_period=1,
                         server=None, folder=None, username=None, password=None,
                         ftp_wei=0, exp_code=0, sub_exp_code=0, plot_pos=0, realtime=False,
                         xaxis="velocity", **kwargs):
                     """
                     Input:
                         dataOut         :
                         id        :
                         wintitle        :
                         channelList     :
                         showProfile     :
                         xmin            :    None,
                         xmax            :    None,
                         ymin            :    None,
                         ymax            :    None,
                         zmin            :    None,
                         zmax            :    None
                     """
                     colormap = kwargs.get('colormap','jet')
                     if realtime:
                         if not(isRealtime(utcdatatime = dataOut.utctime)):
                             print 'Skipping this plot function'
                             return
                     if channelList == None:
                         channelIndexList = dataOut.channelIndexList
                     else:
                         channelIndexList = []
                         for channel in channelList:
                             if channel not in dataOut.channelList:
                                 raise ValueError, "Channel %d is not in dataOut.channelList" %channel
                             channelIndexList.append(dataOut.channelList.index(channel))
                     factor = dataOut.normFactor
                     if xaxis == "frequency":
                         x = dataOut.getFreqRange(1)/1000.
                         xlabel = "Frequency (kHz)"
                     elif xaxis == "time":
                         x = dataOut.getAcfRange(1)
                         xlabel = "Time (ms)"
                     else:
                         x = dataOut.getVelRange(1)
                         xlabel = "Velocity (m/s)"
                     ylabel = "Range (Km)"
                     y = dataOut.getHeiRange()
                     z = dataOut.data_spc/factor
                     z = numpy.where(numpy.isfinite(z), z, numpy.NAN)
                     zdB = 10*numpy.log10(z)
                     avg = numpy.average(z, axis=1)
                     avgdB = 10*numpy.log10(avg)
                     noise = dataOut.getNoise()/factor
                     noisedB = 10*numpy.log10(noise)
                     thisDatetime = datetime.datetime.utcfromtimestamp(dataOut.getTimeRange()[0])
                     title = wintitle + " Spectra"
                     if ((dataOut.azimuth!=None) and (dataOut.zenith!=None)):
                         title = title + '_' + 'azimuth,zenith=%2.2f,%2.2f'%(dataOut.azimuth, dataOut.zenith)
                     if not self.isConfig:
                         nplots = len(channelIndexList)
                         self.setup(id=id,
                                    nplots=nplots,
                                    wintitle=wintitle,
                                    showprofile=showprofile,
                                    show=show)
                         if xmin == None: xmin = numpy.nanmin(x)
                         if xmax == None: xmax = numpy.nanmax(x)
                         if ymin == None: ymin = numpy.nanmin(y)
                         if ymax == None: ymax = numpy.nanmax(y)
                         if zmin == None: zmin = numpy.floor(numpy.nanmin(noisedB)) - 3
                         if zmax == None: zmax = numpy.ceil(numpy.nanmax(avgdB)) + 3
                         self.FTP_WEI = ftp_wei
                         self.EXP_CODE = exp_code
                         self.SUB_EXP_CODE = sub_exp_code
                         self.PLOT_POS = plot_pos
                         self.isConfig = True
                     self.setWinTitle(title)
                     for i in range(self.nplots):
                         index = channelIndexList[i]
                         str_datetime = '%s %s'%(thisDatetime.strftime("%Y/%m/%d"),thisDatetime.strftime("%H:%M:%S"))
                         title = "Channel %d: %4.2fdB: %s" %(dataOut.channelList[index], noisedB[index], str_datetime)
                         if len(dataOut.beam.codeList) != 0:
                             title = "Ch%d:%4.2fdB,%2.2f,%2.2f:%s" %(dataOut.channelList[index], noisedB[index], dataOut.beam.azimuthList[index], dataOut.beam.zenithList[index], str_datetime)
                         axes = self.axesList[i*self.__nsubplots]
                         axes.pcolor(x, y, zdB[index,:,:],
                                     xmin=xmin, xmax=xmax, ymin=ymin, ymax=ymax, zmin=zmin, zmax=zmax,
                                     xlabel=xlabel, ylabel=ylabel, title=title, colormap=colormap,
                                     ticksize=9, cblabel='')
                         if self.__showprofile:
                             axes = self.axesList[i*self.__nsubplots +1]
                             axes.pline(avgdB[index,:], y,
                                     xmin=zmin, xmax=zmax, ymin=ymin, ymax=ymax,
                                     xlabel='dB', ylabel='', title='',
                                     ytick_visible=False,
                                     grid='x')
                             noiseline = numpy.repeat(noisedB[index], len(y))
                             axes.addpline(noiseline, y, idline=1, color="black", linestyle="dashed", lw=2)
                     self.draw()
                     if figfile == None:
                         str_datetime = thisDatetime.strftime("%Y%m%d_%H%M%S")
                         name = str_datetime
                         if ((dataOut.azimuth!=None) and (dataOut.zenith!=None)):
                             name = name + '_az' + '_%2.2f'%(dataOut.azimuth) + '_zn' + '_%2.2f'%(dataOut.zenith)
                         figfile = self.getFilename(name)
                     self.save(figpath=figpath,
                               figfile=figfile,
                               save=save,
                               ftp=ftp,
                               wr_period=wr_period,
                               thisDatetime=thisDatetime)
             class CrossSpectraPlot(Figure):
                 isConfig = None
                 __nsubplots = None
                 WIDTH = None
                 HEIGHT = None
                 WIDTHPROF = None
                 HEIGHTPROF = None
                 PREFIX = 'cspc'
                 def __init__(self, **kwargs):
                     Figure.__init__(self, **kwargs)
                     self.isConfig = False
                     self.__nsubplots = 4
                     self.counter_imagwr = 0
                     self.WIDTH = 250
                     self.HEIGHT = 250
                     self.WIDTHPROF = 0
                     self.HEIGHTPROF = 0
                     self.PLOT_CODE = CROSS_CODE
                     self.FTP_WEI = None
                     self.EXP_CODE = None
                     self.SUB_EXP_CODE = None
                     self.PLOT_POS = None
                 def getSubplots(self):
                     ncol = 4
                     nrow = self.nplots
                     return nrow, ncol
                 def setup(self, id, nplots, wintitle, showprofile=True, show=True):
                     self.__showprofile = showprofile
                     self.nplots = nplots
                     ncolspan = 1
                     colspan = 1
                     self.createFigure(id = id,
                                       wintitle = wintitle,
                                       widthplot = self.WIDTH + self.WIDTHPROF,
                                       heightplot = self.HEIGHT + self.HEIGHTPROF,
                                       show=True)
                     nrow, ncol = self.getSubplots()
                     counter = 0
                     for y in range(nrow):
                         for x in range(ncol):
                             self.addAxes(nrow, ncol*ncolspan, y, x*ncolspan, colspan, 1)
                             counter += 1
                 def run(self, dataOut, id, wintitle="", pairsList=None,
                         xmin=None, xmax=None, ymin=None, ymax=None, zmin=None, zmax=None,
                         coh_min=None, coh_max=None, phase_min=None, phase_max=None,
                         save=False, figpath='./', figfile=None, ftp=False, wr_period=1,
                         power_cmap='jet', coherence_cmap='jet', phase_cmap='RdBu_r', show=True,
                         server=None, folder=None, username=None, password=None,
                         ftp_wei=0, exp_code=0, sub_exp_code=0, plot_pos=0,
                         xaxis='frequency'):
                     """
                     Input:
                         dataOut         :
                         id        :
                         wintitle        :
                         channelList     :
                         showProfile     :
                         xmin            :    None,
                         xmax            :    None,
                         ymin            :    None,
                         ymax            :    None,
                         zmin            :    None,
                         zmax            :    None
                     """
                     if pairsList == None:
                         pairsIndexList = dataOut.pairsIndexList
                     else:
                         pairsIndexList = []
                         for pair in pairsList:
                             if pair not in dataOut.pairsList:
                                 raise ValueError, "Pair %s is not in dataOut.pairsList" %str(pair)
                             pairsIndexList.append(dataOut.pairsList.index(pair))
                     if not pairsIndexList:
                         return
                     if len(pairsIndexList) > 4:
                         pairsIndexList = pairsIndexList[0:4]
                     factor = dataOut.normFactor
                     x = dataOut.getVelRange(1)
                     y = dataOut.getHeiRange()
                     z = dataOut.data_spc[:,:,:]/factor
                     z = numpy.where(numpy.isfinite(z), z, numpy.NAN)
                     noise = dataOut.noise/factor
                     zdB = 10*numpy.log10(z)
                     noisedB = 10*numpy.log10(noise)
                     if coh_min == None:
                         coh_min = 0.0
                     if coh_max == None:
                         coh_max = 1.0
                     if phase_min == None:
                         phase_min = -180
                     if phase_max == None:
                         phase_max = 180
                     #thisDatetime = dataOut.datatime
                     thisDatetime = datetime.datetime.utcfromtimestamp(dataOut.getTimeRange()[0])
                     title = wintitle + " Cross-Spectra: %s" %(thisDatetime.strftime("%d-%b-%Y %H:%M:%S"))
             #         xlabel = "Velocity (m/s)"
                     ylabel = "Range (Km)"
                     if xaxis == "frequency":
                         x = dataOut.getFreqRange(1)/1000.
                         xlabel = "Frequency (kHz)"
                     elif xaxis == "time":
                         x = dataOut.getAcfRange(1)
                         xlabel = "Time (ms)"
                     else:
                         x = dataOut.getVelRange(1)
                         xlabel = "Velocity (m/s)"
                     if not self.isConfig:
                         nplots = len(pairsIndexList)
                         self.setup(id=id,
                                    nplots=nplots,
                                    wintitle=wintitle,
                                    showprofile=False,
                                    show=show)
                         avg = numpy.abs(numpy.average(z, axis=1))
                         avgdB = 10*numpy.log10(avg)
                         if xmin == None: xmin = numpy.nanmin(x)
                         if xmax == None: xmax = numpy.nanmax(x)
                         if ymin == None: ymin = numpy.nanmin(y)
                         if ymax == None: ymax = numpy.nanmax(y)
                         if zmin == None: zmin = numpy.floor(numpy.nanmin(noisedB)) - 3
                         if zmax == None: zmax = numpy.ceil(numpy.nanmax(avgdB)) + 3
                         self.FTP_WEI = ftp_wei
                         self.EXP_CODE = exp_code
                         self.SUB_EXP_CODE = sub_exp_code
                         self.PLOT_POS = plot_pos
                         self.isConfig = True
                     self.setWinTitle(title)
                     for i in range(self.nplots):
                         pair = dataOut.pairsList[pairsIndexList[i]]
                         chan_index0 = dataOut.channelList.index(pair[0])
                         chan_index1 = dataOut.channelList.index(pair[1])
                         str_datetime = '%s %s'%(thisDatetime.strftime("%Y/%m/%d"),thisDatetime.strftime("%H:%M:%S"))
                         title = "Ch%d: %4.2fdB: %s" %(pair[0], noisedB[chan_index0], str_datetime)
                         zdB = 10.*numpy.log10(dataOut.data_spc[chan_index0,:,:]/factor)
                         axes0 = self.axesList[i*self.__nsubplots]
                         axes0.pcolor(x, y, zdB,
                                     xmin=xmin, xmax=xmax, ymin=ymin, ymax=ymax, zmin=zmin, zmax=zmax,
                                     xlabel=xlabel, ylabel=ylabel, title=title,
                                     ticksize=9, colormap=power_cmap, cblabel='')
                         title = "Ch%d: %4.2fdB: %s" %(pair[1], noisedB[chan_index1], str_datetime)
                         zdB = 10.*numpy.log10(dataOut.data_spc[chan_index1,:,:]/factor)
                         axes0 = self.axesList[i*self.__nsubplots+1]
                         axes0.pcolor(x, y, zdB,
                                     xmin=xmin, xmax=xmax, ymin=ymin, ymax=ymax, zmin=zmin, zmax=zmax,
                                     xlabel=xlabel, ylabel=ylabel, title=title,
                                     ticksize=9, colormap=power_cmap, cblabel='')
                         coherenceComplex = dataOut.data_cspc[pairsIndexList[i],:,:]/numpy.sqrt(dataOut.data_spc[chan_index0,:,:]*dataOut.data_spc[chan_index1,:,:])
                         coherence = numpy.abs(coherenceComplex)
             #            phase = numpy.arctan(-1*coherenceComplex.imag/coherenceComplex.real)*180/numpy.pi
                         phase = numpy.arctan2(coherenceComplex.imag, coherenceComplex.real)*180/numpy.pi
                         title = "Coherence Ch%d * Ch%d" %(pair[0], pair[1])
                         axes0 = self.axesList[i*self.__nsubplots+2]
                         axes0.pcolor(x, y, coherence,
                                     xmin=xmin, xmax=xmax, ymin=ymin, ymax=ymax, zmin=coh_min, zmax=coh_max,
                                     xlabel=xlabel, ylabel=ylabel, title=title,
                                     ticksize=9, colormap=coherence_cmap, cblabel='')
                         title = "Phase Ch%d * Ch%d" %(pair[0], pair[1])
                         axes0 = self.axesList[i*self.__nsubplots+3]
                         axes0.pcolor(x, y, phase,
                                     xmin=xmin, xmax=xmax, ymin=ymin, ymax=ymax, zmin=phase_min, zmax=phase_max,
                                     xlabel=xlabel, ylabel=ylabel, title=title,
                                     ticksize=9, colormap=phase_cmap, cblabel='')
                     self.draw()
                     self.save(figpath=figpath,
                               figfile=figfile,
                               save=save,
                               ftp=ftp,
                               wr_period=wr_period,
                               thisDatetime=thisDatetime)
             class RTIPlot(Figure):
                 __isConfig = None
                 __nsubplots = None
                 WIDTHPROF = None
                 HEIGHTPROF = None
                 PREFIX = 'rti'
                 def __init__(self, **kwargs):
                     Figure.__init__(self, **kwargs)
                     self.timerange = None
                     self.isConfig = False
                     self.__nsubplots = 1
                     self.WIDTH = 800
                     self.HEIGHT = 180
                     self.WIDTHPROF = 120
                     self.HEIGHTPROF = 0
                     self.counter_imagwr = 0
                     self.PLOT_CODE = RTI_CODE
                     self.FTP_WEI = None
                     self.EXP_CODE = None
                     self.SUB_EXP_CODE = None
                     self.PLOT_POS = None
                     self.tmin = None
                     self.tmax = None
                     self.xmin = None
                     self.xmax = None
                     self.figfile = None
                 def getSubplots(self):
                     ncol = 1
                     nrow = self.nplots
                     return nrow, ncol
                 def setup(self, id, nplots, wintitle, showprofile=True, show=True):
                     self.__showprofile = showprofile
                     self.nplots = nplots
                     ncolspan = 1
                     colspan = 1
                     if showprofile:
                         ncolspan = 7
                         colspan = 6
                         self.__nsubplots = 2
                     self.createFigure(id = id,
                                       wintitle = wintitle,
                                       widthplot = self.WIDTH + self.WIDTHPROF,
                                       heightplot = self.HEIGHT + self.HEIGHTPROF,
                                       show=show)
                     nrow, ncol = self.getSubplots()
                     counter = 0
                     for y in range(nrow):
                         for x in range(ncol):
                             if counter >= self.nplots:
                                 break
                             self.addAxes(nrow, ncol*ncolspan, y, x*ncolspan, colspan, 1)
                             if showprofile:
                                 self.addAxes(nrow, ncol*ncolspan, y, x*ncolspan+colspan, 1, 1)
                             counter += 1
                 def run(self, dataOut, id, wintitle="", channelList=None, showprofile='True',
                         xmin=None, xmax=None, ymin=None, ymax=None, zmin=None, zmax=None,
                         timerange=None,
                         save=False, figpath='./', lastone=0,figfile=None, ftp=False, wr_period=1, show=True,
                         server=None, folder=None, username=None, password=None,
                         ftp_wei=0, exp_code=0, sub_exp_code=0, plot_pos=0, **kwargs):
                     """
                     Input:
                         dataOut         :
                         id        :
                         wintitle        :
                         channelList     :
                         showProfile     :
                         xmin            :    None,
                         xmax            :    None,
                         ymin            :    None,
                         ymax            :    None,
                         zmin            :    None,
                         zmax            :    None
                     """
                     colormap = kwargs.get('colormap', 'jet')
                     if not isTimeInHourRange(dataOut.datatime, xmin, xmax):
                         return
                     if channelList == None:
                         channelIndexList = dataOut.channelIndexList
                     else:
                         channelIndexList = []
                         for channel in channelList:
                             if channel not in dataOut.channelList:
                                 raise ValueError, "Channel %d is not in dataOut.channelList"
                             channelIndexList.append(dataOut.channelList.index(channel))
                     if hasattr(dataOut, 'normFactor'):
                         factor = dataOut.normFactor
                     else:
                         factor = 1
             #         factor = dataOut.normFactor
                     x = dataOut.getTimeRange()
                     y = dataOut.getHeiRange()
             #         z = dataOut.data_spc/factor
             #         z = numpy.where(numpy.isfinite(z), z, numpy.NAN)
             #         avg = numpy.average(z, axis=1)
             #         avgdB = 10.*numpy.log10(avg)
                     avgdB = dataOut.getPower()
                     thisDatetime = dataOut.datatime
             #         thisDatetime = datetime.datetime.utcfromtimestamp(dataOut.getTimeRange()[0])
                     title = wintitle + " RTI" #: %s" %(thisDatetime.strftime("%d-%b-%Y"))
                     xlabel = ""
                     ylabel = "Range (Km)"
                     update_figfile = False
                     if dataOut.ltctime >= self.xmax:
                         self.counter_imagwr = wr_period
                         self.isConfig = False
                         update_figfile = True
                     if not self.isConfig:
                         nplots = len(channelIndexList)
                         self.setup(id=id,
                                    nplots=nplots,
                                    wintitle=wintitle,
                                    showprofile=showprofile,
                                    show=show)
                         if timerange != None:
                             self.timerange = timerange
                         self.xmin, self.xmax = self.getTimeLim(x, xmin, xmax, timerange)
                         noise = dataOut.noise/factor
                         noisedB = 10*numpy.log10(noise)
                         if ymin == None: ymin = numpy.nanmin(y)
                         if ymax == None: ymax = numpy.nanmax(y)
                         if zmin == None: zmin = numpy.floor(numpy.nanmin(noisedB)) - 3
                         if zmax == None: zmax = numpy.ceil(numpy.nanmax(avgdB)) + 3
                         self.FTP_WEI = ftp_wei
                         self.EXP_CODE = exp_code
                         self.SUB_EXP_CODE = sub_exp_code
                         self.PLOT_POS = plot_pos
                         self.name = thisDatetime.strftime("%Y%m%d_%H%M%S")
                         self.isConfig = True
                         self.figfile = figfile
                         update_figfile = True
                     self.setWinTitle(title)
                     for i in range(self.nplots):
                         index = channelIndexList[i]
                         title = "Channel %d: %s" %(dataOut.channelList[index], thisDatetime.strftime("%Y/%m/%d %H:%M:%S"))
                         if ((dataOut.azimuth!=None) and (dataOut.zenith!=None)):
                             title = title + '_' + 'azimuth,zenith=%2.2f,%2.2f'%(dataOut.azimuth, dataOut.zenith)
                         axes = self.axesList[i*self.__nsubplots]
                         zdB = avgdB[index].reshape((1,-1))
                         axes.pcolorbuffer(x, y, zdB,
                                     xmin=self.xmin, xmax=self.xmax, ymin=ymin, ymax=ymax, zmin=zmin, zmax=zmax,
                                     xlabel=xlabel, ylabel=ylabel, title=title, rti=True, XAxisAsTime=True,
                                     ticksize=9, cblabel='', cbsize="1%", colormap=colormap)
                         if self.__showprofile:
                             axes = self.axesList[i*self.__nsubplots +1]
                             axes.pline(avgdB[index], y,
                                     xmin=zmin, xmax=zmax, ymin=ymin, ymax=ymax,
                                     xlabel='dB', ylabel='', title='',
                                     ytick_visible=False,
                                     grid='x')
                     self.draw()
                     self.save(figpath=figpath,
                               figfile=figfile,
                               save=save,
                               ftp=ftp,
                               wr_period=wr_period,
                               thisDatetime=thisDatetime,
                               update_figfile=update_figfile)
             class CoherenceMap(Figure):
                 isConfig = None
                 __nsubplots = None
                 WIDTHPROF = None
                 HEIGHTPROF = None
                 PREFIX = 'cmap'
                 def __init__(self, **kwargs):
                     Figure.__init__(self, **kwargs)
                     self.timerange = 2*60*60
                     self.isConfig = False
                     self.__nsubplots = 1
                     self.WIDTH = 800
                     self.HEIGHT = 180
                     self.WIDTHPROF = 120
                     self.HEIGHTPROF = 0
                     self.counter_imagwr = 0
                     self.PLOT_CODE = COH_CODE
                     self.FTP_WEI = None
                     self.EXP_CODE = None
                     self.SUB_EXP_CODE = None
                     self.PLOT_POS = None
                     self.counter_imagwr = 0
                     self.xmin = None
                     self.xmax = None
                 def getSubplots(self):
                     ncol = 1
                     nrow = self.nplots*2
                     return nrow, ncol
                 def setup(self, id, nplots, wintitle, showprofile=True, show=True):
                     self.__showprofile = showprofile
                     self.nplots = nplots
                     ncolspan = 1
                     colspan = 1
                     if showprofile:
                         ncolspan = 7
                         colspan = 6
                         self.__nsubplots = 2
                     self.createFigure(id = id,
                                       wintitle = wintitle,
                                       widthplot = self.WIDTH + self.WIDTHPROF,
                                       heightplot = self.HEIGHT + self.HEIGHTPROF,
                                       show=True)
                     nrow, ncol = self.getSubplots()
                     for y in range(nrow):
                         for x in range(ncol):
                             self.addAxes(nrow, ncol*ncolspan, y, x*ncolspan, colspan, 1)
                             if showprofile:
                                 self.addAxes(nrow, ncol*ncolspan, y, x*ncolspan+colspan, 1, 1)
                 def run(self, dataOut, id, wintitle="", pairsList=None, showprofile='True',
                         xmin=None, xmax=None, ymin=None, ymax=None, zmin=None, zmax=None,
                         timerange=None, phase_min=None, phase_max=None,
                         save=False, figpath='./', figfile=None, ftp=False, wr_period=1,
                         coherence_cmap='jet', phase_cmap='RdBu_r', show=True,
                         server=None, folder=None, username=None, password=None,
                         ftp_wei=0, exp_code=0, sub_exp_code=0, plot_pos=0):
                     if not isTimeInHourRange(dataOut.datatime, xmin, xmax):
                         return
                     if pairsList == None:
                         pairsIndexList = dataOut.pairsIndexList
                     else:
                         pairsIndexList = []
                         for pair in pairsList:
                             if pair not in dataOut.pairsList:
                                 raise ValueError, "Pair %s is not in dataOut.pairsList" %(pair)
                             pairsIndexList.append(dataOut.pairsList.index(pair))
                     if pairsIndexList == []:
                         return
                     if len(pairsIndexList) > 4:
                         pairsIndexList = pairsIndexList[0:4]
                     if phase_min == None:
                         phase_min = -180
                     if phase_max == None:
                         phase_max = 180
                     x = dataOut.getTimeRange()
                     y = dataOut.getHeiRange()
                     thisDatetime = dataOut.datatime
                     title = wintitle + " CoherenceMap" #: %s" %(thisDatetime.strftime("%d-%b-%Y"))
                     xlabel = ""
                     ylabel = "Range (Km)"
                     update_figfile = False
                     if not self.isConfig:
                         nplots = len(pairsIndexList)
                         self.setup(id=id,
                                    nplots=nplots,
                                    wintitle=wintitle,
                                    showprofile=showprofile,
                                    show=show)
                         if timerange != None:
                             self.timerange = timerange
                         self.xmin, self.xmax = self.getTimeLim(x, xmin, xmax, timerange)
                         if ymin == None: ymin = numpy.nanmin(y)
                         if ymax == None: ymax = numpy.nanmax(y)
                         if zmin == None: zmin = 0.
                         if zmax == None: zmax = 1.
                         self.FTP_WEI = ftp_wei
                         self.EXP_CODE = exp_code
                         self.SUB_EXP_CODE = sub_exp_code
                         self.PLOT_POS = plot_pos
                         self.name = thisDatetime.strftime("%Y%m%d_%H%M%S")
                         self.isConfig = True
                         update_figfile = True
                     self.setWinTitle(title)
                     for i in range(self.nplots):
                         pair = dataOut.pairsList[pairsIndexList[i]]
                         ccf = numpy.average(dataOut.data_cspc[pairsIndexList[i],:,:],axis=0)
                         powa = numpy.average(dataOut.data_spc[pair[0],:,:],axis=0)
                         powb = numpy.average(dataOut.data_spc[pair[1],:,:],axis=0)
                         avgcoherenceComplex = ccf/numpy.sqrt(powa*powb)
                         coherence = numpy.abs(avgcoherenceComplex)
                         z = coherence.reshape((1,-1))
                         counter = 0
                         title = "Coherence Ch%d * Ch%d: %s" %(pair[0], pair[1], thisDatetime.strftime("%d-%b-%Y %H:%M:%S"))
                         axes = self.axesList[i*self.__nsubplots*2]
                         axes.pcolorbuffer(x, y, z,
                                     xmin=self.xmin, xmax=self.xmax, ymin=ymin, ymax=ymax, zmin=zmin, zmax=zmax,
                                     xlabel=xlabel, ylabel=ylabel, title=title, rti=True, XAxisAsTime=True,
                                     ticksize=9, cblabel='', colormap=coherence_cmap, cbsize="1%")
                         if self.__showprofile:
                             counter += 1
                             axes = self.axesList[i*self.__nsubplots*2 + counter]
                             axes.pline(coherence, y,
                                     xmin=zmin, xmax=zmax, ymin=ymin, ymax=ymax,
                                     xlabel='', ylabel='', title='', ticksize=7,
                                     ytick_visible=False, nxticks=5,
                                     grid='x')
                         counter += 1
                         phase = numpy.arctan2(avgcoherenceComplex.imag, avgcoherenceComplex.real)*180/numpy.pi
                         z = phase.reshape((1,-1))
                         title = "Phase Ch%d * Ch%d: %s" %(pair[0], pair[1], thisDatetime.strftime("%d-%b-%Y %H:%M:%S"))
                         axes = self.axesList[i*self.__nsubplots*2 + counter]
                         axes.pcolorbuffer(x, y, z,
                                     xmin=self.xmin, xmax=self.xmax, ymin=ymin, ymax=ymax, zmin=phase_min, zmax=phase_max,
                                     xlabel=xlabel, ylabel=ylabel, title=title, rti=True, XAxisAsTime=True,
                                     ticksize=9, cblabel='', colormap=phase_cmap, cbsize="1%")
                         if self.__showprofile:
                             counter += 1
                             axes = self.axesList[i*self.__nsubplots*2 + counter]
                             axes.pline(phase, y,
                                     xmin=phase_min, xmax=phase_max, ymin=ymin, ymax=ymax,
                                     xlabel='', ylabel='', title='', ticksize=7,
                                     ytick_visible=False, nxticks=4,
                                     grid='x')
                     self.draw()
                     if dataOut.ltctime >= self.xmax:
                         self.counter_imagwr = wr_period
                         self.isConfig = False
                         update_figfile = True
                     self.save(figpath=figpath,
                               figfile=figfile,
                               save=save,
                               ftp=ftp,
                               wr_period=wr_period,
                               thisDatetime=thisDatetime,
                               update_figfile=update_figfile)
             class PowerProfilePlot(Figure):
                 isConfig = None
                 __nsubplots = None
                 WIDTHPROF = None
                 HEIGHTPROF = None
                 PREFIX = 'spcprofile'
                 def __init__(self, **kwargs):
                     Figure.__init__(self, **kwargs)
                     self.isConfig = False
                     self.__nsubplots = 1
                     self.PLOT_CODE = POWER_CODE
                     self.WIDTH = 300
                     self.HEIGHT = 500
                     self.counter_imagwr = 0
                 def getSubplots(self):
                     ncol = 1
                     nrow = 1
                     return nrow, ncol
                 def setup(self, id, nplots, wintitle, show):
                     self.nplots = nplots
                     ncolspan = 1
                     colspan = 1
                     self.createFigure(id = id,
                                       wintitle = wintitle,
                                       widthplot = self.WIDTH,
                                       heightplot = self.HEIGHT,
                                       show=show)
                     nrow, ncol = self.getSubplots()
                     counter = 0
                     for y in range(nrow):
                         for x in range(ncol):
                             self.addAxes(nrow, ncol*ncolspan, y, x*ncolspan, colspan, 1)
                 def run(self, dataOut, id, wintitle="", channelList=None,
                         xmin=None, xmax=None, ymin=None, ymax=None,
                         save=False, figpath='./', figfile=None, show=True,
                         ftp=False, wr_period=1, server=None,
                         folder=None, username=None, password=None):
                     if channelList == None:
                         channelIndexList = dataOut.channelIndexList
                         channelList = dataOut.channelList
                     else:
                         channelIndexList = []
                         for channel in channelList:
                             if channel not in dataOut.channelList:
                                 raise ValueError, "Channel %d is not in dataOut.channelList"
                             channelIndexList.append(dataOut.channelList.index(channel))
                     factor = dataOut.normFactor
                     y = dataOut.getHeiRange()
                     #for voltage
                     if dataOut.type == 'Voltage':
                         x = dataOut.data[channelIndexList,:] * numpy.conjugate(dataOut.data[channelIndexList,:])
                         x = x.real
                         x = numpy.where(numpy.isfinite(x), x, numpy.NAN)
                     #for spectra
                     if dataOut.type == 'Spectra':
                         x = dataOut.data_spc[channelIndexList,:,:]/factor
                         x = numpy.where(numpy.isfinite(x), x, numpy.NAN)
                         x = numpy.average(x, axis=1)
                     xdB = 10*numpy.log10(x)
                     thisDatetime = datetime.datetime.utcfromtimestamp(dataOut.getTimeRange()[0])
                     title = wintitle + " Power Profile %s" %(thisDatetime.strftime("%d-%b-%Y"))
                     xlabel = "dB"
                     ylabel = "Range (Km)"
                     if not self.isConfig:
                         nplots = 1
                         self.setup(id=id,
                                    nplots=nplots,
                                    wintitle=wintitle,
                                    show=show)
                         if ymin == None: ymin = numpy.nanmin(y)
                         if ymax == None: ymax = numpy.nanmax(y)
                         if xmin == None: xmin = numpy.nanmin(xdB)*0.9
                         if xmax == None: xmax = numpy.nanmax(xdB)*1.1
                         self.isConfig = True
                     self.setWinTitle(title)
                     title = "Power Profile: %s" %(thisDatetime.strftime("%d-%b-%Y %H:%M:%S"))
                     axes = self.axesList[0]
                     legendlabels = ["channel %d"%x for x in channelList]
                     axes.pmultiline(xdB, y,
                             xmin=xmin, xmax=xmax, ymin=ymin, ymax=ymax,
                             xlabel=xlabel, ylabel=ylabel, title=title, legendlabels=legendlabels,
                             ytick_visible=True, nxticks=5,
                             grid='x')
                     self.draw()
                     self.save(figpath=figpath,
                               figfile=figfile,
                               save=save,
                               ftp=ftp,
                               wr_period=wr_period,
                               thisDatetime=thisDatetime)
             class SpectraCutPlot(Figure):
                 isConfig = None
                 __nsubplots = None
                 WIDTHPROF = None
                 HEIGHTPROF = None
                 PREFIX = 'spc_cut'
                 def __init__(self, **kwargs):
                     Figure.__init__(self, **kwargs)
                     self.isConfig = False
                     self.__nsubplots = 1
                     self.PLOT_CODE = POWER_CODE
                     self.WIDTH = 700
                     self.HEIGHT = 500
                     self.counter_imagwr = 0
                 def getSubplots(self):
                     ncol = 1
                     nrow = 1
                     return nrow, ncol
                 def setup(self, id, nplots, wintitle, show):
                     self.nplots = nplots
                     ncolspan = 1
                     colspan = 1
                     self.createFigure(id = id,
                                       wintitle = wintitle,
                                       widthplot = self.WIDTH,
                                       heightplot = self.HEIGHT,
                                       show=show)
                     nrow, ncol = self.getSubplots()
                     counter = 0
                     for y in range(nrow):
                         for x in range(ncol):
                             self.addAxes(nrow, ncol*ncolspan, y, x*ncolspan, colspan, 1)
                 def run(self, dataOut, id, wintitle="", channelList=None,
                         xmin=None, xmax=None, ymin=None, ymax=None,
                         save=False, figpath='./', figfile=None, show=True,
                         ftp=False, wr_period=1, server=None,
                         folder=None, username=None, password=None,
                         xaxis="frequency"):
                     if channelList == None:
                         channelIndexList = dataOut.channelIndexList
                         channelList = dataOut.channelList
                     else:
                         channelIndexList = []
                         for channel in channelList:
                             if channel not in dataOut.channelList:
                                 raise ValueError, "Channel %d is not in dataOut.channelList"
                             channelIndexList.append(dataOut.channelList.index(channel))
                     factor = dataOut.normFactor
                     y = dataOut.getHeiRange()
                     z = dataOut.data_spc/factor
                     z = numpy.where(numpy.isfinite(z), z, numpy.NAN)
                     hei_index = numpy.arange(25)*3 + 20
                     if xaxis == "frequency":
                         x = dataOut.getFreqRange()/1000.
                         zdB = 10*numpy.log10(z[0,:,hei_index])
                         xlabel = "Frequency (kHz)"
                         ylabel = "Power (dB)"
                     elif xaxis == "time":
                         x = dataOut.getAcfRange()
                         zdB = z[0,:,hei_index]
                         xlabel = "Time (ms)"
                         ylabel = "ACF"
                     else:
                         x = dataOut.getVelRange()
                         zdB = 10*numpy.log10(z[0,:,hei_index])
                         xlabel = "Velocity (m/s)"
                         ylabel = "Power (dB)"
                     thisDatetime = datetime.datetime.utcfromtimestamp(dataOut.getTimeRange()[0])
                     title = wintitle + " Range Cuts %s" %(thisDatetime.strftime("%d-%b-%Y"))
                     if not self.isConfig:
                         nplots = 1
                         self.setup(id=id,
                                    nplots=nplots,
                                    wintitle=wintitle,
                                    show=show)
                         if xmin == None: xmin = numpy.nanmin(x)*0.9
                         if xmax == None: xmax = numpy.nanmax(x)*1.1
                         if ymin == None: ymin = numpy.nanmin(zdB)
                         if ymax == None: ymax = numpy.nanmax(zdB)
                         self.isConfig = True
                     self.setWinTitle(title)
                     title = "Spectra Cuts: %s" %(thisDatetime.strftime("%d-%b-%Y %H:%M:%S"))
                     axes = self.axesList[0]
                     legendlabels = ["Range = %dKm" %y[i] for i in hei_index]
                     axes.pmultilineyaxis( x, zdB,
                             xmin=xmin, xmax=xmax, ymin=ymin, ymax=ymax,
                             xlabel=xlabel, ylabel=ylabel, title=title, legendlabels=legendlabels,
                             ytick_visible=True, nxticks=5,
                             grid='x')
                     self.draw()
                     self.save(figpath=figpath,
                               figfile=figfile,
                               save=save,
                               ftp=ftp,
                               wr_period=wr_period,
                               thisDatetime=thisDatetime)
             class Noise(Figure):
                 isConfig = None
                 __nsubplots = None
                 PREFIX = 'noise'
                 def __init__(self, **kwargs):
                     Figure.__init__(self, **kwargs)
                     self.timerange = 24*60*60
                     self.isConfig = False
                     self.__nsubplots = 1
                     self.counter_imagwr = 0
                     self.WIDTH = 800
                     self.HEIGHT = 400
                     self.WIDTHPROF = 120
                     self.HEIGHTPROF = 0
                     self.xdata = None
                     self.ydata = None
                     self.PLOT_CODE = NOISE_CODE
                     self.FTP_WEI = None
                     self.EXP_CODE = None
                     self.SUB_EXP_CODE = None
                     self.PLOT_POS = None
                     self.figfile = None
                     self.xmin = None
                     self.xmax = None
                 def getSubplots(self):
                     ncol = 1
                     nrow = 1
                     return nrow, ncol
                 def openfile(self, filename):
                     dirname = os.path.dirname(filename)
                     if not os.path.exists(dirname):
                         os.mkdir(dirname)
                     f = open(filename,'w+')
                     f.write('\n\n')
                     f.write('JICAMARCA RADIO OBSERVATORY - Noise \n')
                     f.write('DD MM YYYY  HH MM SS   Channel0    Channel1    Channel2    Channel3\n\n' )
                     f.close()
                 def save_data(self, filename_phase, data, data_datetime):
                     f=open(filename_phase,'a')
                     timetuple_data = data_datetime.timetuple()
                     day = str(timetuple_data.tm_mday)
                     month = str(timetuple_data.tm_mon)
                     year = str(timetuple_data.tm_year)
                     hour = str(timetuple_data.tm_hour)
                     minute = str(timetuple_data.tm_min)
                     second = str(timetuple_data.tm_sec)
                     data_msg = ''
                     for i in range(len(data)):
                         data_msg += str(data[i]) + '  '
                     f.write(day+' '+month+' '+year+'  '+hour+' '+minute+' '+second+'   ' + data_msg + '\n')
                     f.close()
                 def setup(self, id, nplots, wintitle, showprofile=True, show=True):
                     self.__showprofile = showprofile
                     self.nplots = nplots
                     ncolspan = 7
                     colspan = 6
                     self.__nsubplots = 2
                     self.createFigure(id = id,
                                       wintitle = wintitle,
                                       widthplot = self.WIDTH+self.WIDTHPROF,
                                       heightplot = self.HEIGHT+self.HEIGHTPROF,
                                       show=show)
                     nrow, ncol = self.getSubplots()
                     self.addAxes(nrow, ncol*ncolspan, 0, 0, colspan, 1)
                 def run(self, dataOut, id, wintitle="", channelList=None, showprofile='True',
                         xmin=None, xmax=None, ymin=None, ymax=None,
                         timerange=None,
                         save=False, figpath='./', figfile=None, show=True, ftp=False, wr_period=1,
                         server=None, folder=None, username=None, password=None,
                         ftp_wei=0, exp_code=0, sub_exp_code=0, plot_pos=0):
                     if not isTimeInHourRange(dataOut.datatime, xmin, xmax):
                         return
                     if channelList == None:
                         channelIndexList = dataOut.channelIndexList
                         channelList = dataOut.channelList
                     else:
                         channelIndexList = []
                         for channel in channelList:
                             if channel not in dataOut.channelList:
                                 raise ValueError, "Channel %d is not in dataOut.channelList"
                             channelIndexList.append(dataOut.channelList.index(channel))
                     x = dataOut.getTimeRange()
                     #y = dataOut.getHeiRange()
                     factor = dataOut.normFactor
                     noise = dataOut.noise[channelIndexList]/factor
                     noisedB = 10*numpy.log10(noise)
                     thisDatetime = dataOut.datatime
                     title = wintitle + " Noise" # : %s" %(thisDatetime.strftime("%d-%b-%Y"))
                     xlabel = ""
                     ylabel = "Intensity (dB)"
                     update_figfile = False
                     if not self.isConfig:
                         nplots = 1
                         self.setup(id=id,
                                    nplots=nplots,
                                    wintitle=wintitle,
                                    showprofile=showprofile,
                                    show=show)
                         if timerange != None:
                             self.timerange = timerange
                         self.xmin, self.xmax = self.getTimeLim(x, xmin, xmax, timerange)
                         if ymin == None: ymin = numpy.floor(numpy.nanmin(noisedB)) - 10.0
                         if ymax == None: ymax = numpy.nanmax(noisedB) + 10.0
                         self.FTP_WEI = ftp_wei
                         self.EXP_CODE = exp_code
                         self.SUB_EXP_CODE = sub_exp_code
                         self.PLOT_POS = plot_pos
                         self.name = thisDatetime.strftime("%Y%m%d_%H%M%S")
                         self.isConfig = True
                         self.figfile = figfile
                         self.xdata = numpy.array([])
                         self.ydata = numpy.array([])
                         update_figfile = True
                         #open file beacon phase
                         path = '%s%03d' %(self.PREFIX, self.id)
                         noise_file = os.path.join(path,'%s.txt'%self.name)
                         self.filename_noise = os.path.join(figpath,noise_file)
                     self.setWinTitle(title)
                     title = "Noise %s" %(thisDatetime.strftime("%Y/%m/%d %H:%M:%S"))
                     legendlabels = ["channel %d"%(idchannel) for idchannel in channelList]
                     axes = self.axesList[0]
                     self.xdata = numpy.hstack((self.xdata, x[0:1]))
                     if len(self.ydata)==0:
                         self.ydata = noisedB.reshape(-1,1)
                     else:
                         self.ydata = numpy.hstack((self.ydata, noisedB.reshape(-1,1)))
                     axes.pmultilineyaxis(x=self.xdata, y=self.ydata,
                                 xmin=self.xmin, xmax=self.xmax, ymin=ymin, ymax=ymax,
                                 xlabel=xlabel, ylabel=ylabel, title=title, legendlabels=legendlabels, marker='x', markersize=8, linestyle="solid",
                                 XAxisAsTime=True, grid='both'
                                 )
                     self.draw()
                     if dataOut.ltctime >= self.xmax:
                         self.counter_imagwr = wr_period
                         self.isConfig = False
                         update_figfile = True
                     self.save(figpath=figpath,
                               figfile=figfile,
                               save=save,
                               ftp=ftp,
                               wr_period=wr_period,
                               thisDatetime=thisDatetime,
                               update_figfile=update_figfile)
                     #store data beacon phase
                     if save:
                         self.save_data(self.filename_noise, noisedB, thisDatetime)
             class BeaconPhase(Figure):
                 __isConfig = None
                 __nsubplots = None
                 PREFIX = 'beacon_phase'
                 def __init__(self, **kwargs):
                     Figure.__init__(self, **kwargs)
                     self.timerange = 24*60*60
                     self.isConfig = False
                     self.__nsubplots = 1
                     self.counter_imagwr = 0
                     self.WIDTH = 800
                     self.HEIGHT = 400
                     self.WIDTHPROF = 120
                     self.HEIGHTPROF = 0
                     self.xdata = None
                     self.ydata = None
                     self.PLOT_CODE = BEACON_CODE
                     self.FTP_WEI = None
                     self.EXP_CODE = None
                     self.SUB_EXP_CODE = None
                     self.PLOT_POS = None
                     self.filename_phase = None
                     self.figfile = None
                     self.xmin = None
                     self.xmax = None
                 def getSubplots(self):
                     ncol = 1
                     nrow = 1
                     return nrow, ncol
                 def setup(self, id, nplots, wintitle, showprofile=True, show=True):
                     self.__showprofile = showprofile
                     self.nplots = nplots
                     ncolspan = 7
                     colspan = 6
                     self.__nsubplots = 2
                     self.createFigure(id = id,
                                       wintitle = wintitle,
                                       widthplot = self.WIDTH+self.WIDTHPROF,
                                       heightplot = self.HEIGHT+self.HEIGHTPROF,
                                       show=show)
                     nrow, ncol = self.getSubplots()
                     self.addAxes(nrow, ncol*ncolspan, 0, 0, colspan, 1)
                 def save_phase(self, filename_phase):
                     f = open(filename_phase,'w+')
                     f.write('\n\n')
                     f.write('JICAMARCA RADIO OBSERVATORY - Beacon Phase \n')
                     f.write('DD MM YYYY  HH MM SS   pair(2,0) pair(2,1) pair(2,3) pair(2,4)\n\n' )
                     f.close()
                 def save_data(self, filename_phase, data, data_datetime):
                     f=open(filename_phase,'a')
                     timetuple_data = data_datetime.timetuple()
                     day = str(timetuple_data.tm_mday)
                     month = str(timetuple_data.tm_mon)
                     year = str(timetuple_data.tm_year)
                     hour = str(timetuple_data.tm_hour)
                     minute = str(timetuple_data.tm_min)
                     second = str(timetuple_data.tm_sec)
                     f.write(day+' '+month+' '+year+'  '+hour+' '+minute+' '+second+'   '+str(data[0])+'   '+str(data[1])+'   '+str(data[2])+'   '+str(data[3])+'\n')
                     f.close()
                 def run(self, dataOut, id, wintitle="", pairsList=None, showprofile='True',
                         xmin=None, xmax=None, ymin=None, ymax=None, hmin=None, hmax=None,
                         timerange=None,
                         save=False, figpath='./', figfile=None, show=True, ftp=False, wr_period=1,
                         server=None, folder=None, username=None, password=None,
                         ftp_wei=0, exp_code=0, sub_exp_code=0, plot_pos=0):
                     if not isTimeInHourRange(dataOut.datatime, xmin, xmax):
                         return
                     if pairsList == None:
                         pairsIndexList = dataOut.pairsIndexList[:10]
                     else:
                         pairsIndexList = []
                         for pair in pairsList:
                             if pair not in dataOut.pairsList:
                                 raise ValueError, "Pair %s is not in dataOut.pairsList" %(pair)
                             pairsIndexList.append(dataOut.pairsList.index(pair))
                     if pairsIndexList == []:
                         return
             #         if len(pairsIndexList) > 4:
             #             pairsIndexList = pairsIndexList[0:4]
                     hmin_index = None
                     hmax_index = None
                     if hmin != None and hmax != None:
                         indexes = numpy.arange(dataOut.nHeights)
                         hmin_list = indexes[dataOut.heightList >= hmin]
                         hmax_list = indexes[dataOut.heightList <= hmax]
                         if hmin_list.any():
                             hmin_index = hmin_list[0]
                         if hmax_list.any():
                             hmax_index = hmax_list[-1]+1
                     x = dataOut.getTimeRange()
                     #y = dataOut.getHeiRange()
                     thisDatetime = dataOut.datatime
                     title = wintitle + " Signal Phase" # : %s" %(thisDatetime.strftime("%d-%b-%Y"))
                     xlabel = "Local Time"
                     ylabel = "Phase (degrees)"
                     update_figfile = False
                     nplots = len(pairsIndexList)
                     #phase = numpy.zeros((len(pairsIndexList),len(dataOut.beacon_heiIndexList)))
                     phase_beacon = numpy.zeros(len(pairsIndexList))
                     for i in range(nplots):
                         pair = dataOut.pairsList[pairsIndexList[i]]
                         ccf = numpy.average(dataOut.data_cspc[pairsIndexList[i], :, hmin_index:hmax_index], axis=0)
                         powa = numpy.average(dataOut.data_spc[pair[0], :, hmin_index:hmax_index], axis=0)
                         powb = numpy.average(dataOut.data_spc[pair[1], :, hmin_index:hmax_index], axis=0)
                         avgcoherenceComplex = ccf/numpy.sqrt(powa*powb)
                         phase = numpy.arctan2(avgcoherenceComplex.imag, avgcoherenceComplex.real)*180/numpy.pi
                         #print "Phase %d%d" %(pair[0], pair[1])
                         #print phase[dataOut.beacon_heiIndexList]
                         if dataOut.beacon_heiIndexList:
                             phase_beacon[i] = numpy.average(phase[dataOut.beacon_heiIndexList])
                         else:
                             phase_beacon[i] = numpy.average(phase)
                     if not self.isConfig:
                         nplots = len(pairsIndexList)
                         self.setup(id=id,
                                    nplots=nplots,
                                    wintitle=wintitle,
                                    showprofile=showprofile,
                                    show=show)
                         if timerange != None:
                             self.timerange = timerange
                         self.xmin, self.xmax = self.getTimeLim(x, xmin, xmax, timerange)
                         if ymin == None: ymin = 0
                         if ymax == None: ymax = 360
                         self.FTP_WEI = ftp_wei
                         self.EXP_CODE = exp_code
                         self.SUB_EXP_CODE = sub_exp_code
                         self.PLOT_POS = plot_pos
                         self.name = thisDatetime.strftime("%Y%m%d_%H%M%S")
                         self.isConfig = True
                         self.figfile = figfile
                         self.xdata = numpy.array([])
                         self.ydata = numpy.array([])
                         update_figfile = True
                         #open file beacon phase
                         path = '%s%03d' %(self.PREFIX, self.id)
                         beacon_file = os.path.join(path,'%s.txt'%self.name)
                         self.filename_phase = os.path.join(figpath,beacon_file)
                         #self.save_phase(self.filename_phase)
                     #store data beacon phase
                     #self.save_data(self.filename_phase, phase_beacon, thisDatetime)
                     self.setWinTitle(title)
                     title = "Phase Plot %s" %(thisDatetime.strftime("%Y/%m/%d %H:%M:%S"))
                     legendlabels = ["Pair (%d,%d)"%(pair[0], pair[1]) for pair in dataOut.pairsList]
                     axes = self.axesList[0]
                     self.xdata = numpy.hstack((self.xdata, x[0:1]))
                     if len(self.ydata)==0:
                         self.ydata = phase_beacon.reshape(-1,1)
                     else:
                         self.ydata = numpy.hstack((self.ydata, phase_beacon.reshape(-1,1)))
                     axes.pmultilineyaxis(x=self.xdata, y=self.ydata,
                                 xmin=self.xmin, xmax=self.xmax, ymin=ymin, ymax=ymax,
                                 xlabel=xlabel, ylabel=ylabel, title=title, legendlabels=legendlabels, marker='x', markersize=8, linestyle="solid",
                                 XAxisAsTime=True, grid='both'
                                 )
                     self.draw()
                     if dataOut.ltctime >= self.xmax:
                         self.counter_imagwr = wr_period
                         self.isConfig = False
                         update_figfile = True
                     self.save(figpath=figpath,
                               figfile=figfile,
                               save=save,
                               ftp=ftp,
                               wr_period=wr_period,
                               thisDatetime=thisDatetime,
                               update_figfile=update_figfile)

schainpy/model/graphics/jroplotter.py +1 -1

             '''
             Created on Jul 9, 2014
             @author: roj-idl71
             '''
             import os, sys
             import datetime
             import numpy
             import traceback
             from time import sleep
             from threading import Lock
             # from threading import Thread
             import schainpy
             import schainpy.admin
             from schainpy.model.proc.jroproc_base import Operation
             from schainpy.model.serializer.data import obj2Dict, dict2Obj
             from jroplot_correlation import *
             from jroplot_heispectra import *
             from jroplot_parameters import *
             from jroplot_spectra import *
             from jroplot_voltage import *
             class Plotter(Operation):
                 isConfig = None
                 name = None
                 __queue = None
                 def __init__(self, plotter_name, plotter_queue=None, **kwargs):
                     Operation.__init__(self, **kwargs)
                     self.isConfig = False
                     self.name = plotter_name
                     self.__queue = plotter_queue
                 def getSubplots(self):
                     nrow = self.nplots
                     ncol = 1
                     return nrow, ncol
                 def setup(self, **kwargs):
                     print "Initializing ..."
                 def run(self, dataOut, id=None, **kwargs):
                     """
                     Input:
                         dataOut         :
                         id              :
                     """
                     packDict = {}
                     packDict['id'] =  id
                     packDict['name'] = self.name
                     packDict['kwargs'] = kwargs
             #         packDict['data'] = obj2Dict(dataOut)
                     packDict['data'] = dataOut
                     self.__queue.put(packDict)
             # class PlotManager(Thread):
             class PlotManager():
                 __err = False
                 __stop = False
                 __realtime = False
                 controllerThreadObj = None
                 plotterList = ['Scope',
                                'SpectraPlot', 'RTIPlot',
                                'SpectraCutPlot',
                                'CrossSpectraPlot', 'CoherenceMap',
                                'PowerProfilePlot', 'Noise', 'BeaconPhase',
                                'CorrelationPlot',
                                'SpectraHeisScope', 'RTIfromSpectraHeis']
                 def __init__(self, plotter_queue):
             #         Thread.__init__(self)
             #         self.setDaemon(True)
                     self.__queue = plotter_queue
                     self.__lock = Lock()
                     self.plotInstanceDict = {}
                     self.__err = False
                     self.__stop = False
                     self.__realtime = False
                 def __handleError(self, name="", send_email=False):
                     err = traceback.format_exception(sys.exc_info()[0],
                                                      sys.exc_info()[1],
                                                      sys.exc_info()[2])
                     print "***** Error occurred in PlotManager *****"
                     print "***** [%s]: %s" %(name, err[-1])
                     message = "\nError ocurred in %s:\n" %name
                     message += "".join(err)
                     sys.stderr.write(message)
                     if not send_email:
                         return
                     import socket
                     subject =  "SChain v%s: Error running %s\n" %(schainpy.__version__, name)
                     subtitle = "%s:\n" %(name)
                     subtitle += "Hostname: %s\n" %socket.gethostbyname(socket.gethostname())
                     subtitle += "Working directory: %s\n" %os.path.abspath("./")
-            #         subtitle += "Configuration file: %s\n" %self.filename
+                    #         subtitle += "Configuration file: %s\n" %self.filename
                     subtitle += "Time: %s\n" %str(datetime.datetime.now())
                     adminObj = schainpy.admin.SchainNotify()
                     adminObj.sendAlert(message=message,
                                        subject=subject,
                                        subtitle=subtitle)
                 def run(self):
                     if self.__queue.empty():
                         return
                     if self.__err:
                         serial_data = self.__queue.get()
                         self.__queue.task_done()
                         return
                     self.__lock.acquire()
             #         if self.__queue.full():
             #             for i in range(int(self.__queue.qsize()/2)):
             #                 serial_data = self.__queue.get()
             #                 self.__queue.task_done()
                     n = int(self.__queue.qsize()/3 + 1)
                     for i in range(n):
                         if self.__queue.empty():
                             break
                         serial_data = self.__queue.get()
                         self.__queue.task_done()
                         plot_id = serial_data['id']
                         plot_name = serial_data['name']
                         kwargs = serial_data['kwargs']
             #             dataDict = serial_data['data']
             #
             #             dataPlot = dict2Obj(dataDict)
                         dataPlot = serial_data['data']
                         if plot_id not in self.plotInstanceDict.keys():
                             className = eval(plot_name)
                             self.plotInstanceDict[plot_id] = className(**kwargs)
                         plotter = self.plotInstanceDict[plot_id]
                         try:
                             plotter.run(dataPlot, plot_id, **kwargs)
                         except:
                             self.__err = True
                             self.__handleError(plot_name, send_email=True)
                             break
                     self.__lock.release()
                 def isEmpty(self):
                     return self.__queue.empty()
                 def stop(self):
                     self.__lock.acquire()
                     self.__stop = True
                     self.__lock.release()
                 def close(self):
                     self.__lock.acquire()
                     for plot_id in self.plotInstanceDict.keys():
                         plotter = self.plotInstanceDict[plot_id]
                         plotter.close()
                     self.__lock.release()
                 def setController(self, controllerThreadObj):
                     self.controllerThreadObj = controllerThreadObj
                 def start(self):
                     if not self.controllerThreadObj.isRunning():
                         raise RuntimeError, "controllerThreadObj has not been initialized. Use controllerThreadObj.start() before call this method"
                     self.join()
                 def join(self):
                     #Execute plotter while controller is running
                     while self.controllerThreadObj.isRunning():
                         self.run()
                     self.controllerThreadObj.stop()
                     #Wait until plotter queue is empty
                     while not self.isEmpty():
                         self.run()
                     self.close()
                 def isErrorDetected(self):
                     self.__lock.acquire()
                     err = self.__err
                     self.__lock.release()
                     return err

schainpy/model/io/jroIO_base.py +29 -62

             '''
             Created on Jul 2, 2014
             @author: roj-idl71
             '''
             import os
             import sys
             import glob
             import time
             import numpy
             import fnmatch
             import inspect
             import time, datetime
             import traceback
             import zmq
             try:
                 from gevent import sleep
             except:
                 from time import sleep
             from schainpy.model.data.jroheaderIO import PROCFLAG, BasicHeader, SystemHeader, RadarControllerHeader, ProcessingHeader
             from schainpy.model.data.jroheaderIO import get_dtype_index, get_numpy_dtype, get_procflag_dtype, get_dtype_width
             LOCALTIME = True
             def isNumber(cad):
                 """
                 Chequea si el conjunto de caracteres que componen un string puede ser convertidos a un numero.
                 Excepciones:
                     Si un determinado string no puede ser convertido a numero
                 Input:
                     str, string al cual se le analiza para determinar si convertible a un numero o no
                 Return:
                     True    :    si el string es uno numerico
                     False   :    no es un string numerico
                 """
                 try:
                     float( cad )
                     return True
                 except:
                     return False
             def isFileInEpoch(filename, startUTSeconds, endUTSeconds):
                 """
                 Esta funcion determina si un archivo de datos se encuentra o no dentro del rango de fecha especificado.
                 Inputs:
                     filename            :    nombre completo del archivo de datos en formato Jicamarca (.r)
                     startUTSeconds      :    fecha inicial del rango seleccionado. La fecha esta dada en
                                              segundos contados desde 01/01/1970.
                     endUTSeconds        :    fecha final del rango seleccionado. La fecha esta dada en
                                              segundos contados desde 01/01/1970.
                 Return:
                     Boolean    :    Retorna True si el archivo de datos contiene datos en el rango de
                                     fecha especificado, de lo contrario retorna False.
                 Excepciones:
                     Si el archivo no existe o no puede ser abierto
                     Si la cabecera no puede ser leida.
                 """
                 basicHeaderObj = BasicHeader(LOCALTIME)
                 try:
                     fp = open(filename,'rb')
                 except IOError:
                     print "The file %s can't be opened" %(filename)
                     return 0
                 sts = basicHeaderObj.read(fp)
                 fp.close()
                 if not(sts):
                     print "Skipping the file %s because it has not a valid header" %(filename)
                     return 0
                 if not ((startUTSeconds <= basicHeaderObj.utc) and (endUTSeconds > basicHeaderObj.utc)):
                     return 0
                 return 1
             def isTimeInRange(thisTime, startTime, endTime):
                 if endTime >= startTime:
                     if (thisTime < startTime) or (thisTime > endTime):
                         return 0
                     return 1
                 else:
                     if (thisTime < startTime) and (thisTime > endTime):
                         return 0
                     return 1
             def isFileInTimeRange(filename, startDate, endDate, startTime, endTime):
                 """
                 Retorna 1 si el archivo de datos se encuentra dentro del rango de horas especificado.
                 Inputs:
                     filename            :    nombre completo del archivo de datos en formato Jicamarca (.r)
                     startDate          :    fecha inicial del rango seleccionado en formato datetime.date
                     endDate            :    fecha final del rango seleccionado en formato datetime.date
                     startTime          :    tiempo inicial del rango seleccionado en formato datetime.time
                     endTime            :    tiempo final del rango seleccionado en formato datetime.time
                 Return:
                     Boolean    :    Retorna True si el archivo de datos contiene datos en el rango de
                                     fecha especificado, de lo contrario retorna False.
                 Excepciones:
                     Si el archivo no existe o no puede ser abierto
                     Si la cabecera no puede ser leida.
                 """
                 try:
                     fp = open(filename,'rb')
                 except IOError:
                     print "The file %s can't be opened" %(filename)
                     return None
                 firstBasicHeaderObj = BasicHeader(LOCALTIME)
                 systemHeaderObj = SystemHeader()
                 radarControllerHeaderObj = RadarControllerHeader()
                 processingHeaderObj = ProcessingHeader()
                 lastBasicHeaderObj = BasicHeader(LOCALTIME)
                 sts = firstBasicHeaderObj.read(fp)
                 if not(sts):
                     print "[Reading] Skipping the file %s because it has not a valid header" %(filename)
                     return None
                 if not systemHeaderObj.read(fp):
                     return None
                 if not radarControllerHeaderObj.read(fp):
                     return None
                 if not processingHeaderObj.read(fp):
                     return None
                 filesize = os.path.getsize(filename)
                 offset = processingHeaderObj.blockSize + 24 #header size
                 if filesize <= offset:
                     print "[Reading] %s: This file has not enough data" %filename
                     return None
                 fp.seek(-offset, 2)
                 sts = lastBasicHeaderObj.read(fp)
                 fp.close()
                 thisDatetime = lastBasicHeaderObj.datatime
                 thisTime_last_block = thisDatetime.time()
                 thisDatetime = firstBasicHeaderObj.datatime
                 thisDate = thisDatetime.date()
                 thisTime_first_block = thisDatetime.time()
                 #General case
                 #           o>>>>>>>>>>>>>><<<<<<<<<<<<<<o
                 #-----------o----------------------------o-----------
                 #       startTime                     endTime
                 if endTime >= startTime:
                     if (thisTime_last_block < startTime) or (thisTime_first_block > endTime):
                         return None
                     return thisDatetime
                 #If endTime < startTime then endTime belongs to the next day
                 #<<<<<<<<<<<o                            o>>>>>>>>>>>
                 #-----------o----------------------------o-----------
                 #        endTime                    startTime
                 if (thisDate == startDate) and (thisTime_last_block < startTime):
                     return None
                 if (thisDate == endDate) and (thisTime_first_block > endTime):
                     return None
                 if (thisTime_last_block < startTime) and (thisTime_first_block > endTime):
                     return None
                 return thisDatetime
             def isFolderInDateRange(folder, startDate=None, endDate=None):
                 """
                 Retorna 1 si el archivo de datos se encuentra dentro del rango de horas especificado.
                 Inputs:
                     folder            :    nombre completo del directorio.
                                            Su formato deberia ser "/path_root/?YYYYDDD"
                                             siendo:
                                                 YYYY    :    Anio         (ejemplo 2015)
                                                 DDD     :    Dia del anio (ejemplo 305)
                     startDate          :    fecha inicial del rango seleccionado en formato datetime.date
                     endDate            :    fecha final del rango seleccionado en formato datetime.date
                 Return:
                     Boolean    :    Retorna True si el archivo de datos contiene datos en el rango de
                                     fecha especificado, de lo contrario retorna False.
                 Excepciones:
                     Si el directorio no tiene el formato adecuado
                 """
                 basename = os.path.basename(folder)
                 if not isRadarFolder(basename):
                     print "The folder %s has not the rigth format" %folder
                     return 0
                 if startDate and endDate:
                     thisDate = getDateFromRadarFolder(basename)
                     if thisDate < startDate:
                         return 0
                     if thisDate > endDate:
                         return 0
                 return 1
             def isFileInDateRange(filename, startDate=None, endDate=None):
                 """
                 Retorna 1 si el archivo de datos se encuentra dentro del rango de horas especificado.
                 Inputs:
                     filename            :    nombre completo del archivo de datos en formato Jicamarca (.r)
                                            Su formato deberia ser "?YYYYDDDsss"
                                             siendo:
                                                 YYYY    :    Anio         (ejemplo 2015)
                                                 DDD     :    Dia del anio (ejemplo 305)
                                                 sss     :    set
                     startDate          :    fecha inicial del rango seleccionado en formato datetime.date
                     endDate            :    fecha final del rango seleccionado en formato datetime.date
                 Return:
                     Boolean    :    Retorna True si el archivo de datos contiene datos en el rango de
                                     fecha especificado, de lo contrario retorna False.
                 Excepciones:
                     Si el archivo no tiene el formato adecuado
                 """
                 basename = os.path.basename(filename)
                 if not isRadarFile(basename):
                     print "The filename %s has not the rigth format" %filename
                     return 0
                 if startDate and endDate:
                     thisDate = getDateFromRadarFile(basename)
                     if thisDate < startDate:
                         return 0
                     if thisDate > endDate:
                         return 0
                 return 1
             def getFileFromSet(path, ext, set):
                 validFilelist = []
                 fileList = os.listdir(path)
                 # 0 1234 567 89A BCDE
                 # H YYYY DDD SSS .ext
                 for thisFile in fileList:
                     try:
                         year = int(thisFile[1:5])
                         doy  = int(thisFile[5:8])
                     except:
                         continue
                     if (os.path.splitext(thisFile)[-1].lower() != ext.lower()):
                         continue
                     validFilelist.append(thisFile)
                 myfile = fnmatch.filter(validFilelist,'*%4.4d%3.3d%3.3d*'%(year,doy,set))
                 if len(myfile)!= 0:
                     return myfile[0]
                 else:
                     filename = '*%4.4d%3.3d%3.3d%s'%(year,doy,set,ext.lower())
                     print 'the filename %s does not exist'%filename
                     print '...going to the last file: '
                 if validFilelist:
                     validFilelist = sorted( validFilelist, key=str.lower )
                     return validFilelist[-1]
                 return None
             def getlastFileFromPath(path, ext):
                 """
                 Depura el fileList dejando solo los que cumplan el formato de "PYYYYDDDSSS.ext"
                 al final de la depuracion devuelve el ultimo file de la lista que quedo.
                 Input:
                     fileList    :    lista conteniendo todos los files (sin path) que componen una determinada carpeta
                     ext         :    extension de los files contenidos en una carpeta
                 Return:
                     El ultimo file de una determinada carpeta, no se considera el path.
                 """
                 validFilelist = []
                 fileList = os.listdir(path)
                 # 0 1234 567 89A BCDE
                 # H YYYY DDD SSS .ext
                 for thisFile in fileList:
                     year = thisFile[1:5]
                     if not isNumber(year):
                         continue
                     doy = thisFile[5:8]
                     if not isNumber(doy):
                         continue
                     year = int(year)
                     doy = int(doy)
                     if (os.path.splitext(thisFile)[-1].lower() != ext.lower()):
                         continue
                     validFilelist.append(thisFile)
                 if validFilelist:
                     validFilelist = sorted( validFilelist, key=str.lower )
                     return validFilelist[-1]
                 return None
             def checkForRealPath(path, foldercounter, year, doy, set, ext):
                 """
                 Por ser Linux Case Sensitive entonces checkForRealPath encuentra el nombre correcto de un path,
                 Prueba por varias combinaciones de nombres entre mayusculas y minusculas para determinar
                 el path exacto de un determinado file.
                 Example    :
                     nombre correcto del file es  .../.../D2009307/P2009307367.ext
                     Entonces la funcion prueba con las siguientes combinaciones
                         .../.../y2009307367.ext
                         .../.../Y2009307367.ext
                         .../.../x2009307/y2009307367.ext
                         .../.../x2009307/Y2009307367.ext
                         .../.../X2009307/y2009307367.ext
                         .../.../X2009307/Y2009307367.ext
                     siendo para este caso, la ultima combinacion de letras, identica al file buscado
                 Return:
                     Si encuentra la cobinacion adecuada devuelve el path completo y el nombre del file
                     caso contrario devuelve None como path y el la ultima combinacion de nombre en mayusculas
                     para el filename
                 """
                 fullfilename = None
                 find_flag = False
                 filename = None
                 prefixDirList = [None,'d','D']
                 if ext.lower() == ".r": #voltage
                     prefixFileList = ['d','D']
                 elif ext.lower() == ".pdata": #spectra
                     prefixFileList = ['p','P']
                 else:
                     return None, filename
                 #barrido por las combinaciones posibles
                 for prefixDir in prefixDirList:
                     thispath = path
                     if prefixDir != None:
                         #formo el nombre del directorio xYYYYDDD (x=d o x=D)
                         if foldercounter == 0:
                             thispath = os.path.join(path, "%s%04d%03d" % ( prefixDir, year, doy ))
                         else:
                             thispath = os.path.join(path, "%s%04d%03d_%02d" % ( prefixDir, year, doy , foldercounter))
                     for prefixFile in prefixFileList: #barrido por las dos combinaciones posibles de "D"
                         filename = "%s%04d%03d%03d%s" % ( prefixFile, year, doy, set, ext ) #formo el nombre del file xYYYYDDDSSS.ext
                         fullfilename = os.path.join( thispath, filename ) #formo el path completo
                         if os.path.exists( fullfilename ): #verifico que exista
                             find_flag = True
                             break
                     if find_flag:
                         break
                 if not(find_flag):
                     return None, filename
                 return fullfilename, filename
             def isRadarFolder(folder):
                 try:
                     year = int(folder[1:5])
                     doy = int(folder[5:8])
                 except:
                     return 0
                 return 1
             def isRadarFile(file):
                     try:
                         year = int(file[1:5])
                         doy = int(file[5:8])
                         set = int(file[8:11])
                     except:
                         return 0
                     return 1
             def getDateFromRadarFile(file):
                 try:
                     year = int(file[1:5])
                     doy = int(file[5:8])
                     set = int(file[8:11])
                 except:
                     return None
                 thisDate = datetime.date(year, 1, 1) + datetime.timedelta(doy-1)
                 return thisDate
             def getDateFromRadarFolder(folder):
                 try:
                     year = int(folder[1:5])
                     doy = int(folder[5:8])
                 except:
                     return None
                 thisDate = datetime.date(year, 1, 1) + datetime.timedelta(doy-1)
                 return thisDate
             class JRODataIO:
                 c = 3E8
                 isConfig = False
                 basicHeaderObj = None
                 systemHeaderObj = None
                 radarControllerHeaderObj = None
                 processingHeaderObj = None
                 dtype = None
                 pathList = []
                 filenameList = []
                 filename = None
                 ext = None
                 flagIsNewFile = 1
                 flagDiscontinuousBlock = 0
                 flagIsNewBlock = 0
                 fp = None
                 firstHeaderSize = 0
                 basicHeaderSize = 24
                 versionFile = 1103
                 fileSize = None
             #     ippSeconds = None
                 fileSizeByHeader = None
                 fileIndex = None
                 profileIndex = None
                 blockIndex = None
                 nTotalBlocks = None
                 maxTimeStep = 30
                 lastUTTime = None
                 datablock = None
                 dataOut = None
                 blocksize = None
                 getByBlock = False
                 def __init__(self):
                     raise NotImplementedError
                 def run(self):
                     raise NotImplementedError
                 def getDtypeWidth(self):
                     dtype_index = get_dtype_index(self.dtype)
                     dtype_width = get_dtype_width(dtype_index)
                     return dtype_width
                 def getAllowedArgs(self):
                     return inspect.getargspec(self.run).args
             class JRODataReader(JRODataIO):
                 online = 0
                 realtime = 0
                 nReadBlocks = 0
                 delay  = 10   #number of seconds waiting a new file
                 nTries  = 3  #quantity tries
                 nFiles = 3   #number of files for searching
                 path = None
                 foldercounter = 0
                 flagNoMoreFiles = 0
                 datetimeList = []
                 __isFirstTimeOnline = 1
                 __printInfo = True
                 profileIndex = None
                 nTxs = 1
                 txIndex = None
                 #Added--------------------
                 selBlocksize = None
                 selBlocktime = None
                 def __init__(self):
                     """
                     This class is used to find data files
                     Example:
                         reader = JRODataReader()
                         fileList = reader.findDataFiles()
                     """
                     pass
                 def createObjByDefault(self):
                     """
                     """
                     raise NotImplementedError
                 def getBlockDimension(self):
                     raise NotImplementedError
                 def __searchFilesOffLine(self,
                                         path,
                                         startDate=None,
                                         endDate=None,
                                         startTime=datetime.time(0,0,0),
                                         endTime=datetime.time(23,59,59),
                                         set=None,
                                         expLabel='',
                                         ext='.r',
                                         queue=None,
                                         cursor=None,
                                         skip=None,
                                         walk=True):
                     self.filenameList = []
                     self.datetimeList = []
                     pathList = []
                     dateList, pathList = self.findDatafiles(path, startDate, endDate, expLabel, ext, walk, include_path=True)
                     if dateList == []:
-            #             print "[Reading] Date range selected invalid [%s - %s]: No *%s files in %s)" %(startDate, endDate, ext, path)
+                    #             print "[Reading] Date range selected invalid [%s - %s]: No *%s files in %s)" %(startDate, endDate, ext, path)
                         return None, None
                     if len(dateList) > 1:
                         print "[Reading] Data found for date range [%s - %s]: total days = %d" %(startDate, endDate, len(dateList))
                     else:
                         print "[Reading] Data found for date range [%s - %s]: date = %s" %(startDate, endDate, dateList[0])
                     filenameList = []
                     datetimeList = []
                     for thisPath in pathList:
-            #             thisPath = pathList[pathDict[file]]
+                    #             thisPath = pathList[pathDict[file]]
                         fileList = glob.glob1(thisPath, "*%s" %ext)
                         fileList.sort()
                         skippedFileList = []
                         if cursor is not None and skip is not None:
                             # if cursor*skip > len(fileList):
                             if skip == 0:
                                 if queue is not None:
                                     queue.put(len(fileList))
                                 skippedFileList = []
                             else:
                                 skippedFileList = fileList[cursor*skip: cursor*skip + skip]
                         else:
                             skippedFileList = fileList
                         for file in skippedFileList:
                             filename = os.path.join(thisPath,file)
                             if not isFileInDateRange(filename, startDate, endDate):
                                 continue
                             thisDatetime = isFileInTimeRange(filename, startDate, endDate, startTime, endTime)
                             if not(thisDatetime):
                                 continue
                             filenameList.append(filename)
                             datetimeList.append(thisDatetime)
                     if not(filenameList):
                         print "[Reading] Time range selected invalid [%s - %s]: No *%s files in %s)" %(startTime, endTime, ext, path)
                         return None, None
                     print "[Reading] %d file(s) was(were) found in time range: %s - %s" %(len(filenameList), startTime, endTime)
                     print
                     for i in range(len(filenameList)):
                         print "[Reading] %s -> [%s]" %(filenameList[i], datetimeList[i].ctime())
                     self.filenameList = filenameList
                     self.datetimeList = datetimeList
                     return pathList, filenameList
                 def __searchFilesOnLine(self, path, expLabel = "", ext = None, walk=True, set=None):
                     """
                     Busca el ultimo archivo de la ultima carpeta (determinada o no por startDateTime) y
                     devuelve el archivo encontrado ademas de otros datos.
                     Input:
                         path           :    carpeta donde estan contenidos los files que contiene data
                         expLabel        :     Nombre del subexperimento (subfolder)
                         ext              :    extension de los files
                         walk        :    Si es habilitado no realiza busquedas dentro de los ubdirectorios (doypath)
                     Return:
                         directory   :    eL directorio donde esta el file encontrado
                         filename    :    el ultimo file de una determinada carpeta
                         year        :    el anho
                         doy         :    el numero de dia del anho
                         set         :    el set del archivo
                     """
                     if not os.path.isdir(path):
                         return None, None, None, None, None, None
                     dirList = []
                     if not walk:
                         fullpath = path
                         foldercounter = 0
                     else:
                         #Filtra solo los directorios
                         for thisPath in os.listdir(path):
                             if not os.path.isdir(os.path.join(path,thisPath)):
                                 continue
                             if not isRadarFolder(thisPath):
                                 continue
                             dirList.append(thisPath)
                         if not(dirList):
                             return None, None, None, None, None, None
                         dirList = sorted( dirList, key=str.lower )
                         doypath = dirList[-1]
                         foldercounter = int(doypath.split('_')[1]) if len(doypath.split('_'))>1 else 0
                         fullpath = os.path.join(path, doypath, expLabel)
                     print "[Reading] %s folder was found: " %(fullpath )
                     if set == None:
                         filename = getlastFileFromPath(fullpath, ext)
                     else:
                         filename = getFileFromSet(fullpath, ext, set)
                     if not(filename):
                         return None, None, None, None, None, None
                     print "[Reading] %s file was found" %(filename)
                     if not(self.__verifyFile(os.path.join(fullpath, filename))):
                         return None, None, None, None, None, None
                     year = int( filename[1:5] )
                     doy  = int( filename[5:8] )
                     set  = int( filename[8:11] )
                     return fullpath, foldercounter, filename, year, doy, set
                 def __setNextFileOffline(self):
                     idFile = self.fileIndex
                     while (True):
                         idFile += 1
                         if not(idFile < len(self.filenameList)):
                             self.flagNoMoreFiles = 1
-            #                 print "[Reading] No more Files"
+                    #                 print "[Reading] No more Files"
                             return 0
                         filename = self.filenameList[idFile]
                         if not(self.__verifyFile(filename)):
                             continue
                         fileSize = os.path.getsize(filename)
                         fp = open(filename,'rb')
                         break
                     self.flagIsNewFile = 1
                     self.fileIndex = idFile
                     self.filename = filename
                     self.fileSize = fileSize
                     self.fp = fp
-            #         print "[Reading] Setting the file: %s"%self.filename
+                    #         print "[Reading] Setting the file: %s"%self.filename
                     return 1
                 def __setNextFileOnline(self):
                     """
                     Busca el siguiente file que tenga suficiente data para ser leida, dentro de un folder especifico, si
                     no encuentra un file valido espera un tiempo determinado y luego busca en los posibles n files
                     siguientes.
                     Affected:
                         self.flagIsNewFile
                         self.filename
                         self.fileSize
                         self.fp
                         self.set
                         self.flagNoMoreFiles
                     Return:
 : si luego de una busqueda del siguiente file valido este no pudo ser encontrado
 : si el file fue abierto con exito y esta listo a ser leido
                     Excepciones:
                         Si un determinado file no puede ser abierto
                     """
                     nFiles = 0
                     fileOk_flag = False
                     firstTime_flag = True
                     self.set += 1
                     if self.set > 999:
                         self.set = 0
                         self.foldercounter += 1
                     #busca el 1er file disponible
                     fullfilename, filename = checkForRealPath( self.path, self.foldercounter, self.year, self.doy, self.set, self.ext )
                     if fullfilename:
                         if self.__verifyFile(fullfilename, False):
                             fileOk_flag = True
                     #si no encuentra un file entonces espera y vuelve a buscar
                     if not(fileOk_flag):
                         for nFiles in range(self.nFiles+1): #busco en los siguientes self.nFiles+1 files posibles
                             if firstTime_flag: #si es la 1era vez entonces hace el for self.nTries veces
                                 tries = self.nTries
                             else:
                                 tries = 1 #si no es la 1era vez entonces solo lo hace una vez
                             for nTries in range( tries ):
                                 if firstTime_flag:
                                     print "\t[Reading] Waiting %0.2f sec for the next file: \"%s\" , try %03d ..." % ( self.delay, filename, nTries+1 )
                                     sleep( self.delay )
                                 else:
                                     print "\t[Reading] Searching the next \"%s%04d%03d%03d%s\" file ..." % (self.optchar, self.year, self.doy, self.set, self.ext)
                                 fullfilename, filename = checkForRealPath( self.path, self.foldercounter, self.year, self.doy, self.set, self.ext )
                                 if fullfilename:
                                     if self.__verifyFile(fullfilename):
                                         fileOk_flag = True
                                         break
                             if fileOk_flag:
                                 break
                             firstTime_flag = False
                             print "\t[Reading] Skipping the file \"%s\" due to this file doesn't exist" % filename
                             self.set += 1
                             if nFiles == (self.nFiles-1): #si no encuentro el file buscado cambio de carpeta y busco en la siguiente carpeta
                                 self.set = 0
                                 self.doy += 1
                                 self.foldercounter = 0
                     if fileOk_flag:
                         self.fileSize = os.path.getsize( fullfilename )
                         self.filename = fullfilename
                         self.flagIsNewFile = 1
                         if self.fp != None: self.fp.close()
                         self.fp = open(fullfilename, 'rb')
                         self.flagNoMoreFiles = 0
-            #             print '[Reading] Setting the file: %s' % fullfilename
+                    #             print '[Reading] Setting the file: %s' % fullfilename
                     else:
                         self.fileSize = 0
                         self.filename = None
                         self.flagIsNewFile = 0
                         self.fp = None
                         self.flagNoMoreFiles = 1
-            #             print '[Reading] No more files to read'
+                    #             print '[Reading] No more files to read'
                     return fileOk_flag
                 def setNextFile(self):
                     if self.fp != None:
                         self.fp.close()
                     if self.online:
                         newFile = self.__setNextFileOnline()
                     else:
                         newFile = self.__setNextFileOffline()
                     if not(newFile):
                         print '[Reading] No more files to read'
                         return 0
                     if self.verbose:
                         print '[Reading] Setting the file: %s' % self.filename
                     self.__readFirstHeader()
                     self.nReadBlocks = 0
                     return 1
                 def __waitNewBlock(self):
                     """
                     Return 1 si se encontro un nuevo bloque de datos, 0 de otra forma.
                     Si el modo de lectura es OffLine siempre retorn 0
                     """
                     if not self.online:
                         return 0
                     if (self.nReadBlocks >= self.processingHeaderObj.dataBlocksPerFile):
                         return 0
                     currentPointer = self.fp.tell()
                     neededSize = self.processingHeaderObj.blockSize + self.basicHeaderSize
                     for nTries in range( self.nTries ):
                         self.fp.close()
                         self.fp = open( self.filename, 'rb' )
                         self.fp.seek( currentPointer )
                         self.fileSize = os.path.getsize( self.filename )
                         currentSize = self.fileSize - currentPointer
                         if ( currentSize >= neededSize ):
                             self.basicHeaderObj.read(self.fp)
                             return 1
                         if self.fileSize == self.fileSizeByHeader:
-            #                self.flagEoF = True
+                    #                self.flagEoF = True
                             return 0
                         print "[Reading] Waiting %0.2f seconds for the next block, try %03d ..." % (self.delay, nTries+1)
                         sleep( self.delay )
                     return 0
                 def waitDataBlock(self,pointer_location):
                     currentPointer = pointer_location
                     neededSize = self.processingHeaderObj.blockSize #+ self.basicHeaderSize
                     for nTries in range( self.nTries ):
                         self.fp.close()
                         self.fp = open( self.filename, 'rb' )
                         self.fp.seek( currentPointer )
                         self.fileSize = os.path.getsize( self.filename )
                         currentSize = self.fileSize - currentPointer
                         if ( currentSize >= neededSize ):
                             return 1
                         print "[Reading] Waiting %0.2f seconds for the next block, try %03d ..." % (self.delay, nTries+1)
                         sleep( self.delay )
                     return 0
                 def __jumpToLastBlock(self):
                     if not(self.__isFirstTimeOnline):
                         return
                     csize = self.fileSize - self.fp.tell()
                     blocksize = self.processingHeaderObj.blockSize
                     #salta el primer bloque de datos
                     if csize > self.processingHeaderObj.blockSize:
                         self.fp.seek(self.fp.tell() + blocksize)
                     else:
                         return
                     csize = self.fileSize - self.fp.tell()
                     neededsize = self.processingHeaderObj.blockSize + self.basicHeaderSize
                     while True:
                         if self.fp.tell()<self.fileSize:
                             self.fp.seek(self.fp.tell() + neededsize)
                         else:
                             self.fp.seek(self.fp.tell() - neededsize)
                             break
-            #        csize = self.fileSize - self.fp.tell()
+                    #        csize = self.fileSize - self.fp.tell()
-            #        neededsize = self.processingHeaderObj.blockSize + self.basicHeaderSize
+                    #        neededsize = self.processingHeaderObj.blockSize + self.basicHeaderSize
-            #        factor = int(csize/neededsize)
+                    #        factor = int(csize/neededsize)
-            #        if factor > 0:
+                    #        if factor > 0:
-            #            self.fp.seek(self.fp.tell() + factor*neededsize)
+                    #            self.fp.seek(self.fp.tell() + factor*neededsize)
                     self.flagIsNewFile = 0
                     self.__isFirstTimeOnline = 0
                 def __setNewBlock(self):
                     #if self.server is None:
                     if self.fp == None:
                         return 0
-            #         if self.online:
+                    #         if self.online:
-            #             self.__jumpToLastBlock()
+                    #             self.__jumpToLastBlock()
                     if self.flagIsNewFile:
                         self.lastUTTime = self.basicHeaderObj.utc
                         return 1
                     if self.realtime:
                         self.flagDiscontinuousBlock = 1
                         if not(self.setNextFile()):
                             return 0
                         else:
                             return 1
                     #if self.server is None:
                     currentSize = self.fileSize - self.fp.tell()
                     neededSize = self.processingHeaderObj.blockSize + self.basicHeaderSize
                     if (currentSize >= neededSize):
                         self.basicHeaderObj.read(self.fp)
                         self.lastUTTime = self.basicHeaderObj.utc
                         return 1
                     # else:
                     #     self.basicHeaderObj.read(self.zHeader)
                     #     self.lastUTTime = self.basicHeaderObj.utc
                     #     return 1
                     if self.__waitNewBlock():
                         self.lastUTTime = self.basicHeaderObj.utc
                         return 1
                     #if self.server is None:
                     if not(self.setNextFile()):
                         return 0
                     deltaTime = self.basicHeaderObj.utc - self.lastUTTime #
                     self.lastUTTime = self.basicHeaderObj.utc
                     self.flagDiscontinuousBlock = 0
                     if deltaTime > self.maxTimeStep:
                         self.flagDiscontinuousBlock = 1
                     return 1
                 def readNextBlock(self):
                     #Skip block out of startTime and endTime
                     while True:
                         if not(self.__setNewBlock()):
                             print 'returning'
                             return 0
                         if not(self.readBlock()):
                             return 0
                         self.getBasicHeader()
                         if not isTimeInRange(self.dataOut.datatime.time(), self.startTime, self.endTime):
                             print "[Reading] Block No. %d/%d -> %s [Skipping]" %(self.nReadBlocks,
                                                                           self.processingHeaderObj.dataBlocksPerFile,
                                                                           self.dataOut.datatime.ctime())
                             continue
                         break
                     if self.verbose:
                         print "[Reading] Block No. %d/%d -> %s" %(self.nReadBlocks,
                                                                   self.processingHeaderObj.dataBlocksPerFile,
                                                                   self.dataOut.datatime.ctime())
                     return 1
                 def __readFirstHeader(self):
                     self.basicHeaderObj.read(self.fp)
                     self.systemHeaderObj.read(self.fp)
                     self.radarControllerHeaderObj.read(self.fp)
                     self.processingHeaderObj.read(self.fp)
                     self.firstHeaderSize = self.basicHeaderObj.size
                     datatype = int(numpy.log2((self.processingHeaderObj.processFlags & PROCFLAG.DATATYPE_MASK))-numpy.log2(PROCFLAG.DATATYPE_CHAR))
                     if datatype == 0:
                         datatype_str = numpy.dtype([('real','<i1'),('imag','<i1')])
                     elif datatype == 1:
                         datatype_str = numpy.dtype([('real','<i2'),('imag','<i2')])
                     elif datatype == 2:
                         datatype_str = numpy.dtype([('real','<i4'),('imag','<i4')])
                     elif datatype == 3:
                         datatype_str = numpy.dtype([('real','<i8'),('imag','<i8')])
                     elif datatype == 4:
                         datatype_str = numpy.dtype([('real','<f4'),('imag','<f4')])
                     elif datatype == 5:
                         datatype_str = numpy.dtype([('real','<f8'),('imag','<f8')])
                     else:
                         raise ValueError, 'Data type was not defined'
                     self.dtype = datatype_str
                     #self.ippSeconds = 2 * 1000 * self.radarControllerHeaderObj.ipp / self.c
                     self.fileSizeByHeader = self.processingHeaderObj.dataBlocksPerFile * self.processingHeaderObj.blockSize + self.firstHeaderSize + self.basicHeaderSize*(self.processingHeaderObj.dataBlocksPerFile - 1)
-            #        self.dataOut.channelList = numpy.arange(self.systemHeaderObj.numChannels)
+                    #        self.dataOut.channelList = numpy.arange(self.systemHeaderObj.numChannels)
-            #        self.dataOut.channelIndexList = numpy.arange(self.systemHeaderObj.numChannels)
+                    #        self.dataOut.channelIndexList = numpy.arange(self.systemHeaderObj.numChannels)
                     self.getBlockDimension()
                 def __verifyFile(self, filename, msgFlag=True):
                     msg = None
                     try:
                         fp = open(filename, 'rb')
                     except IOError:
                         if msgFlag:
                             print "[Reading] File %s can't be opened" % (filename)
                         return False
                     currentPosition = fp.tell()
                     neededSize = self.processingHeaderObj.blockSize + self.firstHeaderSize
                     if neededSize == 0:
                         basicHeaderObj = BasicHeader(LOCALTIME)
                         systemHeaderObj = SystemHeader()
                         radarControllerHeaderObj = RadarControllerHeader()
                         processingHeaderObj = ProcessingHeader()
                         if not( basicHeaderObj.read(fp) ):
                             fp.close()
                             return False
                         if not( systemHeaderObj.read(fp) ):
                             fp.close()
                             return False
                         if not( radarControllerHeaderObj.read(fp) ):
                             fp.close()
                             return False
                         if not( processingHeaderObj.read(fp) ):
                             fp.close()
                             return False
                         neededSize = processingHeaderObj.blockSize + basicHeaderObj.size
                     else:
                         msg = "[Reading] Skipping the file %s due to it hasn't enough data" %filename
                     fp.close()
                     fileSize = os.path.getsize(filename)
                     currentSize = fileSize - currentPosition
                     if currentSize < neededSize:
                         if msgFlag and (msg != None):
                             print msg
                         return False
                     return True
                 def findDatafiles(self, path, startDate=None, endDate=None, expLabel='', ext='.r', walk=True, include_path=False):
                     path_empty = True
                     dateList = []
                     pathList = []
                     multi_path = path.split(',')
                     if not walk:
                         for single_path in multi_path:
                             if not os.path.isdir(single_path):
                                 continue
                             fileList = glob.glob1(single_path, "*"+ext)
                             if not fileList:
                                 continue
                             path_empty = False
                             fileList.sort()
                             for thisFile in fileList:
                                 if not os.path.isfile(os.path.join(single_path, thisFile)):
                                     continue
                                 if not isRadarFile(thisFile):
                                     continue
                                 if not isFileInDateRange(thisFile, startDate, endDate):
                                     continue
                                 thisDate = getDateFromRadarFile(thisFile)
                                 if thisDate in dateList:
                                     continue
                                 dateList.append(thisDate)
                                 pathList.append(single_path)
                     else:
                         for single_path in multi_path:
                             if not os.path.isdir(single_path):
                                 continue
                             dirList = []
                             for thisPath in os.listdir(single_path):
                                 if not os.path.isdir(os.path.join(single_path,thisPath)):
                                     continue
                                 if not isRadarFolder(thisPath):
                                     continue
                                 if not isFolderInDateRange(thisPath, startDate, endDate):
                                     continue
                                 dirList.append(thisPath)
                             if not dirList:
                                 continue
                             dirList.sort()
                             for thisDir in dirList:
                                 datapath = os.path.join(single_path, thisDir, expLabel)
                                 fileList = glob.glob1(datapath, "*"+ext)
                                 if not fileList:
                                     continue
                                 path_empty = False
                                 thisDate = getDateFromRadarFolder(thisDir)
                                 pathList.append(datapath)
                                 dateList.append(thisDate)
                     dateList.sort()
                     if walk:
                         pattern_path = os.path.join(multi_path[0], "[dYYYYDDD]", expLabel)
                     else:
                         pattern_path = multi_path[0]
                     if path_empty:
                         print "[Reading] No *%s files in %s for %s to %s" %(ext, pattern_path, startDate, endDate)
                     else:
                         if not dateList:
                             print "[Reading] Date range selected invalid [%s - %s]: No *%s files in %s)" %(startDate, endDate, ext, path)
                     if include_path:
                         return dateList, pathList
                     return dateList
                 def setup(self,
                             path=None,
                             startDate=None,
                             endDate=None,
                             startTime=datetime.time(0,0,0),
                             endTime=datetime.time(23,59,59),
                             set=None,
                             expLabel = "",
                             ext = None,
                             online = False,
                             delay = 60,
                             walk = True,
                             getblock = False,
                             nTxs = 1,
                             realtime=False,
                             blocksize=None,
                             blocktime=None,
                             queue=None,
                             skip=None,
                             cursor=None,
                             warnings=True,
                             verbose=True,
                             server=None):
                     if server is not None:
                         if 'tcp://' in server:
                             address = server
                         else:
                             address = 'ipc:///tmp/%s' % server
                         self.server = address
                         self.context = zmq.Context()
                         self.receiver = self.context.socket(zmq.PULL)
                         self.receiver.connect(self.server)
                         time.sleep(0.5)
                         print '[Starting] ReceiverData from {}'.format(self.server)
                     else:
                         self.server = None
                         if path == None:
                             raise ValueError, "[Reading] The path is not valid"
                         if ext == None:
                             ext = self.ext
                         if online:
                             print "[Reading] Searching files in online mode..."
                             for nTries in range( self.nTries ):
                                 fullpath, foldercounter, file, year, doy, set = self.__searchFilesOnLine(path=path, expLabel=expLabel, ext=ext, walk=walk, set=set)
                                 if fullpath:
                                     break
                                 print '[Reading] Waiting %0.2f sec for an valid file in %s: try %02d ...' % (self.delay, path, nTries+1)
                                 sleep( self.delay )
                             if not(fullpath):
                                 print "[Reading] There 'isn't any valid file in %s" % path
                                 return
                             self.year = year
                             self.doy  = doy
                             self.set  = set - 1
                             self.path = path
                             self.foldercounter = foldercounter
                             last_set = None
                         else:
                             print "[Reading] Searching files in offline mode ..."
                             pathList, filenameList = self.__searchFilesOffLine(path, startDate=startDate, endDate=endDate,
                                                                             startTime=startTime, endTime=endTime,
                                                                             set=set, expLabel=expLabel, ext=ext,
                                                                             walk=walk, cursor=cursor,
                                                                             skip=skip, queue=queue)
                             if not(pathList):
                         #      print "[Reading] No *%s files in %s (%s - %s)"%(ext, path,
                         #                                          datetime.datetime.combine(startDate,startTime).ctime(),
                         #                                           datetime.datetime.combine(endDate,endTime).ctime())
                         #     sys.exit(-1)
                                 self.fileIndex = -1
                                 self.pathList = []
                                 self.filenameList = []
                                 return
                             self.fileIndex = -1
-                            self.pathList = pathList
+                            self.pathList = []
-                            self.filenameList = filenameList
+                            self.filenameList = []
-                            file_name = os.path.basename(filenameList[-1])
+                            return
-                            basename, ext = os.path.splitext(file_name)
-                            last_set = int(basename[-3:])
+                    #         self.getBasicHeader()
-                        self.online = online
+                    if last_set != None:
-                        self.realtime = realtime
+                        self.dataOut.last_block = last_set * self.processingHeaderObj.dataBlocksPerFile + self.basicHeaderObj.dataBlock
-                        self.delay = delay
-                        ext = ext.lower()
-                        self.ext = ext
-                        self.getByBlock = getblock
-                        self.nTxs = nTxs
-                        self.startTime = startTime
-                        self.endTime = endTime
-                        #Added-----------------
-                        self.selBlocksize = blocksize
-                        self.selBlocktime = blocktime
-                        # Verbose-----------
-                        self.verbose = verbose
-                        self.warnings = warnings
-                        if not(self.setNextFile()):
-                            if (startDate!=None) and (endDate!=None):
-                                print "[Reading] No files in range: %s - %s" %(datetime.datetime.combine(startDate,startTime).ctime(), datetime.datetime.combine(endDate,endTime).ctime())
-                            elif startDate != None:
-                                print "[Reading] No files in range: %s" %(datetime.datetime.combine(startDate,startTime).ctime())
-                            else:
-                                print "[Reading] No files"
-                                self.fileIndex = -1
-                                self.pathList = []
-                                self.filenameList = []
-                                return
-                #         self.getBasicHeader()
-                        if last_set != None:
-                            self.dataOut.last_block = last_set * self.processingHeaderObj.dataBlocksPerFile + self.basicHeaderObj.dataBlock
                     return
                 def getBasicHeader(self):
                     self.dataOut.utctime = self.basicHeaderObj.utc + self.basicHeaderObj.miliSecond/1000. + self.profileIndex * self.radarControllerHeaderObj.ippSeconds
                     self.dataOut.flagDiscontinuousBlock = self.flagDiscontinuousBlock
                     self.dataOut.timeZone = self.basicHeaderObj.timeZone
                     self.dataOut.dstFlag = self.basicHeaderObj.dstFlag
                     self.dataOut.errorCount = self.basicHeaderObj.errorCount
                     self.dataOut.useLocalTime = self.basicHeaderObj.useLocalTime
                     self.dataOut.ippSeconds = self.radarControllerHeaderObj.ippSeconds/self.nTxs
             #         self.dataOut.nProfiles = self.processingHeaderObj.profilesPerBlock*self.nTxs
                 def getFirstHeader(self):
                     raise NotImplementedError
                 def getData(self):
                     raise NotImplementedError
                 def hasNotDataInBuffer(self):
                     raise NotImplementedError
                 def readBlock(self):
                     raise NotImplementedError
                 def isEndProcess(self):
                     return self.flagNoMoreFiles
                 def printReadBlocks(self):
                     print "[Reading] Number of read blocks per file %04d" %self.nReadBlocks
                 def printTotalBlocks(self):
                     print "[Reading] Number of read blocks %04d" %self.nTotalBlocks
                 def printNumberOfBlock(self):
                     if self.flagIsNewBlock:
                         print "[Reading] Block No. %d/%d -> %s" %(self.nReadBlocks,
                                                                   self.processingHeaderObj.dataBlocksPerFile,
                                                                   self.dataOut.datatime.ctime())
                 def printInfo(self):
                     if self.__printInfo == False:
                         return
                     self.basicHeaderObj.printInfo()
                     self.systemHeaderObj.printInfo()
                     self.radarControllerHeaderObj.printInfo()
                     self.processingHeaderObj.printInfo()
                     self.__printInfo = False
                 def run(self,
                             path=None,
                             startDate=None,
                             endDate=None,
                             startTime=datetime.time(0,0,0),
                             endTime=datetime.time(23,59,59),
                             set=None,
                             expLabel = "",
                             ext = None,
                             online = False,
                             delay = 60,
                             walk = True,
                             getblock = False,
                             nTxs = 1,
                             realtime=False,
                             blocksize=None,
                             blocktime=None,
                             queue=None,
                             skip=None,
                             cursor=None,
                             warnings=True,
                             server=None,
                             verbose=True, **kwargs):
                     if not(self.isConfig):
             #            self.dataOut = dataOut
                         self.setup( path=path,
                                     startDate=startDate,
                                     endDate=endDate,
                                     startTime=startTime,
                                     endTime=endTime,
                                     set=set,
                                     expLabel=expLabel,
                                     ext=ext,
                                     online=online,
                                     delay=delay,
                                     walk=walk,
                                     getblock=getblock,
                                     nTxs=nTxs,
                                     realtime=realtime,
                                     blocksize=blocksize,
                                     blocktime=blocktime,
                                     queue=queue,
                                     skip=skip,
                                     cursor=cursor,
                                     warnings=warnings,
                                     server=server,
                                     verbose=verbose)
                         self.isConfig = True
                     if server is None:
                         self.getData()
                     else:
                         self.getFromServer()
             class JRODataWriter(JRODataIO):
                 """
                 Esta clase permite escribir datos a archivos procesados (.r o ,pdata). La escritura
                 de los datos siempre se realiza por bloques.
                 """
                 blockIndex = 0
                 path = None
                 setFile = None
                 profilesPerBlock = None
                 blocksPerFile = None
                 nWriteBlocks = 0
                 fileDate = None
                 def __init__(self, dataOut=None):
                     raise NotImplementedError
                 def hasAllDataInBuffer(self):
                     raise NotImplementedError
                 def setBlockDimension(self):
                     raise NotImplementedError
                 def writeBlock(self):
                     raise NotImplementedError
                 def putData(self):
                     raise NotImplementedError
                 def getProcessFlags(self):
                     processFlags = 0
                     dtype_index = get_dtype_index(self.dtype)
                     procflag_dtype = get_procflag_dtype(dtype_index)
                     processFlags += procflag_dtype
                     if self.dataOut.flagDecodeData:
                         processFlags += PROCFLAG.DECODE_DATA
                     if self.dataOut.flagDeflipData:
                         processFlags += PROCFLAG.DEFLIP_DATA
                     if self.dataOut.code is not None:
                         processFlags += PROCFLAG.DEFINE_PROCESS_CODE
                     if self.dataOut.nCohInt > 1:
                         processFlags += PROCFLAG.COHERENT_INTEGRATION
                     if self.dataOut.type == "Spectra":
                         if self.dataOut.nIncohInt > 1:
                             processFlags += PROCFLAG.INCOHERENT_INTEGRATION
                         if self.dataOut.data_dc is not None:
                             processFlags += PROCFLAG.SAVE_CHANNELS_DC
                         if self.dataOut.flagShiftFFT:
                             processFlags += PROCFLAG.SHIFT_FFT_DATA
                     return processFlags
                 def setBasicHeader(self):
                     self.basicHeaderObj.size = self.basicHeaderSize #bytes
                     self.basicHeaderObj.version = self.versionFile
                     self.basicHeaderObj.dataBlock = self.nTotalBlocks
                     utc = numpy.floor(self.dataOut.utctime)
                     milisecond  = (self.dataOut.utctime - utc)* 1000.0
                     self.basicHeaderObj.utc = utc
                     self.basicHeaderObj.miliSecond = milisecond
                     self.basicHeaderObj.timeZone = self.dataOut.timeZone
                     self.basicHeaderObj.dstFlag = self.dataOut.dstFlag
                     self.basicHeaderObj.errorCount = self.dataOut.errorCount
                 def setFirstHeader(self):
                     """
                     Obtiene una copia del First Header
                     Affected:
                         self.basicHeaderObj
                         self.systemHeaderObj
                         self.radarControllerHeaderObj
                         self.processingHeaderObj self.
                     Return:
                         None
                     """
                     raise NotImplementedError
                 def __writeFirstHeader(self):
                     """
                     Escribe el primer header del file es decir el Basic header y el Long header (SystemHeader, RadarControllerHeader, ProcessingHeader)
                     Affected:
                         __dataType
                     Return:
                         None
                     """
             #        CALCULAR PARAMETROS
                     sizeLongHeader = self.systemHeaderObj.size + self.radarControllerHeaderObj.size + self.processingHeaderObj.size
                     self.basicHeaderObj.size = self.basicHeaderSize + sizeLongHeader
                     self.basicHeaderObj.write(self.fp)
                     self.systemHeaderObj.write(self.fp)
                     self.radarControllerHeaderObj.write(self.fp)
                     self.processingHeaderObj.write(self.fp)
                 def __setNewBlock(self):
                     """
                     Si es un nuevo file escribe el First Header caso contrario escribe solo el Basic Header
                     Return:
 :    si no pudo escribir nada
 :    Si escribio el Basic el First Header
                     """
                     if self.fp == None:
                         self.setNextFile()
                     if self.flagIsNewFile:
                         return 1
                     if self.blockIndex < self.processingHeaderObj.dataBlocksPerFile:
                         self.basicHeaderObj.write(self.fp)
                         return 1
                     if not( self.setNextFile() ):
                         return 0
                     return 1
                 def writeNextBlock(self):
                     """
                     Selecciona el bloque siguiente de datos y los escribe en un file
                     Return:
 :    Si no hizo pudo escribir el bloque de datos
 :    Si no pudo escribir el bloque de datos
                     """
                     if not( self.__setNewBlock() ):
                         return 0
                     self.writeBlock()
                     print "[Writing] Block No. %d/%d" %(self.blockIndex,
                                                         self.processingHeaderObj.dataBlocksPerFile)
                     return 1
                 def setNextFile(self):
                     """
                     Determina el siguiente file que sera escrito
                     Affected:
                         self.filename
                         self.subfolder
                         self.fp
                         self.setFile
                         self.flagIsNewFile
                     Return:
 :    Si el archivo no puede ser escrito
 :    Si el archivo esta listo para ser escrito
                     """
                     ext = self.ext
                     path = self.path
                     if self.fp != None:
                         self.fp.close()
                     timeTuple = time.localtime( self.dataOut.utctime)
                     subfolder = 'd%4.4d%3.3d' % (timeTuple.tm_year,timeTuple.tm_yday)
                     fullpath = os.path.join( path, subfolder )
                     setFile = self.setFile
                     if not( os.path.exists(fullpath) ):
                         os.mkdir(fullpath)
                         setFile = -1 #inicializo mi contador de seteo
                     else:
                         filesList = os.listdir( fullpath )
                         if len( filesList ) > 0:
                             filesList = sorted( filesList, key=str.lower )
                             filen = filesList[-1]
                             # el filename debera tener el siguiente formato
                             # 0 1234 567 89A BCDE (hex)
                             # x YYYY DDD SSS .ext
                             if isNumber( filen[8:11] ):
                                 setFile = int( filen[8:11] ) #inicializo mi contador de seteo al seteo del ultimo file
                             else:
                                 setFile = -1
                         else:
                             setFile = -1 #inicializo mi contador de seteo
                     setFile += 1
                     #If this is a new day it resets some values
                     if self.dataOut.datatime.date() > self.fileDate:
                         setFile = 0
                         self.nTotalBlocks = 0
                     filen = '%s%4.4d%3.3d%3.3d%s' % (self.optchar, timeTuple.tm_year, timeTuple.tm_yday, setFile, ext )
                     filename = os.path.join( path, subfolder, filen )
                     fp = open( filename,'wb' )
                     self.blockIndex = 0
                     #guardando atributos
                     self.filename = filename
                     self.subfolder = subfolder
                     self.fp = fp
                     self.setFile = setFile
                     self.flagIsNewFile = 1
                     self.fileDate = self.dataOut.datatime.date()
                     self.setFirstHeader()
                     print '[Writing] Opening file: %s'%self.filename
                     self.__writeFirstHeader()
                     return 1
                 def setup(self, dataOut, path, blocksPerFile, profilesPerBlock=64, set=None, ext=None, datatype=4):
                     """
                     Setea el tipo de formato en la cual sera guardada la data y escribe el First Header
                     Inputs:
                         path                :    directory where data will be saved
                         profilesPerBlock    :    number of profiles per block
                         set                 :    initial file set
                         datatype            :    An integer number that defines data type:
 : int8  (1 byte)
 : int16 (2 bytes)
 : int32 (4 bytes)
 : int64 (8 bytes)
 : float32 (4 bytes)
 : double64 (8 bytes)
                     Return:
 :    Si no realizo un buen seteo
 :    Si realizo un buen seteo
                     """
                     if ext == None:
                         ext = self.ext
                     self.ext = ext.lower()
                     self.path = path
                     if set is None:
                         self.setFile = -1
                     else:
                         self.setFile = set - 1
                     self.blocksPerFile = blocksPerFile
                     self.profilesPerBlock = profilesPerBlock
                     self.dataOut = dataOut
                     self.fileDate = self.dataOut.datatime.date()
                     #By default
                     self.dtype = self.dataOut.dtype
                     if datatype is not None:
                         self.dtype = get_numpy_dtype(datatype)
                     if not(self.setNextFile()):
                         print "[Writing] There isn't a next file"
                         return 0
                     self.setBlockDimension()
                     return 1
                 def run(self, dataOut, path, blocksPerFile, profilesPerBlock=64, set=None, ext=None, datatype=4, **kwargs):
                     if not(self.isConfig):
                         self.setup(dataOut, path, blocksPerFile, profilesPerBlock=profilesPerBlock, set=set, ext=ext, datatype=datatype, **kwargs)
                         self.isConfig = True
                     self.putData()

schainpy/model/io/jroIO_digitalRF.py +56 -34

             '''
             Created on Jul 3, 2014
             @author: roj-idl71
             '''
             # SUBCHANNELS EN VEZ DE CHANNELS
             # BENCHMARKS -> PROBLEMAS CON ARCHIVOS GRANDES -> INCONSTANTE EN EL TIEMPO
             # ACTUALIZACION DE VERSION
             # HEADERS
             # MODULO DE ESCRITURA
             # METADATA
             import os
             import datetime
             import numpy
             import timeit
             from profilehooks import coverage, profile
             from fractions import Fraction
             try:
                 from gevent import sleep
             except:
                 from time import sleep
             from schainpy.model.data.jroheaderIO import RadarControllerHeader, SystemHeader
             from schainpy.model.data.jrodata import Voltage
             from schainpy.model.proc.jroproc_base import ProcessingUnit, Operation
             from time import time
             import cPickle
             try:
                 import digital_rf
             except:
                 print 'You should install "digital_rf" module if you want to read Digital RF data'
             class DigitalRFReader(ProcessingUnit):
                 '''
                 classdocs
                 '''
                 def __init__(self, **kwargs):
                     '''
                     Constructor
                     '''
                     ProcessingUnit.__init__(self, **kwargs)
                     self.dataOut = Voltage()
                     self.__printInfo = True
                     self.__flagDiscontinuousBlock = False
                     self.__bufferIndex = 9999999
                     self.__ippKm = None
                     self.__codeType = 0
                     self.__nCode = None
                     self.__nBaud = None
                     self.__code = None
+                    self.dtype = None
                 def close(self):
                     print 'Average of writing to digital rf format is ', self.oldAverage * 1000
                     return
                 def __getCurrentSecond(self):
                     return self.__thisUnixSample/self.__sample_rate
                 thisSecond = property(__getCurrentSecond, "I'm the 'thisSecond' property.")
                 def __setFileHeader(self):
                     '''
                     In this method will be initialized every parameter of dataOut object (header, no data)
                     '''
                     ippSeconds = 1.0*self.__nSamples/self.__sample_rate
                     nProfiles = 1.0/ippSeconds  # Number of profiles in one second
-                    self.dataOut.radarControllerHeaderObj = RadarControllerHeader(self.__radarControllerHeader)
+                    try:
+                        self.dataOut.radarControllerHeaderObj = RadarControllerHeader(self.__radarControllerHeader)
-                    self.dataOut.systemHeaderObj = SystemHeader(self.__systemHeader)
+                    except:
+                        self.dataOut.radarControllerHeaderObj = RadarControllerHeader(
+                                                                                    txA=0,
+                                                                                    txB=0,
+                                                                                    nWindows=1,
+                                                                                    nHeights=self.__nSamples,
+                                                                                    firstHeight=self.__firstHeigth,
+                                                                                    deltaHeight=self.__deltaHeigth,
+                                                                                    codeType=self.__codeType,
+                                                                                    nCode=self.__nCode, nBaud=self.__nBaud,
+                                                                                    code = self.__code)
+                    try:
+                        self.dataOut.systemHeaderObj = SystemHeader(self.__systemHeader)
+                    except:
+                        self.dataOut.systemHeaderObj = SystemHeader(nSamples=self.__nSamples,
+                                                                    nProfiles=nProfiles,
+                                                                    nChannels=len(self.__channelList),
+                                                                    adcResolution=14)
                     self.dataOut.type = "Voltage"
                     self.dataOut.data = None
                     self.dataOut.dtype = self.dtype
                     # self.dataOut.nChannels = 0
                     # self.dataOut.nHeights = 0
                     self.dataOut.nProfiles = nProfiles
                     self.dataOut.heightList = self.__firstHeigth + numpy.arange(self.__nSamples, dtype = numpy.float)*self.__deltaHeigth
                     self.dataOut.channelList = range(self.__num_subchannels)
                     self.dataOut.blocksize = self.dataOut.getNChannels() * self.dataOut.getNHeights()
                     # self.dataOut.channelIndexList = None
                     self.dataOut.flagNoData = True
                     self.dataOut.flagDataAsBlock = False
                     # Set to TRUE if the data is discontinuous
                     self.dataOut.flagDiscontinuousBlock = False
                     self.dataOut.utctime = None
                     self.dataOut.timeZone = self.__timezone/60  # timezone like jroheader, difference in minutes between UTC and localtime
                     self.dataOut.dstFlag = 0
                     self.dataOut.errorCount = 0
-                    self.dataOut.nCohInt = self.fixed_metadata_dict['nCohInt']
+                    try:
+                        self.dataOut.nCohInt = self.fixed_metadata_dict.get('nCohInt', 1)
-                    self.dataOut.flagDecodeData = self.fixed_metadata_dict['flagDecodeData'] # asumo que la data esta decodificada
+                        self.dataOut.flagDecodeData = self.fixed_metadata_dict['flagDecodeData'] # asumo que la data esta decodificada
-                    self.dataOut.flagDeflipData = self.fixed_metadata_dict['flagDeflipData'] # asumo que la data esta sin flip
+                        self.dataOut.flagDeflipData = self.fixed_metadata_dict['flagDeflipData'] # asumo que la data esta sin flip
-                    self.dataOut.flagShiftFFT = self.fixed_metadata_dict['flagShiftFFT']
+                        self.dataOut.flagShiftFFT = self.fixed_metadata_dict['flagShiftFFT']
-                    self.dataOut.useLocalTime = self.fixed_metadata_dict['useLocalTime']
+                        self.dataOut.useLocalTime = self.fixed_metadata_dict['useLocalTime']
+                    except:
+                        pass
                     self.dataOut.ippSeconds = ippSeconds
                     # Time interval between profiles
                     # self.dataOut.timeInterval = self.dataOut.ippSeconds * self.dataOut.nCohInt
                     self.dataOut.frequency = self.__frequency
                     self.dataOut.realtime = self.__online
                 def findDatafiles(self, path, startDate=None, endDate=None):
                     if not os.path.isdir(path):
                         return []
                     try:
                         digitalReadObj = digital_rf.DigitalRFReader(path, load_all_metadata=True)
                     except:
                         digitalReadObj = digital_rf.DigitalRFReader(path)
                     channelNameList = digitalReadObj.get_channels()
                     if not channelNameList:
                         return []
                     metadata_dict = digitalReadObj.get_rf_file_metadata(channelNameList[0])
                     sample_rate = metadata_dict['sample_rate'][0]
                     this_metadata_file = digitalReadObj.get_metadata(channelNameList[0])
                     try:
                         timezone = this_metadata_file['timezone'].value
                     except:
                         timezone = 0
                     startUTCSecond, endUTCSecond = digitalReadObj.get_bounds(channelNameList[0])/sample_rate - timezone
                     startDatetime = datetime.datetime.utcfromtimestamp(startUTCSecond)
                     endDatatime = datetime.datetime.utcfromtimestamp(endUTCSecond)
                     if not startDate:
                         startDate = startDatetime.date()
                     if not endDate:
                         endDate = endDatatime.date()
                     dateList = []
                     thisDatetime = startDatetime
                     while(thisDatetime<=endDatatime):
                         thisDate = thisDatetime.date()
                         if thisDate < startDate:
                             continue
                         if thisDate > endDate:
                             break
                         dateList.append(thisDate)
                         thisDatetime += datetime.timedelta(1)
                     return dateList
                 def setup(self, path = None,
                                 startDate = None,
                                 endDate = None,
                                 startTime = datetime.time(0,0,0),
                                 endTime = datetime.time(23,59,59),
                                 channelList = None,
                                 nSamples = None,
                                 online = False,
                                 delay = 60,
                                 buffer_size = 1024,
                                 ippKm=None,
                                 **kwargs):
                     '''
                     In this method we should set all initial parameters.
                     Inputs:
                         path
                         startDate
                         endDate
                         startTime
                         endTime
                         set
                         expLabel
                         ext
                         online
                         delay
                     '''
                     self.i = 0
                     if not os.path.isdir(path):
                         raise ValueError, "[Reading] Directory %s does not exist" %path
                     try:
                         self.digitalReadObj = digital_rf.DigitalRFReader(path, load_all_metadata=True)
                     except:
                         self.digitalReadObj = digital_rf.DigitalRFReader(path)
                     channelNameList = self.digitalReadObj.get_channels()
                     if not channelNameList:
                         raise ValueError, "[Reading] Directory %s does not have any files" %path
                     if not channelList:
                         channelList = range(len(channelNameList))
                     ##########  Reading metadata ######################
                     top_properties = self.digitalReadObj.get_properties(channelNameList[channelList[0]])
                     self.__num_subchannels = top_properties['num_subchannels']
                     self.__sample_rate = 1.0 * top_properties['sample_rate_numerator'] / top_properties['sample_rate_denominator']
                     # self.__samples_per_file = top_properties['samples_per_file'][0]
                     self.__deltaHeigth = 1e6*0.15/self.__sample_rate ## why 0.15?
                     this_metadata_file = self.digitalReadObj.get_digital_metadata(channelNameList[channelList[0]])
                     metadata_bounds = this_metadata_file.get_bounds()
                     self.fixed_metadata_dict = this_metadata_file.read(metadata_bounds[0])[metadata_bounds[0]] ## GET FIRST HEADER
-                    self.__processingHeader = self.fixed_metadata_dict['processingHeader']
-                    self.__radarControllerHeader = self.fixed_metadata_dict['radarControllerHeader']
+                    try:
-                    self.__systemHeader = self.fixed_metadata_dict['systemHeader']
+                        self.__processingHeader = self.fixed_metadata_dict['processingHeader']
-                    self.dtype = cPickle.loads(self.fixed_metadata_dict['dtype'])
+                        self.__radarControllerHeader = self.fixed_metadata_dict['radarControllerHeader']
+                        self.__systemHeader = self.fixed_metadata_dict['systemHeader']
+                        self.dtype = cPickle.loads(self.fixed_metadata_dict['dtype'])
+                    except:
+                        pass
                     self.__frequency = None
-                    try:
+                    self.__frequency = self.fixed_metadata_dict.get('frequency', 1)
-                        self.__frequency = self.fixed_metadata_dict['frequency']
-                    except:
-                        self.__frequency = None
-                    try:
+                    self.__timezone = self.fixed_metadata_dict.get('timezone', 300)
-                        self.__timezone = self.fixed_metadata_dict['timezone'] * 60
-                    except:
-                        self.__timezone = 0
                     try:
                         nSamples = self.fixed_metadata_dict['nSamples']
                     except:
                         nSamples = None
                     self.__firstHeigth = 0
                     try:
                         codeType = self.__radarControllerHeader['codeType']
                     except:
                         codeType = 0
                     nCode = 1
                     nBaud = 1
                     code = numpy.ones((nCode, nBaud), dtype=numpy.int)
-                    if codeType:
+                    try:
-                        nCode = self.__radarControllerHeader['nCode']
+                        if codeType:
-                        nBaud = self.__radarControllerHeader['nBaud']
+                            nCode = self.__radarControllerHeader['nCode']
-                        code = self.__radarControllerHeader['code']
+                            nBaud = self.__radarControllerHeader['nBaud']
+                            code = self.__radarControllerHeader['code']
+                    except:
+                        pass
                     if not ippKm:
                         try:
                             # seconds to km
                             ippKm = self.__radarControllerHeader['ipp']
                         except:
                             ippKm = None
                     ####################################################
+                    self.__ippKm = ippKm
                     startUTCSecond = None
                     endUTCSecond = None
                     if startDate:
                         startDatetime = datetime.datetime.combine(startDate, startTime)
                         startUTCSecond = (startDatetime-datetime.datetime(1970,1,1)).total_seconds() + self.__timezone
                     if endDate:
                         endDatetime = datetime.datetime.combine(endDate, endTime)
                         endUTCSecond = (endDatetime-datetime.datetime(1970,1,1)).total_seconds() + self.__timezone
                     start_index, end_index = self.digitalReadObj.get_bounds(channelNameList[channelList[0]])
                     if not startUTCSecond:
                         startUTCSecond = start_index/self.__sample_rate
                     if start_index > startUTCSecond*self.__sample_rate:
                         startUTCSecond = start_index/self.__sample_rate
                     if not endUTCSecond:
                         endUTCSecond = end_index/self.__sample_rate
                     if end_index < endUTCSecond*self.__sample_rate:
                         endUTCSecond = end_index/self.__sample_rate
                     if not nSamples:
                         if not ippKm:
                             raise ValueError, "[Reading] nSamples or ippKm should be defined"
                         nSamples = int(ippKm / (1e6*0.15/self.__sample_rate))
                     channelBoundList = []
                     channelNameListFiltered = []
                     for thisIndexChannel in channelList:
                         thisChannelName =  channelNameList[thisIndexChannel]
                         start_index, end_index = self.digitalReadObj.get_bounds(thisChannelName)
                         channelBoundList.append((start_index, end_index))
                         channelNameListFiltered.append(thisChannelName)
                     self.profileIndex = 0
                     self.i= 0
                     self.__delay = delay
-                    self.__ippKm = ippKm
                     self.__codeType = codeType
                     self.__nCode = nCode
                     self.__nBaud = nBaud
                     self.__code = code
                     self.__datapath = path
                     self.__online = online
                     self.__channelList = channelList
                     self.__channelNameList = channelNameListFiltered
                     self.__channelBoundList = channelBoundList
                     self.__nSamples = nSamples
                     self.__samples_to_read = long(nSamples) # FIJO: AHORA 40
                     self.__nChannels = len(self.__channelList)
                     self.__startUTCSecond = startUTCSecond
                     self.__endUTCSecond = endUTCSecond
                     self.__timeInterval = 1.0 * self.__samples_to_read/self.__sample_rate # Time interval
                     if online:
                     # self.__thisUnixSample = int(endUTCSecond*self.__sample_rate - 4*self.__samples_to_read)
                         startUTCSecond = numpy.floor(endUTCSecond)
                     self.__thisUnixSample = long(startUTCSecond*self.__sample_rate) - self.__samples_to_read ## por que en el otro metodo lo primero q se hace es sumar samplestoread
                     self.__data_buffer = numpy.zeros((self.__num_subchannels, self.__samples_to_read), dtype = numpy.complex)
                     self.__setFileHeader()
                     self.isConfig = True
                     print "[Reading] Digital RF Data was found from %s to %s " %(
                                                                   datetime.datetime.utcfromtimestamp(self.__startUTCSecond - self.__timezone),
                                                                   datetime.datetime.utcfromtimestamp(self.__endUTCSecond - self.__timezone)
                                                                   )
                     print "[Reading] Starting process from %s to %s" %(datetime.datetime.utcfromtimestamp(startUTCSecond - self.__timezone),
                                                                        datetime.datetime.utcfromtimestamp(endUTCSecond - self.__timezone)
                                                                        )
                     self.oldAverage = None
                     self.count = 0
                     self.executionTime = 0
                 def __reload(self):
                     #         print
                     #         print "%s not in range [%s, %s]" %(
                     #                                           datetime.datetime.utcfromtimestamp(self.thisSecond - self.__timezone),
                     #                                           datetime.datetime.utcfromtimestamp(self.__startUTCSecond - self.__timezone),
                     #                                           datetime.datetime.utcfromtimestamp(self.__endUTCSecond - self.__timezone)
                     #                                           )
                     print "[Reading] reloading metadata ..."
                     try:
                         self.digitalReadObj.reload(complete_update=True)
                     except:
                         self.digitalReadObj.reload()
                     start_index, end_index = self.digitalReadObj.get_bounds(self.__channelNameList[self.__channelList[0]])
                     if start_index > self.__startUTCSecond*self.__sample_rate:
                         self.__startUTCSecond = 1.0*start_index/self.__sample_rate
                     if end_index > self.__endUTCSecond*self.__sample_rate:
                         self.__endUTCSecond = 1.0*end_index/self.__sample_rate
                         print
                         print "[Reading] New timerange found [%s, %s] " %(
                                                                   datetime.datetime.utcfromtimestamp(self.__startUTCSecond - self.__timezone),
                                                                   datetime.datetime.utcfromtimestamp(self.__endUTCSecond - self.__timezone)
                                                                   )
                         return True
                     return False
                 def timeit(self, toExecute):
                     t0 = time()
                     toExecute()
                     self.executionTime = time() - t0
                     if self.oldAverage is None: self.oldAverage = self.executionTime
                     self.oldAverage = (self.executionTime + self.count*self.oldAverage) / (self.count + 1.0)
                     self.count = self.count + 1.0
                     return
                 def __readNextBlock(self, seconds=30, volt_scale = 1):
                     '''
                     '''
                     # Set the next data
                     self.__flagDiscontinuousBlock = False
                     self.__thisUnixSample += self.__samples_to_read
                     if self.__thisUnixSample + 2*self.__samples_to_read > self.__endUTCSecond*self.__sample_rate:
                         print "[Reading] There are no more data into selected time-range"
                         if self.__online:
                             self.__reload()
                         else:
                             return False
                         if self.__thisUnixSample + 2*self.__samples_to_read > self.__endUTCSecond*self.__sample_rate:
                             return False
                             self.__thisUnixSample -=  self.__samples_to_read
                     indexChannel = 0
                     dataOk = False
                     for thisChannelName in self.__channelNameList: ##TODO VARIOS CHANNELS?
                         for indexSubchannel in range(self.__num_subchannels):
                             try:
                                 t0 = time()
                                 result = self.digitalReadObj.read_vector_c81d(self.__thisUnixSample,
                                                                               self.__samples_to_read,
                                                                               thisChannelName, sub_channel=indexSubchannel)
                                 self.executionTime = time() - t0
                                 if self.oldAverage is None: self.oldAverage = self.executionTime
                                 self.oldAverage = (self.executionTime + self.count*self.oldAverage) / (self.count + 1.0)
                                 self.count = self.count + 1.0
                             except IOError, e:
                                 #read next profile
                                 self.__flagDiscontinuousBlock = True
                                 print "[Reading] %s" %datetime.datetime.utcfromtimestamp(self.thisSecond - self.__timezone), e
                                 break
                             if result.shape[0] != self.__samples_to_read:
                                 self.__flagDiscontinuousBlock = True
                                 print "[Reading] %s: Too few samples were found, just %d/%d  samples" %(datetime.datetime.utcfromtimestamp(self.thisSecond - self.__timezone),
                                                                                                 result.shape[0],
                                                                                                 self.__samples_to_read)
                                 break
                             self.__data_buffer[indexSubchannel,:] = result*volt_scale
                             indexChannel += 1
                             dataOk = True
                     self.__utctime = self.__thisUnixSample/self.__sample_rate
                     if not dataOk:
                         return False
                     print "[Reading] %s: %d samples <> %f sec" %(datetime.datetime.utcfromtimestamp(self.thisSecond - self.__timezone),
                                                                  self.__samples_to_read,
                                                                  self.__timeInterval)
                     self.__bufferIndex = 0
                     return True
                 def __isBufferEmpty(self):
                     return self.__bufferIndex > self.__samples_to_read - self.__nSamples #40960 - 40
                 def getData(self, seconds=30, nTries=5):
                     '''
                         This method gets the data from files and put the data into the dataOut object
                         In addition, increase el the buffer counter in one.
                         Return:
                             data    :    retorna un perfil de voltages (alturas * canales) copiados desde el
                                         buffer. Si no hay mas archivos a leer retorna None.
                         Affected:
                             self.dataOut
                             self.profileIndex
                             self.flagDiscontinuousBlock
                             self.flagIsNewBlock
                     '''
                     err_counter = 0
                     self.dataOut.flagNoData = True
                     if self.__isBufferEmpty():
                         self.__flagDiscontinuousBlock = False
                         while True:
                             if self.__readNextBlock():
                                 break
                             if self.__thisUnixSample > self.__endUTCSecond*self.__sample_rate:
                                 return False
                             if self.__flagDiscontinuousBlock:
                                 print '[Reading] discontinuous block found ... continue with the next block'
                                 continue
                             if not self.__online:
                                 return False
                             err_counter += 1
                             if err_counter > nTries:
                                 return False
                             print '[Reading] waiting %d seconds to read a new block' %seconds
                             sleep(seconds)
                     self.dataOut.data = self.__data_buffer[:,self.__bufferIndex:self.__bufferIndex+self.__nSamples]
                     self.dataOut.utctime = (self.__thisUnixSample + self.__bufferIndex)/self.__sample_rate
                     self.dataOut.flagNoData = False
                     self.dataOut.flagDiscontinuousBlock = self.__flagDiscontinuousBlock
                     self.dataOut.profileIndex = self.profileIndex
                     self.__bufferIndex += self.__nSamples
                     self.profileIndex += 1
                     if self.profileIndex == self.dataOut.nProfiles:
                         self.profileIndex = 0
                     return True
                 def printInfo(self):
                     '''
                     '''
                     if self.__printInfo == False:
                         return
                     # self.systemHeaderObj.printInfo()
                     # self.radarControllerHeaderObj.printInfo()
                     self.__printInfo = False
                 def printNumberOfBlock(self):
                     '''
                     '''
                     return
                     # print self.profileIndex
                 def run(self, **kwargs):
                     '''
                     This method will be called many times so here you should put all your code
                     '''
                     if not self.isConfig:
                         self.setup(**kwargs)
                     #self.i = self.i+1
                     self.getData(seconds=self.__delay)
                     return
             class DigitalRFWriter(Operation):
                 '''
                 classdocs
                 '''
                 def __init__(self, **kwargs):
                     '''
                     Constructor
                     '''
                     Operation.__init__(self, **kwargs)
                     self.metadata_dict = {}
                     self.dataOut = None
+                    self.dtype = None
                 def setHeader(self):
                     self.metadata_dict['frequency'] = self.dataOut.frequency
                     self.metadata_dict['timezone'] = self.dataOut.timeZone
                     self.metadata_dict['dtype'] = cPickle.dumps(self.dataOut.dtype)
                     self.metadata_dict['nProfiles'] = self.dataOut.nProfiles
                     self.metadata_dict['heightList'] = self.dataOut.heightList
                     self.metadata_dict['channelList'] = self.dataOut.channelList
                     self.metadata_dict['flagDecodeData'] = self.dataOut.flagDecodeData
                     self.metadata_dict['flagDeflipData'] = self.dataOut.flagDeflipData
                     self.metadata_dict['flagShiftFFT'] = self.dataOut.flagShiftFFT
                     self.metadata_dict['flagDataAsBlock'] = self.dataOut.flagDataAsBlock
                     self.metadata_dict['useLocalTime'] = self.dataOut.useLocalTime
                     self.metadata_dict['nCohInt'] = self.dataOut.nCohInt
                     return
                 def setup(self, dataOut, path, frequency, fileCadence, dirCadence, metadataCadence, set=0, metadataFile='metadata', ext='.h5'):
                     '''
                     In this method we should set all initial parameters.
                     Input:
                         dataOut: Input data will also be outputa data
                     '''
                     self.setHeader()
                     self.__ippSeconds = dataOut.ippSeconds
                     self.__deltaH = dataOut.getDeltaH()
                     self.__sample_rate = 1e6*0.15/self.__deltaH
                     self.__dtype = dataOut.dtype
                     if len(dataOut.dtype) == 2:
                         self.__dtype = dataOut.dtype[0]
                     self.__nSamples = dataOut.systemHeaderObj.nSamples
                     self.__nProfiles = dataOut.nProfiles
                     self.__blocks_per_file = dataOut.processingHeaderObj.dataBlocksPerFile
                     self.arr_data = arr_data = numpy.ones((self.__nSamples, len(self.dataOut.channelList)), dtype=[('r', self.__dtype), ('i', self.__dtype)])
                     file_cadence_millisecs = long(1.0 * self.__blocks_per_file * self.__nProfiles * self.__nSamples / self.__sample_rate) * 1000
                     sub_cadence_secs = file_cadence_millisecs / 500
                     sample_rate_fraction = Fraction(self.__sample_rate).limit_denominator()
                     sample_rate_numerator = long(sample_rate_fraction.numerator)
                     sample_rate_denominator = long(sample_rate_fraction.denominator)
                     start_global_index = dataOut.utctime * self.__sample_rate
                     uuid = 'prueba'
                     compression_level = 1
                     checksum = False
                     is_complex = True
                     num_subchannels = len(dataOut.channelList)
                     is_continuous = True
                     marching_periods = False
                     self.digitalWriteObj = digital_rf.DigitalRFWriter(path, self.__dtype, dirCadence,
                                                             fileCadence, start_global_index,
                                                             sample_rate_numerator, sample_rate_denominator, uuid, compression_level, checksum,
                                                             is_complex, num_subchannels, is_continuous, marching_periods)
                     metadata_dir = os.path.join(path, 'metadata')
                     os.system('mkdir %s' % (metadata_dir))
                     self.digitalMetadataWriteObj = digital_rf.DigitalMetadataWriter(metadata_dir, dirCadence, 1, ##236, file_cadence_millisecs / 1000
                                                             sample_rate_numerator, sample_rate_denominator,
                                                             metadataFile)
                     self.isConfig = True
                     self.currentSample = 0
                     self.oldAverage = 0
                     self.count = 0
                     return
                 def writeMetadata(self):
                     print '[Writing] - Writing metadata'
                     start_idx = self.__sample_rate * self.dataOut.utctime
                     self.metadata_dict['processingHeader'] = self.dataOut.processingHeaderObj.getAsDict()
                     self.metadata_dict['radarControllerHeader'] = self.dataOut.radarControllerHeaderObj.getAsDict()
                     self.metadata_dict['systemHeader'] = self.dataOut.systemHeaderObj.getAsDict()
                     self.digitalMetadataWriteObj.write(start_idx, self.metadata_dict)
                     return
                 def timeit(self, toExecute):
                     t0 = time()
                     toExecute()
                     self.executionTime = time() - t0
                     if self.oldAverage is None: self.oldAverage = self.executionTime
                     self.oldAverage = (self.executionTime + self.count*self.oldAverage) / (self.count + 1.0)
                     self.count = self.count + 1.0
                     return
                 def writeData(self):
                     for i in range(self.dataOut.systemHeaderObj.nSamples):
                         for channel in self.dataOut.channelList:
                             self.arr_data[i][channel]['r'] = self.dataOut.data[channel][i].real
                             self.arr_data[i][channel]['i'] = self.dataOut.data[channel][i].imag
                     def f(): return self.digitalWriteObj.rf_write(self.arr_data)
                     self.timeit(f)
                     return
                 def run(self, dataOut, frequency=49.92e6, path=None, fileCadence=100, dirCadence=25, metadataCadence=1, **kwargs):
                     '''
                     This method will be called many times so here you should put all your code
                     Inputs:
                         dataOut: object with the data
                     '''
                     # print dataOut.__dict__
                     self.dataOut = dataOut
                     if not self.isConfig:
                         self.setup(dataOut, path, frequency, fileCadence, dirCadence, metadataCadence, **kwargs)
                         self.writeMetadata()
                     self.writeData()
                     ## self.currentSample += 1
                     ## if self.dataOut.flagDataAsBlock or self.currentSample == 1:
                         ## self.writeMetadata()
                     ## if self.currentSample == self.__nProfiles: self.currentSample = 0
                 def close(self):
                     print '[Writing] - Closing files '
                     print 'Average of writing to digital rf format is ', self.oldAverage * 1000
                     try:
                         self.digitalWriteObj.close()
                     except:
                         pass
                     # raise
             if __name__ == '__main__':
                 readObj = DigitalRFReader()
                 while True:
                     readObj.run(path='/home/jchavez/jicamarca/mocked_data/')
                     # readObj.printInfo()
                     # readObj.printNumberOfBlock()

schainpy/model/proc/jroproc_spectra.py +1 -2

             import numpy
             from jroproc_base import ProcessingUnit, Operation
             from schainpy.model.data.jrodata import Spectra
             from schainpy.model.data.jrodata import hildebrand_sekhon
             class SpectraProc(ProcessingUnit):
                 def __init__(self, **kwargs):
                     ProcessingUnit.__init__(self, **kwargs)
                     self.buffer = None
                     self.firstdatatime = None
                     self.profIndex = 0
                     self.dataOut = Spectra()
                     self.id_min = None
                     self.id_max = None
                 def __updateSpecFromVoltage(self):
                     self.dataOut.timeZone = self.dataIn.timeZone
                     self.dataOut.dstFlag = self.dataIn.dstFlag
                     self.dataOut.errorCount = self.dataIn.errorCount
                     self.dataOut.useLocalTime = self.dataIn.useLocalTime
                     self.dataOut.radarControllerHeaderObj = self.dataIn.radarControllerHeaderObj.copy()
                     self.dataOut.systemHeaderObj = self.dataIn.systemHeaderObj.copy()
                     self.dataOut.channelList = self.dataIn.channelList
                     self.dataOut.heightList = self.dataIn.heightList
                     self.dataOut.dtype = numpy.dtype([('real','<f4'),('imag','<f4')])
                     self.dataOut.nBaud = self.dataIn.nBaud
                     self.dataOut.nCode = self.dataIn.nCode
                     self.dataOut.code = self.dataIn.code
                     self.dataOut.nProfiles = self.dataOut.nFFTPoints
                     self.dataOut.flagDiscontinuousBlock = self.dataIn.flagDiscontinuousBlock
                     self.dataOut.utctime = self.firstdatatime
                     self.dataOut.flagDecodeData = self.dataIn.flagDecodeData #asumo q la data esta decodificada
                     self.dataOut.flagDeflipData = self.dataIn.flagDeflipData #asumo q la data esta sin flip
                     self.dataOut.flagShiftFFT = False
                     self.dataOut.nCohInt = self.dataIn.nCohInt
                     self.dataOut.nIncohInt = 1
                     self.dataOut.windowOfFilter = self.dataIn.windowOfFilter
                     self.dataOut.frequency = self.dataIn.frequency
                     self.dataOut.realtime = self.dataIn.realtime
                     self.dataOut.azimuth = self.dataIn.azimuth
                     self.dataOut.zenith = self.dataIn.zenith
                     self.dataOut.beam.codeList = self.dataIn.beam.codeList
                     self.dataOut.beam.azimuthList = self.dataIn.beam.azimuthList
                     self.dataOut.beam.zenithList = self.dataIn.beam.zenithList
                 def __getFft(self):
                     """
                     Convierte valores de Voltaje a Spectra
                     Affected:
                         self.dataOut.data_spc
                         self.dataOut.data_cspc
                         self.dataOut.data_dc
                         self.dataOut.heightList
                         self.profIndex
                         self.buffer
                         self.dataOut.flagNoData
                     """
                     fft_volt = numpy.fft.fft(self.buffer,n=self.dataOut.nFFTPoints,axis=1)
                     fft_volt = fft_volt.astype(numpy.dtype('complex'))
                     dc = fft_volt[:,0,:]
                     #calculo de self-spectra
                     fft_volt = numpy.fft.fftshift(fft_volt,axes=(1,))
                     spc = fft_volt * numpy.conjugate(fft_volt)
                     spc = spc.real
                     blocksize = 0
                     blocksize += dc.size
                     blocksize += spc.size
                     cspc = None
                     pairIndex = 0
                     if self.dataOut.pairsList != None:
                         #calculo de cross-spectra
                         cspc = numpy.zeros((self.dataOut.nPairs, self.dataOut.nFFTPoints, self.dataOut.nHeights), dtype='complex')
                         for pair in self.dataOut.pairsList:
                             if pair[0] not in self.dataOut.channelList:
                                 raise ValueError, "Error getting CrossSpectra: pair 0 of %s is not in channelList = %s" %(str(pair), str(self.dataOut.channelList))
                             if pair[1] not in self.dataOut.channelList:
                                 raise ValueError, "Error getting CrossSpectra: pair 1 of %s is not in channelList = %s" %(str(pair), str(self.dataOut.channelList))
                             cspc[pairIndex,:,:] = fft_volt[pair[0],:,:] * numpy.conjugate(fft_volt[pair[1],:,:])
                             pairIndex += 1
                         blocksize += cspc.size
                     self.dataOut.data_spc = spc
                     self.dataOut.data_cspc = cspc
                     self.dataOut.data_dc = dc
                     self.dataOut.blockSize = blocksize
                     self.dataOut.flagShiftFFT = True
                 def run(self, nProfiles=None, nFFTPoints=None, pairsList=[], ippFactor=None):
                     self.dataOut.flagNoData = True
                     if self.dataIn.type == "Spectra":
                         self.dataOut.copy(self.dataIn)
-            #             self.__selectPairs(pairsList)
+                        #             self.__selectPairs(pairsList)
                         return True
                     if self.dataIn.type == "Voltage":
                         if nFFTPoints == None:
                             raise ValueError, "This SpectraProc.run() need nFFTPoints input variable"
                         if nProfiles == None:
                             nProfiles = nFFTPoints
                         if ippFactor == None:
                             ippFactor = 1
                         self.dataOut.ippFactor = ippFactor
                         self.dataOut.nFFTPoints = nFFTPoints
                         self.dataOut.pairsList = pairsList
                         if self.buffer is None:
                             self.buffer = numpy.zeros( (self.dataIn.nChannels,
                                                         nProfiles,
                                                         self.dataIn.nHeights),
                                                       dtype='complex')
                         if self.dataIn.flagDataAsBlock:
                             #data dimension: [nChannels, nProfiles, nSamples]
                             nVoltProfiles = self.dataIn.data.shape[1]
             #                 nVoltProfiles = self.dataIn.nProfiles
                             if nVoltProfiles == nProfiles:
                                 self.buffer = self.dataIn.data.copy()
                                 self.profIndex = nVoltProfiles
                             elif nVoltProfiles < nProfiles:
                                 if self.profIndex == 0:
                                     self.id_min = 0
                                     self.id_max = nVoltProfiles
                                 self.buffer[:,self.id_min:self.id_max,:] = self.dataIn.data
                                 self.profIndex += nVoltProfiles
                                 self.id_min += nVoltProfiles
                                 self.id_max += nVoltProfiles
                             else:
                                 raise ValueError, "The type object %s has %d profiles, it should just has %d profiles"%(self.dataIn.type,self.dataIn.data.shape[1],nProfiles)
                                 self.dataOut.flagNoData = True
                                 return 0
                         else:
                             self.buffer[:,self.profIndex,:] = self.dataIn.data.copy()
                             self.profIndex += 1
                         if self.firstdatatime == None:
                             self.firstdatatime = self.dataIn.utctime
                         if self.profIndex == nProfiles:
                             self.__updateSpecFromVoltage()
                             self.__getFft()
                             self.dataOut.flagNoData = False
                             self.firstdatatime = None
                             self.profIndex = 0
                         return True
                     raise ValueError, "The type of input object '%s' is not valid"%(self.dataIn.type)
                 def __selectPairs(self, pairsList):
                     if channelList == None:
                         return
                     pairsIndexListSelected = []
                     for thisPair in pairsList:
                         if thisPair not in self.dataOut.pairsList:
                             continue
                         pairIndex = self.dataOut.pairsList.index(thisPair)
                         pairsIndexListSelected.append(pairIndex)
                     if not pairsIndexListSelected:
                         self.dataOut.data_cspc = None
                         self.dataOut.pairsList = []
                         return
                     self.dataOut.data_cspc = self.dataOut.data_cspc[pairsIndexListSelected]
                     self.dataOut.pairsList = [self.dataOut.pairsList[i] for i in pairsIndexListSelected]
                     return
                 def __selectPairsByChannel(self, channelList=None):
                     if channelList == None:
                         return
                     pairsIndexListSelected = []
                     for pairIndex in self.dataOut.pairsIndexList:
                         #First pair
                         if self.dataOut.pairsList[pairIndex][0] not in channelList:
                             continue
                         #Second pair
                         if self.dataOut.pairsList[pairIndex][1] not in channelList:
                             continue
                         pairsIndexListSelected.append(pairIndex)
                     if not pairsIndexListSelected:
                         self.dataOut.data_cspc = None
                         self.dataOut.pairsList = []
                         return
                     self.dataOut.data_cspc = self.dataOut.data_cspc[pairsIndexListSelected]
                     self.dataOut.pairsList = [self.dataOut.pairsList[i] for i in pairsIndexListSelected]
                     return
                 def selectChannels(self, channelList):
                     channelIndexList = []
                     for channel in channelList:
                         if channel not in self.dataOut.channelList:
                             raise ValueError, "Error selecting channels, Channel %d is not valid.\nAvailable channels = %s" %(channel, str(self.dataOut.channelList))
                         index = self.dataOut.channelList.index(channel)
                         channelIndexList.append(index)
                     self.selectChannelsByIndex(channelIndexList)
                 def selectChannelsByIndex(self, channelIndexList):
                     """
                     Selecciona un bloque de datos en base a canales segun el channelIndexList
                     Input:
                         channelIndexList    :    lista sencilla de canales a seleccionar por ej. [2,3,7]
                     Affected:
                         self.dataOut.data_spc
                         self.dataOut.channelIndexList
                         self.dataOut.nChannels
                     Return:
                         None
                     """
                     for channelIndex in channelIndexList:
                         if channelIndex not in self.dataOut.channelIndexList:
                             raise ValueError, "Error selecting channels: The value %d in channelIndexList is not valid.\nAvailable channel indexes = " %(channelIndex, self.dataOut.channelIndexList)
             #         nChannels = len(channelIndexList)
                     data_spc = self.dataOut.data_spc[channelIndexList,:]
                     data_dc = self.dataOut.data_dc[channelIndexList,:]
                     self.dataOut.data_spc = data_spc
                     self.dataOut.data_dc = data_dc
                     self.dataOut.channelList = [self.dataOut.channelList[i] for i in channelIndexList]
             #        self.dataOut.nChannels = nChannels
                     self.__selectPairsByChannel(self.dataOut.channelList)
                     return 1
                 def selectHeights(self, minHei, maxHei):
                     """
                     Selecciona un bloque de datos en base a un grupo de valores de alturas segun el rango
                     minHei <= height <= maxHei
                     Input:
                         minHei    :    valor minimo de altura a considerar
                         maxHei    :    valor maximo de altura a considerar
                     Affected:
                         Indirectamente son cambiados varios valores a travez del metodo selectHeightsByIndex
                     Return:
 si el metodo se ejecuto con exito caso contrario devuelve 0
                     """
                     if (minHei > maxHei):
                         raise ValueError, "Error selecting heights: Height range (%d,%d) is not valid" % (minHei, maxHei)
                     if (minHei < self.dataOut.heightList[0]):
                         minHei = self.dataOut.heightList[0]
                     if (maxHei > self.dataOut.heightList[-1]):
                         maxHei = self.dataOut.heightList[-1]
                     minIndex = 0
                     maxIndex = 0
                     heights = self.dataOut.heightList
                     inda = numpy.where(heights >= minHei)
                     indb = numpy.where(heights <= maxHei)
                     try:
                         minIndex = inda[0][0]
                     except:
                         minIndex = 0
                     try:
                         maxIndex = indb[0][-1]
                     except:
                         maxIndex = len(heights)
                     self.selectHeightsByIndex(minIndex, maxIndex)
                     return 1
                 def getBeaconSignal(self, tauindex = 0, channelindex = 0, hei_ref=None):
                     newheis = numpy.where(self.dataOut.heightList>self.dataOut.radarControllerHeaderObj.Taus[tauindex])
                     if hei_ref != None:
                         newheis = numpy.where(self.dataOut.heightList>hei_ref)
                     minIndex = min(newheis[0])
                     maxIndex = max(newheis[0])
                     data_spc = self.dataOut.data_spc[:,:,minIndex:maxIndex+1]
                     heightList = self.dataOut.heightList[minIndex:maxIndex+1]
                     # determina indices
                     nheis = int(self.dataOut.radarControllerHeaderObj.txB/(self.dataOut.heightList[1]-self.dataOut.heightList[0]))
                     avg_dB = 10*numpy.log10(numpy.sum(data_spc[channelindex,:,:],axis=0))
                     beacon_dB = numpy.sort(avg_dB)[-nheis:]
                     beacon_heiIndexList = []
                     for val in avg_dB.tolist():
                         if val >= beacon_dB[0]:
                             beacon_heiIndexList.append(avg_dB.tolist().index(val))
                     #data_spc = data_spc[:,:,beacon_heiIndexList]
                     data_cspc = None
                     if self.dataOut.data_cspc is not None:
                         data_cspc = self.dataOut.data_cspc[:,:,minIndex:maxIndex+1]
                         #data_cspc = data_cspc[:,:,beacon_heiIndexList]
                     data_dc = None
                     if self.dataOut.data_dc is not None:
                         data_dc = self.dataOut.data_dc[:,minIndex:maxIndex+1]
                         #data_dc = data_dc[:,beacon_heiIndexList]
                     self.dataOut.data_spc = data_spc
                     self.dataOut.data_cspc = data_cspc
                     self.dataOut.data_dc = data_dc
                     self.dataOut.heightList = heightList
                     self.dataOut.beacon_heiIndexList = beacon_heiIndexList
                     return 1
                 def selectHeightsByIndex(self, minIndex, maxIndex):
                     """
                     Selecciona un bloque de datos en base a un grupo indices de alturas segun el rango
                     minIndex <= index <= maxIndex
                     Input:
                         minIndex    :    valor de indice minimo de altura a considerar
                         maxIndex    :    valor de indice maximo de altura a considerar
                     Affected:
                         self.dataOut.data_spc
                         self.dataOut.data_cspc
                         self.dataOut.data_dc
                         self.dataOut.heightList
                     Return:
 si el metodo se ejecuto con exito caso contrario devuelve 0
                     """
                     if (minIndex < 0) or (minIndex > maxIndex):
                         raise ValueError, "Error selecting heights: Index range (%d,%d) is not valid" % (minIndex, maxIndex)
                     if (maxIndex >= self.dataOut.nHeights):
                         maxIndex = self.dataOut.nHeights-1
                     #Spectra
                     data_spc = self.dataOut.data_spc[:,:,minIndex:maxIndex+1]
                     data_cspc = None
                     if self.dataOut.data_cspc is not None:
                         data_cspc = self.dataOut.data_cspc[:,:,minIndex:maxIndex+1]
                     data_dc = None
                     if self.dataOut.data_dc is not None:
                         data_dc = self.dataOut.data_dc[:,minIndex:maxIndex+1]
                     self.dataOut.data_spc = data_spc
                     self.dataOut.data_cspc = data_cspc
                     self.dataOut.data_dc = data_dc
                     self.dataOut.heightList = self.dataOut.heightList[minIndex:maxIndex+1]
                     return 1
                 def removeDC(self, mode = 2):
                     jspectra = self.dataOut.data_spc
                     jcspectra = self.dataOut.data_cspc
                     num_chan = jspectra.shape[0]
                     num_hei = jspectra.shape[2]
                     if jcspectra is not None:
                         jcspectraExist = True
                         num_pairs = jcspectra.shape[0]
                     else:   jcspectraExist = False
                     freq_dc = jspectra.shape[1]/2
                     ind_vel = numpy.array([-2,-1,1,2]) + freq_dc
                     if ind_vel[0]<0:
                         ind_vel[range(0,1)] = ind_vel[range(0,1)] + self.num_prof
                     if mode == 1:
                         jspectra[:,freq_dc,:] = (jspectra[:,ind_vel[1],:] + jspectra[:,ind_vel[2],:])/2 #CORRECCION
                         if jcspectraExist:
                             jcspectra[:,freq_dc,:] = (jcspectra[:,ind_vel[1],:] + jcspectra[:,ind_vel[2],:])/2
                     if mode == 2:
                         vel = numpy.array([-2,-1,1,2])
                         xx = numpy.zeros([4,4])
                         for fil in range(4):
                             xx[fil,:] = vel[fil]**numpy.asarray(range(4))
                         xx_inv = numpy.linalg.inv(xx)
                         xx_aux = xx_inv[0,:]
                         for ich in range(num_chan):
                             yy = jspectra[ich,ind_vel,:]
                             jspectra[ich,freq_dc,:] = numpy.dot(xx_aux,yy)
                             junkid = jspectra[ich,freq_dc,:]<=0
                             cjunkid = sum(junkid)
                             if cjunkid.any():
                                 jspectra[ich,freq_dc,junkid.nonzero()] = (jspectra[ich,ind_vel[1],junkid] + jspectra[ich,ind_vel[2],junkid])/2
                         if jcspectraExist:
                             for ip in range(num_pairs):
                                 yy = jcspectra[ip,ind_vel,:]
                                 jcspectra[ip,freq_dc,:] = numpy.dot(xx_aux,yy)
                     self.dataOut.data_spc = jspectra
                     self.dataOut.data_cspc = jcspectra
                     return 1
                 def removeInterference(self,  interf = 2,hei_interf = None, nhei_interf = None, offhei_interf = None):
                     jspectra = self.dataOut.data_spc
                     jcspectra = self.dataOut.data_cspc
                     jnoise = self.dataOut.getNoise()
                     num_incoh = self.dataOut.nIncohInt
                     num_channel  = jspectra.shape[0]
                     num_prof  = jspectra.shape[1]
                     num_hei   = jspectra.shape[2]
                     #hei_interf
                     if hei_interf is None:
                         count_hei = num_hei/2   #Como es entero no importa
                         hei_interf = numpy.asmatrix(range(count_hei)) + num_hei - count_hei
                         hei_interf = numpy.asarray(hei_interf)[0]
                     #nhei_interf
                     if (nhei_interf == None):
                         nhei_interf = 5
                     if (nhei_interf < 1):
                         nhei_interf = 1
                     if (nhei_interf > count_hei):
                         nhei_interf = count_hei
                     if (offhei_interf == None):
                         offhei_interf = 0
                     ind_hei = range(num_hei)
             #         mask_prof = numpy.asarray(range(num_prof - 2)) + 1
             #         mask_prof[range(num_prof/2 - 1,len(mask_prof))] += 1
                     mask_prof = numpy.asarray(range(num_prof))
                     num_mask_prof = mask_prof.size
                     comp_mask_prof = [0, num_prof/2]
                     #noise_exist:    Determina si la variable jnoise ha sido definida y contiene la informacion del ruido de cada canal
                     if (jnoise.size < num_channel or numpy.isnan(jnoise).any()):
                         jnoise = numpy.nan
                     noise_exist = jnoise[0] < numpy.Inf
                     #Subrutina de Remocion de la Interferencia
                     for ich in range(num_channel):
                         #Se ordena los espectros segun su potencia (menor a mayor)
                         power = jspectra[ich,mask_prof,:]
                         power = power[:,hei_interf]
                         power = power.sum(axis = 0)
                         psort = power.ravel().argsort()
                         #Se estima la interferencia promedio en los Espectros de Potencia empleando
                         junkspc_interf = jspectra[ich,:,hei_interf[psort[range(offhei_interf, nhei_interf + offhei_interf)]]]
                         if noise_exist:
                         #    tmp_noise = jnoise[ich] / num_prof
                             tmp_noise = jnoise[ich]
                         junkspc_interf = junkspc_interf - tmp_noise
                         #junkspc_interf[:,comp_mask_prof] = 0
                         jspc_interf = junkspc_interf.sum(axis = 0) / nhei_interf
                         jspc_interf = jspc_interf.transpose()
                         #Calculando el espectro de interferencia promedio
                         noiseid =  numpy.where(jspc_interf <= tmp_noise/ numpy.sqrt(num_incoh))
                         noiseid = noiseid[0]
                         cnoiseid = noiseid.size
                         interfid = numpy.where(jspc_interf > tmp_noise/ numpy.sqrt(num_incoh))
                         interfid = interfid[0]
                         cinterfid = interfid.size
                         if (cnoiseid > 0):   jspc_interf[noiseid] = 0
                         #Expandiendo los perfiles a limpiar
                         if (cinterfid > 0):
                             new_interfid = (numpy.r_[interfid - 1, interfid, interfid + 1] + num_prof)%num_prof
                             new_interfid = numpy.asarray(new_interfid)
                             new_interfid = {x for x in new_interfid}
                             new_interfid = numpy.array(list(new_interfid))
                             new_cinterfid = new_interfid.size
                         else: new_cinterfid = 0
                         for ip in range(new_cinterfid):
                             ind = junkspc_interf[:,new_interfid[ip]].ravel().argsort()
                             jspc_interf[new_interfid[ip]] = junkspc_interf[ind[nhei_interf/2],new_interfid[ip]]
                         jspectra[ich,:,ind_hei] = jspectra[ich,:,ind_hei] - jspc_interf #Corregir indices
                         #Removiendo la interferencia del punto de mayor interferencia
                         ListAux = jspc_interf[mask_prof].tolist()
                         maxid = ListAux.index(max(ListAux))
                         if cinterfid > 0:
                             for ip in range(cinterfid*(interf == 2) - 1):
                                 ind = (jspectra[ich,interfid[ip],:] < tmp_noise*(1 + 1/numpy.sqrt(num_incoh))).nonzero()
                                 cind = len(ind)
                                 if (cind > 0):
                                     jspectra[ich,interfid[ip],ind] = tmp_noise*(1 + (numpy.random.uniform(cind) - 0.5)/numpy.sqrt(num_incoh))
                             ind = numpy.array([-2,-1,1,2])
                             xx = numpy.zeros([4,4])
                             for id1 in range(4):
                                 xx[:,id1] = ind[id1]**numpy.asarray(range(4))
                             xx_inv = numpy.linalg.inv(xx)
                             xx = xx_inv[:,0]
                             ind = (ind + maxid + num_mask_prof)%num_mask_prof
                             yy = jspectra[ich,mask_prof[ind],:]
                             jspectra[ich,mask_prof[maxid],:] = numpy.dot(yy.transpose(),xx)
                         indAux = (jspectra[ich,:,:] < tmp_noise*(1-1/numpy.sqrt(num_incoh))).nonzero()
                         jspectra[ich,indAux[0],indAux[1]] = tmp_noise * (1 - 1/numpy.sqrt(num_incoh))
                     #Remocion de Interferencia en el Cross Spectra
                     if jcspectra is None: return jspectra, jcspectra
                     num_pairs = jcspectra.size/(num_prof*num_hei)
                     jcspectra = jcspectra.reshape(num_pairs, num_prof, num_hei)
                     for ip in range(num_pairs):
                         #-------------------------------------------
                         cspower = numpy.abs(jcspectra[ip,mask_prof,:])
                         cspower = cspower[:,hei_interf]
                         cspower = cspower.sum(axis = 0)
                         cspsort = cspower.ravel().argsort()
                         junkcspc_interf = jcspectra[ip,:,hei_interf[cspsort[range(offhei_interf, nhei_interf + offhei_interf)]]]
                         junkcspc_interf = junkcspc_interf.transpose()
                         jcspc_interf = junkcspc_interf.sum(axis = 1)/nhei_interf
                         ind = numpy.abs(jcspc_interf[mask_prof]).ravel().argsort()
                         median_real = numpy.median(numpy.real(junkcspc_interf[mask_prof[ind[range(3*num_prof/4)]],:]))
                         median_imag = numpy.median(numpy.imag(junkcspc_interf[mask_prof[ind[range(3*num_prof/4)]],:]))
                         junkcspc_interf[comp_mask_prof,:] = numpy.complex(median_real, median_imag)
                         for iprof in range(num_prof):
                             ind = numpy.abs(junkcspc_interf[iprof,:]).ravel().argsort()
                             jcspc_interf[iprof] = junkcspc_interf[iprof, ind[nhei_interf/2]]
                         #Removiendo la Interferencia
                         jcspectra[ip,:,ind_hei] = jcspectra[ip,:,ind_hei] - jcspc_interf
                         ListAux = numpy.abs(jcspc_interf[mask_prof]).tolist()
                         maxid = ListAux.index(max(ListAux))
                         ind = numpy.array([-2,-1,1,2])
                         xx = numpy.zeros([4,4])
                         for id1 in range(4):
                             xx[:,id1] = ind[id1]**numpy.asarray(range(4))
                         xx_inv = numpy.linalg.inv(xx)
                         xx = xx_inv[:,0]
                         ind = (ind + maxid + num_mask_prof)%num_mask_prof
                         yy = jcspectra[ip,mask_prof[ind],:]
                         jcspectra[ip,mask_prof[maxid],:] = numpy.dot(yy.transpose(),xx)
                     #Guardar Resultados
                     self.dataOut.data_spc = jspectra
                     self.dataOut.data_cspc = jcspectra
                     return 1
                 def setRadarFrequency(self, frequency=None):
                     if frequency != None:
                         self.dataOut.frequency = frequency
                     return 1
                 def getNoise(self, minHei=None, maxHei=None, minVel=None, maxVel=None):
                     #validacion de rango
                     if minHei == None:
                         minHei = self.dataOut.heightList[0]
                     if maxHei == None:
                         maxHei = self.dataOut.heightList[-1]
                     if (minHei < self.dataOut.heightList[0]) or (minHei > maxHei):
                         print 'minHei: %.2f is out of the heights range'%(minHei)
                         print 'minHei is setting to %.2f'%(self.dataOut.heightList[0])
                         minHei = self.dataOut.heightList[0]
                     if (maxHei > self.dataOut.heightList[-1]) or (maxHei < minHei):
                         print 'maxHei: %.2f is out of the heights range'%(maxHei)
                         print 'maxHei is setting to %.2f'%(self.dataOut.heightList[-1])
                         maxHei = self.dataOut.heightList[-1]
                     # validacion de velocidades
                     velrange = self.dataOut.getVelRange(1)
                     if minVel == None:
                         minVel = velrange[0]
                     if maxVel == None:
                         maxVel = velrange[-1]
                     if (minVel < velrange[0]) or (minVel > maxVel):
                         print 'minVel: %.2f is out of the velocity range'%(minVel)
                         print 'minVel is setting to %.2f'%(velrange[0])
                         minVel = velrange[0]
                     if (maxVel > velrange[-1]) or (maxVel < minVel):
                         print 'maxVel: %.2f is out of the velocity range'%(maxVel)
                         print 'maxVel is setting to %.2f'%(velrange[-1])
                         maxVel = velrange[-1]
                     # seleccion de indices para rango
                     minIndex = 0
                     maxIndex = 0
                     heights = self.dataOut.heightList
                     inda = numpy.where(heights >= minHei)
                     indb = numpy.where(heights <= maxHei)
                     try:
                         minIndex = inda[0][0]
                     except:
                         minIndex = 0
                     try:
                         maxIndex = indb[0][-1]
                     except:
                         maxIndex = len(heights)
                     if (minIndex < 0) or (minIndex > maxIndex):
                         raise ValueError, "some value in (%d,%d) is not valid" % (minIndex, maxIndex)
                     if (maxIndex >= self.dataOut.nHeights):
                         maxIndex = self.dataOut.nHeights-1
                     # seleccion de indices para velocidades
                     indminvel = numpy.where(velrange >= minVel)
                     indmaxvel = numpy.where(velrange <= maxVel)
                     try:
                         minIndexVel = indminvel[0][0]
                     except:
                         minIndexVel = 0
                     try:
                         maxIndexVel = indmaxvel[0][-1]
                     except:
                         maxIndexVel = len(velrange)
                     #seleccion del espectro
                     data_spc = self.dataOut.data_spc[:,minIndexVel:maxIndexVel+1,minIndex:maxIndex+1]
                     #estimacion de ruido
                     noise = numpy.zeros(self.dataOut.nChannels)
                     for channel in range(self.dataOut.nChannels):
                         daux = data_spc[channel,:,:]
                         noise[channel] = hildebrand_sekhon(daux, self.dataOut.nIncohInt)
                     self.dataOut.noise_estimation = noise.copy()
                     return 1
             class IncohInt(Operation):
                 __profIndex = 0
                 __withOverapping  = False
                 __byTime = False
                 __initime = None
                 __lastdatatime = None
                 __integrationtime = None
                 __buffer_spc = None
                 __buffer_cspc = None
                 __buffer_dc = None
                 __dataReady = False
                 __timeInterval = None
                 n = None
                 def __init__(self, **kwargs):
                     Operation.__init__(self, **kwargs)
             #         self.isConfig = False
                 def setup(self, n=None, timeInterval=None, overlapping=False):
                     """
                     Set the parameters of the integration class.
                     Inputs:
                         n        :    Number of coherent integrations
                         timeInterval   :    Time of integration. If the parameter "n" is selected this one does not work
                         overlapping    :
                     """
                     self.__initime = None
                     self.__lastdatatime = 0
                     self.__buffer_spc = 0
                     self.__buffer_cspc = 0
                     self.__buffer_dc = 0
                     self.__profIndex = 0
                     self.__dataReady = False
                     self.__byTime = False
                     if n is None and timeInterval is None:
                         raise ValueError, "n or timeInterval should be specified ..."
                     if n is not None:
                         self.n = int(n)
                     else:
                         self.__integrationtime = int(timeInterval) #if (type(timeInterval)!=integer) -> change this line
                         self.n = None
                         self.__byTime = True
                 def putData(self, data_spc, data_cspc, data_dc):
                     """
                     Add a profile to the __buffer_spc and increase in one the __profileIndex
                     """
                     self.__buffer_spc += data_spc
                     if data_cspc is None:
                         self.__buffer_cspc = None
                     else:
                         self.__buffer_cspc += data_cspc
                     if data_dc is None:
                         self.__buffer_dc = None
                     else:
                         self.__buffer_dc += data_dc
                     self.__profIndex += 1
                     return
                 def pushData(self):
                     """
                     Return the sum of the last profiles and the profiles used in the sum.
                     Affected:
                     self.__profileIndex
                     """
                     data_spc = self.__buffer_spc
                     data_cspc = self.__buffer_cspc
                     data_dc = self.__buffer_dc
                     n = self.__profIndex
                     self.__buffer_spc = 0
                     self.__buffer_cspc = 0
                     self.__buffer_dc = 0
                     self.__profIndex = 0
                     return data_spc, data_cspc, data_dc, n
                 def byProfiles(self, *args):
                     self.__dataReady = False
                     avgdata_spc = None
                     avgdata_cspc = None
                     avgdata_dc = None
                     self.putData(*args)
                     if self.__profIndex == self.n:
                         avgdata_spc, avgdata_cspc, avgdata_dc, n = self.pushData()
                         self.n = n
                         self.__dataReady = True
                     return avgdata_spc, avgdata_cspc, avgdata_dc
                 def byTime(self, datatime, *args):
                     self.__dataReady = False
                     avgdata_spc = None
                     avgdata_cspc = None
                     avgdata_dc = None
                     self.putData(*args)
                     if (datatime - self.__initime) >= self.__integrationtime:
                         avgdata_spc, avgdata_cspc, avgdata_dc, n = self.pushData()
                         self.n = n
                         self.__dataReady = True
                     return avgdata_spc, avgdata_cspc, avgdata_dc
                 def integrate(self, datatime, *args):
                     if self.__profIndex == 0:
                         self.__initime = datatime
                     if self.__byTime:
                         avgdata_spc, avgdata_cspc, avgdata_dc = self.byTime(datatime, *args)
                     else:
                         avgdata_spc, avgdata_cspc, avgdata_dc = self.byProfiles(*args)
                     if not self.__dataReady:
                         return None, None, None, None
                     return self.__initime, avgdata_spc, avgdata_cspc, avgdata_dc
                 def run(self, dataOut, n=None, timeInterval=None, overlapping=False):
                     if n==1:
                         return
                     dataOut.flagNoData = True
                     if not self.isConfig:
                         self.setup(n, timeInterval, overlapping)
                         self.isConfig = True
                     avgdatatime, avgdata_spc, avgdata_cspc, avgdata_dc = self.integrate(dataOut.utctime,
                                                                                         dataOut.data_spc,
                                                                                         dataOut.data_cspc,
                                                                                         dataOut.data_dc)
                     if self.__dataReady:
                         dataOut.data_spc = avgdata_spc
                         dataOut.data_cspc = avgdata_cspc
                         dataOut.data_dc = avgdata_dc
                         dataOut.nIncohInt *= self.n
                         dataOut.utctime = avgdatatime
                         dataOut.flagNoData = False

schainpy/model/utils/jroutils_ftp.py +62 -62

             '''
             @author: Daniel Suarez
             '''
             import os
             import glob
             import ftplib
             try:
                 import paramiko
                 import scp
             except:
                 print "You should install paramiko and scp libraries \nif you want to use SSH protocol to upload files to the server"
             import time
             import threading
             Thread = threading.Thread
             # try:
             #     from gevent import sleep
             # except:
             from time import sleep
             from schainpy.model.proc.jroproc_base import ProcessingUnit, Operation
             class Remote(Thread):
                 """
                 Remote is a parent class used to define the behaviour of FTP and SSH class. These clases are
                 used to upload or download files remotely.
                 Non-standard Python modules used:
                     None
                 Written by:
+                    "Miguel Urco":mailto:miguel.urco@jro.igp.gob.pe Jun. 03, 2015
-                    "Miguel Urco":mailto:miguel.urco@jro.igp.gob.pe  Jun. 03, 2015
                 """
                 server = None
                 username = None
                 password = None
                 remotefolder = None
                 period = 60
                 fileList = []
                 bussy = False
                 def __init__(self, server, username, password, remotefolder, period=60):
                     Thread.__init__(self)
                     self.setDaemon(True)
                     self.status = 0
                     self.__server = server
                     self.__username = username
                     self.__password = password
                     self.__remotefolder = remotefolder
                     self.period = period
                     self.fileList = []
                     self.bussy = False
                     self.stopFlag = False
                     print "[Remote Server] Opening server: %s" %self.__server
                     if self.open(self.__server, self.__username, self.__password, self.__remotefolder):
                         print "[Remote Server] %s server was opened successfully" %self.__server
                     self.close()
                     self.mutex = threading.Lock()
                 def stop(self):
                     self.stopFlag = True
                     self.join(10)
                 def open(self):
                     """
                     Connect to server and create a connection class (FTP or SSH) to remote server.
                     """
                     raise NotImplementedError, "Implement this method in child class"
                 def close(self):
                     """
                     Close connection to server
                     """
                     raise NotImplementedError, "Implement this method in child class"
                 def mkdir(self, remotefolder):
                     """
                     Create a folder remotely
                     """
                     raise NotImplementedError, "Implement this method in child class"
                 def cd(self, remotefolder):
                     """
                     Change working directory in remote server
                     """
                     raise NotImplementedError, "Implement this method in child class"
                 def download(self, filename, localfolder=None):
                     """
                     Download a file from server to local host
                     """
                     raise NotImplementedError, "Implement this method in child class"
                 def sendFile(self, fullfilename):
                     """
                     sendFile method is used to upload a local file to the current directory in remote server
                     Inputs:
                         fullfilename        - full path name of local file to store in remote directory
                     Returns:
 in error case else 1
                     """
                     raise NotImplementedError, "Implement this method in child class"
                 def upload(self, fullfilename, remotefolder=None):
                     """
                     upload method is used to upload a local file to remote directory. This method changes
                     working directory before sending a file.
                     Inputs:
                         fullfilename        - full path name of local file to store in remote directory
                         remotefolder    - remote directory
                     Returns:
 in error case else 1
                     """
                     print "[Remote Server] Uploading %s to %s:%s" %(fullfilename, self.server, self.remotefolder)
                     if not self.status:
                         return 0
                     if remotefolder == None:
                         remotefolder = self.remotefolder
                     if not self.cd(remotefolder):
                         return 0
                     if not self.sendFile(fullfilename):
                         print "[Remote Server] Error uploading file %s" %fullfilename
                         return 0
                     print "[Remote Server] upload finished successfully"
                     return 1
                 def delete(self, filename):
                     """
                     Remove a file from remote server
                     """
                     pass
                 def updateFileList(self, fileList):
                     """
                     Remove a file from remote server
                     """
                     if fileList == self.fileList:
                         return 0
                     self.mutex.acquire()
-            #         init = time.time()
+                    #         init = time.time()
+                    #
-            #         while(self.bussy):
+                    #         while(self.bussy):
-            #             sleep(0.1)
+                    #             sleep(0.1)
-            #             if time.time() - init > 2*self.period:
+                    #             if time.time() - init > 2*self.period:
-            #                 return 0
+                    #                 return 0
                     self.fileList = fileList
                     self.mutex.release()
                     return 1
                 def run(self):
                     if not self.status:
                         print "Finishing FTP service"
                         return
                     if not self.cd(self.remotefolder):
                         raise ValueError, "Could not access to the new remote directory: %s" %self.remotefolder
                     while True:
                         for i in range(self.period):
                             if self.stopFlag:
                                 break
                             sleep(1)
                         if self.stopFlag:
                             break
-            #             self.bussy = True
+                #             self.bussy = True
                         self.mutex.acquire()
                         print "[Remote Server] Opening %s" %self.__server
                         if not self.open(self.__server, self.__username, self.__password, self.__remotefolder):
                             self.mutex.release()
                             continue
                         for thisFile in self.fileList:
                             self.upload(thisFile, self.remotefolder)
                         print "[Remote Server] Closing %s" %self.__server
                         self.close()
                         self.mutex.release()
             #             self.bussy = False
                     print "[Remote Server] Thread stopped successfully"
             class FTPClient(Remote):
                 __ftpClientObj = None
                 def __init__(self, server, username, password, remotefolder, period=60):
                     """
                     """
                     Remote.__init__(self, server, username, password, remotefolder, period)
                 def open(self, server, username, password, remotefolder):
                     """
                     This method is used to set FTP parameters and establish a connection to remote server
                     Inputs:
                         server    - remote server IP Address
                         username    - remote server Username
                         password    - remote server password
                         remotefolder    - remote server current working directory
                     Return:
                         Boolean     -     Returns 1 if a connection has been established, 0 otherwise
                     Affects:
                         self.status        - in case of error or fail connection this parameter is set to 0 else 1
                     """
                     if server == None:
                         raise ValueError, "FTP server should be defined"
                     if username == None:
                         raise ValueError, "FTP username should be defined"
                     if password == None:
                         raise ValueError, "FTP password should be defined"
                     if remotefolder == None:
                         raise ValueError, "FTP remote folder should be defined"
                     try:
                         ftpClientObj = ftplib.FTP(server)
                     except ftplib.all_errors, e:
                         print "[FTP Server]: FTP server connection fail: %s" %server
                         print "[FTP Server]:", e
                         self.status = 0
                         return 0
                     try:
                         ftpClientObj.login(username, password)
                     except ftplib.all_errors:
                         print "[FTP Server]: FTP username or password are incorrect"
                         self.status = 0
                         return 0
                     if remotefolder == None:
                         remotefolder = ftpClientObj.pwd()
                     else:
                         try:
                             ftpClientObj.cwd(remotefolder)
                         except ftplib.all_errors:
                             print "[FTP Server]: FTP remote folder is invalid: %s" %remotefolder
                             remotefolder = ftpClientObj.pwd()
                     self.server = server
                     self.username = username
                     self.password = password
                     self.remotefolder = remotefolder
                     self.__ftpClientObj = ftpClientObj
                     self.status = 1
                     return 1
                 def close(self):
                     """
                     Close connection to remote server
                     """
                     if not self.status:
                         return 0
                     self.__ftpClientObj.close()
                 def mkdir(self, remotefolder):
                     """
                     mkdir is used to make a new directory in remote server
                     Input:
                         remotefolder    - directory name
                     Return:
 in error case else 1
                     """
                     if not self.status:
                         return 0
                     try:
                         self.__ftpClientObj.mkd(dirname)
                     except ftplib.all_errors:
                         print "[FTP Server]: Error creating remote folder: %s" %remotefolder
                         return 0
                     return 1
                 def cd(self, remotefolder):
                     """
                     cd is used to change remote working directory on server
                     Input:
                         remotefolder    - current working directory
                     Affects:
                         self.remotefolder
                     Return:
 in case of error else 1
                     """
                     if not self.status:
                         return 0
                     if remotefolder == self.remotefolder:
                         return 1
                     try:
                         self.__ftpClientObj.cwd(remotefolder)
                     except ftplib.all_errors:
                         print '[FTP Server]: Error changing to %s' %remotefolder
                         print '[FTP Server]: Trying to create remote folder'
                         if not self.mkdir(remotefolder):
                             print '[FTP Server]: Remote folder could not be created'
                             return 0
                         try:
                             self.__ftpClientObj.cwd(remotefolder)
                         except ftplib.all_errors:
                             return 0
                     self.remotefolder = remotefolder
                     return 1
                 def sendFile(self, fullfilename):
                     if not self.status:
                         return 0
                     fp = open(fullfilename, 'rb')
                     filename = os.path.basename(fullfilename)
                     command = "STOR %s" %filename
                     try:
                         self.__ftpClientObj.storbinary(command, fp)
                     except ftplib.all_errors, e:
                         print "[FTP Server]:", e
                         return 0
                     try:
                         self.__ftpClientObj.sendcmd('SITE CHMOD 755 ' + filename)
                     except ftplib.all_errors, e:
                         print "[FTP Server]:", e
                     fp.close()
                     return 1
             class SSHClient(Remote):
                 __sshClientObj = None
                 __scpClientObj = None
                 def __init__(self, server, username, password, remotefolder, period=60):
                     """
                     """
                     Remote.__init__(self, server, username, password, remotefolder, period)
                 def open(self, server, username, password, remotefolder, port=22):
                     """
-                    This method is used to set SSH parameters and establish a connection to a remote server
+                        This method is used to set SSH parameters and establish a connection to a remote server
-                    Inputs:
+                        Inputs:
-                        server    - remote server IP Address
+                            server    - remote server IP Address
-                        username    - remote server Username
+                            username    - remote server Username
-                        password    - remote server password
+                            password    - remote server password
-                        remotefolder    - remote server current working directory
+                            remotefolder    - remote server current working directory
-                    Return: void
+                        Return: void
-                    Affects:
+                        Affects:
-                        self.status        - in case of error or fail connection this parameter is set to 0 else 1
+                            self.status        - in case of error or fail connection this parameter is set to 0 else 1
                     """
                     import socket
                     if server == None:
                         raise ValueError, "SSH server should be defined"
                     if username == None:
                         raise ValueError, "SSH username should be defined"
                     if password == None:
                         raise ValueError, "SSH password should be defined"
                     if remotefolder == None:
                         raise ValueError, "SSH remote folder should be defined"
                     sshClientObj = paramiko.SSHClient()
                     sshClientObj.load_system_host_keys()
                     sshClientObj.set_missing_host_key_policy(paramiko.WarningPolicy())
                     self.status = 0
                     try:
                         sshClientObj.connect(server, username=username, password=password, port=port)
                     except paramiko.AuthenticationException, e:
-            #             print "SSH username or password are incorrect: %s"
+                #             print "SSH username or password are incorrect: %s"
                         print "[SSH Server]:", e
                         return 0
                     except SSHException, e:
                         print "[SSH Server]:", e
                         return 0
                     except socket.error:
                         self.status = 0
                         print "[SSH Server]:", e
                         return 0
                     self.status = 1
                     scpClientObj = scp.SCPClient(sshClientObj.get_transport(), socket_timeout=30)
                     if remotefolder == None:
                         remotefolder = self.pwd()
                     self.server = server
                     self.username = username
                     self.password = password
                     self.__sshClientObj = sshClientObj
                     self.__scpClientObj = scpClientObj
                     self.status = 1
                     if not self.cd(remotefolder):
                         raise ValueError, "[SSH Server]: Could not access to remote folder: %s" %remotefolder
                         return 0
                     self.remotefolder = remotefolder
                     return 1
                 def close(self):
                     """
-                    Close connection to remote server
+                        Close connection to remote server
                     """
                     if not self.status:
                         return 0
                     self.__scpClientObj.close()
                     self.__sshClientObj.close()
                 def __execute(self, command):
                     """
-                    __execute a command on remote server
+                        __execute a command on remote server
-                    Input:
+                        Input:
-                        command    - Exmaple 'ls -l'
+                            command    - Exmaple 'ls -l'
-                    Return:
+                        Return:
 in error case else 1
                     """
                     if not self.status:
                         return 0
                     stdin, stdout, stderr = self.__sshClientObj.exec_command(command)
                     result = stderr.readlines()
                     if len(result) > 1:
                         return 0
                     result = stdout.readlines()
                     if len(result) > 1:
                         return result[0][:-1]
                     return 1
                 def mkdir(self, remotefolder):
                     """
-                    mkdir is used to make a new directory in remote server
+                        mkdir is used to make a new directory in remote server
-                    Input:
+                        Input:
-                        remotefolder    - directory name
+                            remotefolder    - directory name
-                    Return:
+                        Return:
 in error case else 1
                     """
                     command = 'mkdir %s' %remotefolder
                     return self.__execute(command)
                 def pwd(self):
                     command = 'pwd'
                     return self.__execute(command)
                 def cd(self, remotefolder):
                     """
-                    cd is used to change remote working directory on server
+                        cd is used to change remote working directory on server
-                    Input:
+                        Input:
-                        remotefolder    - current working directory
+                            remotefolder    - current working directory
-                    Affects:
+                        Affects:
-                        self.remotefolder
+                            self.remotefolder
-                    Return:
+                        Return:
 in case of error else 1
                     """
                     if not self.status:
                         return 0
                     if remotefolder == self.remotefolder:
                         return 1
                     chk_command = "cd %s; pwd" %remotefolder
                     mkdir_command = "mkdir %s" %remotefolder
                     if not self.__execute(chk_command):
                         if not self.__execute(mkdir_command):
                             self.remotefolder = None
                             return 0
                     self.remotefolder = remotefolder
                     return 1
                 def sendFile(self, fullfilename):
                     if not self.status:
                         return 0
                     try:
                         self.__scpClientObj.put(fullfilename, remote_path=self.remotefolder)
                     except scp.ScpError, e:
                         print "[SSH Server]", str(e)
                         return 0
                     remotefile = os.path.join(self.remotefolder, os.path.split(fullfilename)[-1])
                     command = 'chmod 775 %s' %remotefile
                     return self.__execute(command)
             class SendToServer(ProcessingUnit):
                 def __init__(self, **kwargs):
                     ProcessingUnit.__init__(self, **kwargs)
                     self.isConfig = False
                     self.clientObj = None
                 def setup(self, server, username, password, remotefolder, localfolder, ext='.png', period=60, protocol='ftp', **kwargs):
                     self.clientObj = None
                     self.localfolder = localfolder
                     self.ext = ext
                     self.period = period
                     if str.lower(protocol) == 'ftp':
                         self.clientObj = FTPClient(server, username, password, remotefolder, period)
                     if str.lower(protocol) == 'ssh':
                         self.clientObj = SSHClient(server, username, password, remotefolder, period)
                     if not self.clientObj:
                         raise ValueError, "%s has been chosen as remote access protocol but it is not valid" %protocol
                     self.clientObj.start()
                 def findFiles(self):
                     if not type(self.localfolder) == list:
                         folderList = [self.localfolder]
                     else:
                         folderList = self.localfolder
                     #Remove duplicate items
                     folderList = list(set(folderList))
                     fullfilenameList = []
                     for thisFolder in folderList:
                         print "[Remote Server]: Searching files on %s" %thisFolder
                         filenameList = glob.glob1(thisFolder, '*%s' %self.ext)
                         if len(filenameList) < 1:
                             continue
                         for thisFile in filenameList:
                             fullfilename = os.path.join(thisFolder, thisFile)
                             if fullfilename in fullfilenameList:
                                 continue
                             #Only files modified in the last 30 minutes are considered
                             if os.path.getmtime(fullfilename) < time.time() - 30*60:
                                 continue
                             fullfilenameList.append(fullfilename)
                     return fullfilenameList
                 def run(self, **kwargs):
                     if not self.isConfig:
                         self.init = time.time()
                         self.setup(**kwargs)
                         self.isConfig = True
+                    if not self.clientObj.is_alive():
+                        print "[Remote Server]: Restarting connection "
+                        self.setup(**kwargs)
                     if time.time() - self.init >= self.period:
                         fullfilenameList = self.findFiles()
                         if self.clientObj.updateFileList(fullfilenameList):
                             print "[Remote Server]: Sending the next files ", str(fullfilenameList)
                         self.init = time.time()
                 def close(self):
                     print "[Remote Server] Stopping thread"
                     self.clientObj.stop()
             class FTP(object):
                 """
                 Ftp is a public class used to define custom File Transfer Protocol from "ftplib" python module
                 Non-standard Python modules used: None
                 Written by "Daniel Suarez":mailto:daniel.suarez@jro.igp.gob.pe  Oct. 26, 2010
                 """
                 def __init__(self,server = None, username=None, password=None, remotefolder=None):
                     """
                     This method is used to setting parameters for FTP and establishing connection to remote server
                     Inputs:
                         server    - remote server IP Address
                         username    - remote server Username
                         password    - remote server password
                         remotefolder    - remote server current working directory
                     Return: void
                     Affects:
                         self.status    - in Error Case or Connection Failed this parameter is set to 1 else 0
                         self.folderList    - sub-folder list of remote folder
                         self.fileList    - file list of remote folder
                     """
                     if ((server == None) and (username==None) and (password==None) and (remotefolder==None)):
                         server, username, password, remotefolder = self.parmsByDefault()
                     self.server = server
                     self.username = username
                     self.password = password
                     self.remotefolder = remotefolder
                     self.file = None
                     self.ftp = None
                     self.status = 0
                     try:
                         self.ftp = ftplib.FTP(self.server)
                         self.ftp.login(self.username,self.password)
                         self.ftp.cwd(self.remotefolder)
-            #            print 'Connect to FTP Server: Successfully'
+                #            print 'Connect to FTP Server: Successfully'
                     except ftplib.all_errors:
                         print 'Error FTP Service'
                         self.status = 1
                         return
                     self.dirList = []
                     try:
                         self.dirList = self.ftp.nlst()
                     except ftplib.error_perm, resp:
                         if str(resp) == "550 No files found":
                                 print "no files in this directory"
                                 self.status = 1
                                 return
                     except ftplib.all_errors:
                         print 'Error Displaying Dir-Files'
                         self.status = 1
                         return
                     self.fileList = []
                     self.folderList = []
                     #only for test
                     for f in self.dirList:
                         name, ext = os.path.splitext(f)
                         if ext != '':
                             self.fileList.append(f)
-            #                print 'filename: %s - size: %d'%(f,self.ftp.size(f))
+                #                print 'filename: %s - size: %d'%(f,self.ftp.size(f))
                 def parmsByDefault(self):
                     server = 'jro-app.igp.gob.pe'
                     username = 'wmaster'
                     password = 'mst2010vhf'
                     remotefolder = '/home/wmaster/graficos'
                     return server, username, password, remotefolder
                 def mkd(self,dirname):
                     """
                     mkd is used to make directory in remote server
                     Input:
                         dirname    - directory name
                     Return:
 in error case else 0
                     """
                     try:
                         self.ftp.mkd(dirname)
                     except:
                         print 'Error creating remote folder:%s'%dirname
                         return 1
                     return 0
                 def delete(self,filename):
                     """
                     delete is used to delete file in current working directory of remote server
                     Input:
                         filename    - filename to delete in remote folder
                     Return:
 in error case else 0
                     """
                     try:
                         self.ftp.delete(filename)
                     except:
                         print 'Error deleting remote file:%s'%filename
                         return 1
                     return 0
                 def download(self,filename,localfolder):
                     """
                     download is used to downloading file from remote folder into local folder
                     Inputs:
                         filename    - filename to donwload
                         localfolder    - directory local to store filename
                     Returns:
                         self.status    - 1 in error case else 0
                     """
                     self.status = 0
                     if not(filename in self.fileList):
                         print 'filename:%s not exists'%filename
                         self.status = 1
                         return self.status
                     newfilename = os.path.join(localfolder,filename)
                     self.file = open(newfilename, 'wb')
                     try:
                         print 'Download: ' + filename
                         self.ftp.retrbinary('RETR ' + filename, self.__handleDownload)
                         print 'Download Complete'
                     except ftplib.all_errors:
                         print 'Error Downloading ' + filename
                         self.status = 1
                         return self.status
                     self.file.close()
                     return self.status
                 def __handleDownload(self,block):
                     """
                     __handleDownload is used to handle writing file
                     """
                     self.file.write(block)
                 def upload(self,filename,remotefolder=None):
                     """
                     upload is used to uploading local file to remote directory
                     Inputs:
                         filename    - full path name of local file to store in remote directory
                         remotefolder    - remote directory
                     Returns:
                         self.status    - 1 in error case else 0
                     """
                     if remotefolder == None:
                         remotefolder = self.remotefolder
                     self.status = 0
                     try:
                         self.ftp.cwd(remotefolder)
                         self.file = open(filename, 'rb')
                         (head, tail) = os.path.split(filename)
                         command = "STOR " + tail
                         print 'Uploading: ' + tail
                         self.ftp.storbinary(command, self.file)
                         print 'Upload Completed'
                     except ftplib.all_errors:
                         print 'Error Uploading ' + tail
                         self.status = 1
                         return self.status
                     self.file.close()
                     #back to initial directory in __init__()
                     self.ftp.cwd(self.remotefolder)
                     return self.status
                 def dir(self,remotefolder):
                     """
                     dir is used to change working directory of remote server and get folder and file list
                     Input:
                         remotefolder    - current working directory
                     Affects:
                         self.fileList    - file list of working directory
                     Return:
                         infoList    - list with filenames and size of file in bytes
                         self.folderList    -    folder list
                     """
                     self.remotefolder = remotefolder
                     print 'Change to ' + self.remotefolder
                     try:
                         self.ftp.cwd(remotefolder)
                     except ftplib.all_errors:
                         print 'Error Change to ' + self.remotefolder
                         infoList = None
                         self.folderList = None
                         return infoList,self.folderList
                     self.dirList = []
                     try:
                         self.dirList = self.ftp.nlst()
                     except ftplib.error_perm, resp:
                         if str(resp) == "550 No files found":
                                 print "no files in this directory"
                                 infoList = None
                                 self.folderList = None
                                 return infoList,self.folderList
                     except ftplib.all_errors:
                         print 'Error Displaying Dir-Files'
                         infoList = None
                         self.folderList = None
                         return infoList,self.folderList
                     infoList = []
                     self.fileList = []
                     self.folderList = []
                     for f in self.dirList:
                         name,ext = os.path.splitext(f)
                         if ext != '':
                             self.fileList.append(f)
                             value = (f,self.ftp.size(f))
                             infoList.append(value)
                         if ext == '':
                             self.folderList.append(f)
                     return infoList,self.folderList
                 def close(self):
                     """
                     close is used to close and end FTP connection
                     Inputs: None
                     Return: void
                     """
                     self.ftp.close()
             class SendByFTP(Operation):
                 def __init__(self, **kwargs):
                     Operation.__init__(self, **kwargs)
                     self.status = 1
                     self.counter = 0
                 def error_print(self, ValueError):
                     print ValueError, 'Error FTP'
                     print "don't worry the program is running..."
                 def worker_ftp(self, server, username, password, remotefolder, filenameList):
                     self.ftpClientObj = FTP(server, username, password, remotefolder)
                     for filename in filenameList:
                         self.ftpClientObj.upload(filename)
                     self.ftpClientObj.close()
                 def ftp_thread(self, server, username, password, remotefolder):
                     if not(self.status):
                         return
                     import multiprocessing
                     p = multiprocessing.Process(target=self.worker_ftp, args=(server, username, password, remotefolder, self.filenameList,))
                     p.start()
                     p.join(3)
                     if p.is_alive():
                         p.terminate()
                         p.join()
                         print 'killing ftp process...'
                         self.status = 0
                         return
                     self.status = 1
                     return
                 def filterByExt(self, ext, localfolder):
                     fnameList = glob.glob1(localfolder,ext)
                     self.filenameList = [os.path.join(localfolder,x) for x in fnameList]
                     if len(self.filenameList) == 0:
                         self.status = 0
                 def run(self, dataOut, ext, localfolder, remotefolder, server, username, password, period=1):
                     self.counter += 1
                     if self.counter >= period:
                         self.filterByExt(ext, localfolder)
                         self.ftp_thread(server, username, password, remotefolder)
                         self.counter = 0
                     self.status = 1

schainpy/scripts/schain.xml 0 0

	1	NO CONTENT: modified file			NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff

setup.py 0 0

	1	NO CONTENT: modified file			NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff

schainpy/scripts/150km_January_longPulse.py removed 0 -185

NO CONTENT: file was removed

schainpy/scripts/150km_January_shortPulse.py removed 0 -152

NO CONTENT: file was removed

schainpy/scripts/150km_january_1.py removed 0 -119

NO CONTENT: file was removed

schainpy/scripts/150km_january_3.py removed 0 -128

NO CONTENT: file was removed

schainpy/scripts/150km_jicamarca.xml removed 0 -61

NO CONTENT: file was removed

schainpy/scripts/EWDrifts_estimation01.py removed 0 -140

NO CONTENT: file was removed

schainpy/scripts/EWDrifts_estimation02.py removed 0 -65

NO CONTENT: file was removed

schainpy/scripts/JASMET30_Online_spc.py removed 0 -118

NO CONTENT: file was removed

schainpy/scripts/JASMET30_PhaseCal.py removed 0 -77

NO CONTENT: file was removed

schainpy/scripts/JASMET30_PreProc.py removed 0 -53

NO CONTENT: file was removed

schainpy/scripts/JASMET30_Winds.py removed 0 -91

NO CONTENT: file was removed

schainpy/scripts/NSM1.py removed 0 -138

NO CONTENT: file was removed

schainpy/scripts/NSM11.py removed 0 -90

NO CONTENT: file was removed

schainpy/scripts/NSM2.py removed 0 -180

NO CONTENT: file was removed

schainpy/scripts/NSM21.py removed 0 -89

NO CONTENT: file was removed

schainpy/scripts/NSM_HDF5.py removed 0 -77

NO CONTENT: file was removed

schainpy/scripts/NSM_PreProc1.py removed 0 -76

NO CONTENT: file was removed

schainpy/scripts/NSM_Preproc2.py removed 0 -79

NO CONTENT: file was removed

schainpy/scripts/WindProfiler_DBS01.py removed 0 -168

NO CONTENT: file was removed

schainpy/scripts/WindProfiler_SA01.py removed 0 -129

NO CONTENT: file was removed

schainpy/scripts/__init__.py removed 0 -5

NO CONTENT: file was removed

schainpy/scripts/amisr_EEJ.py removed 0 -123

NO CONTENT: file was removed

schainpy/scripts/amisr_eej_proc_offline.py removed 0 -194

NO CONTENT: file was removed

schainpy/scripts/amisr_eej_proc_online.py removed 0 -205

NO CONTENT: file was removed

schainpy/scripts/amisr_esf_proc_offline.py removed 0 -197

NO CONTENT: file was removed

schainpy/scripts/amisr_esf_proc_online.py removed 0 -220

NO CONTENT: file was removed

schainpy/scripts/amisr_proc.py removed 0 -92

NO CONTENT: file was removed

schainpy/scripts/amisr_reader.py removed 0 -39

NO CONTENT: file was removed

schainpy/scripts/amisr_reader_exp.py removed 0 -47

NO CONTENT: file was removed

schainpy/scripts/amisr_reader_exp_ex2.py removed 0 -404

NO CONTENT: file was removed

schainpy/scripts/amisr_spreadF.py removed 0 -119

NO CONTENT: file was removed

schainpy/scripts/amisr_windEstimation.py removed 0 -147

NO CONTENT: file was removed

schainpy/scripts/amisr_winds_proc_offline.py removed 0 -222

NO CONTENT: file was removed

schainpy/scripts/amisr_winds_proc_online.py removed 0 -176

NO CONTENT: file was removed

schainpy/scripts/beacon_phase.py removed 0 -56

NO CONTENT: file was removed

schainpy/scripts/hf_plot_pdata.py removed 0 0

	1	NO CONTENT: file was removed			NO CONTENT: file was removed
The requested commit or file is too big and content was truncated. Show full diff

schainpy/scripts/hf_test.py removed 0 0

	1	NO CONTENT: file was removed			NO CONTENT: file was removed
The requested commit or file is too big and content was truncated. Show full diff

schainpy/scripts/hf_write_pdata.py removed 0 0

	1	NO CONTENT: file was removed			NO CONTENT: file was removed
The requested commit or file is too big and content was truncated. Show full diff

schainpy/scripts/jacktotalrunner.py removed 0 0

	1	NO CONTENT: file was removed			NO CONTENT: file was removed
The requested commit or file is too big and content was truncated. Show full diff

schainpy/scripts/jasmet50_16april.py removed 0 0

	1	NO CONTENT: file was removed			NO CONTENT: file was removed
The requested commit or file is too big and content was truncated. Show full diff

schainpy/scripts/mst_isr_eej_blocks.py removed 0 0

	1	NO CONTENT: file was removed			NO CONTENT: file was removed
The requested commit or file is too big and content was truncated. Show full diff

schainpy/scripts/mst_isr_eej_processing.py removed 0 0

	1	NO CONTENT: file was removed			NO CONTENT: file was removed
The requested commit or file is too big and content was truncated. Show full diff

schainpy/scripts/plot_eej.py removed 0 0

	1	NO CONTENT: file was removed			NO CONTENT: file was removed
The requested commit or file is too big and content was truncated. Show full diff

schainpy/scripts/plot_isr.py removed 0 0

	1	NO CONTENT: file was removed			NO CONTENT: file was removed
The requested commit or file is too big and content was truncated. Show full diff

schainpy/scripts/plot_mst.py removed 0 0

	1	NO CONTENT: file was removed			NO CONTENT: file was removed
The requested commit or file is too big and content was truncated. Show full diff

schainpy/scripts/schain removed 0 0

	1	NO CONTENT: file was removed			NO CONTENT: file was removed
The requested commit or file is too big and content was truncated. Show full diff

schainpy/scripts/schain_paralel.py removed 0 0

	1	NO CONTENT: file was removed			NO CONTENT: file was removed
The requested commit or file is too big and content was truncated. Show full diff

schainpy/scripts/sousy.py removed 0 0

	1	NO CONTENT: file was removed			NO CONTENT: file was removed
The requested commit or file is too big and content was truncated. Show full diff

schainpy/scripts/testPlotter.py removed 0 0

	1	NO CONTENT: file was removed			NO CONTENT: file was removed
The requested commit or file is too big and content was truncated. Show full diff

schainpy/scripts/testProcData.py removed 0 0

	1	NO CONTENT: file was removed			NO CONTENT: file was removed
The requested commit or file is too big and content was truncated. Show full diff

schainpy/scripts/testRawData.py removed 0 0

	1	NO CONTENT: file was removed			NO CONTENT: file was removed
The requested commit or file is too big and content was truncated. Show full diff

schainpy/scripts/testUSRPData.py removed 0 0

	1	NO CONTENT: file was removed			NO CONTENT: file was removed
The requested commit or file is too big and content was truncated. Show full diff

schainpy/scripts/test_eej_blocks.py removed 0 0

	1	NO CONTENT: file was removed			NO CONTENT: file was removed
The requested commit or file is too big and content was truncated. Show full diff

schainpy/scripts/test_isr_blocks.py removed 0 0

	1	NO CONTENT: file was removed			NO CONTENT: file was removed
The requested commit or file is too big and content was truncated. Show full diff

schainpy/scripts/test_mst_blocks.py removed 0 0

	1	NO CONTENT: file was removed			NO CONTENT: file was removed
The requested commit or file is too big and content was truncated. Show full diff

schainpy/scripts/zerorpc_client.py removed 0 0

	1	NO CONTENT: file was removed			NO CONTENT: file was removed
The requested commit or file is too big and content was truncated. Show full diff

schainpy/scripts/zerorpc_server.py removed 0 0

	1	NO CONTENT: file was removed			NO CONTENT: file was removed
The requested commit or file is too big and content was truncated. Show full diff

General Comments 0

Write
Preview

You need to be logged in to leave comments. Login now

No TODOs yet

	Site-wide shortcuts
/	Use quick search box
g h	Goto home page
g g	Goto my private gists page
g G	Goto my public gists page
g 0-9	Goto bookmarked items from 0-9
n r	New repository page
n g	New gist page

	Repositories
g s	Goto summary page
g c	Goto changelog page
g f	Goto files page
g F	Goto files page with file search activated
g p	Goto pull requests page
g o	Goto repository settings
g O	Goto repository access permissions settings
t s	Toggle sidebar on some pages