diff --git a/.gitignore b/.gitignore index c78f88d..e0bc00e 100644 --- a/.gitignore +++ b/.gitignore @@ -112,5 +112,3 @@ schainpy/scripts/ .vscode trash *.log -schainpy/scripts/testDigitalRF.py -schainpy/scripts/testDigitalRFWriter.py diff --git a/schainpy/CHANGELOG.md b/CHANGELOG.md similarity index 78% rename from schainpy/CHANGELOG.md rename to CHANGELOG.md index 4f4d159..ab3a8ca 100644 --- a/schainpy/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,6 +1,15 @@ -## CHANGELOG: - -### 2.3 +# CHANGELOG: + +## 3.0 +* Python 3.x & 2.X compatible +* New architecture with multiprocessing support +* Add @MPDecorator for multiprocessing Operations (Plots, Writers and Publishers) +* Added new type of operation `external` for non-locking operations +* New plotting architecture with buffering/throttle capabilities to speed up plots +* Clean controller to optimize scripts (format & optype are no longer required) +* New GUI with dinamic load of Units and operations (use Kivy framework) + +## 2.3 * Added support for Madrigal formats (reading/writing). * Added support for reading BLTR parameters (*.sswma). * Added support for reading Julia format (*.dat). @@ -21,83 +30,83 @@ * Updated README for MAC OS GUI installation. * Setup now installs numpy. -### 2.2.6 +## 2.2.6 * Graphics generated by the GUI are now the same as generated by scripts. Issue #1074. * Added support for C extensions. * Function `hildebrand_sehkon` optimized with a C wrapper. * Numpy version updated. * Migration to GIT. -### 2.2.5: +## 2.2.5: * splitProfiles and combineProfiles modules were added to VoltageProc and Signal Chain GUI. * nProfiles of USRP data (hdf5) is the number of profiles thera are in one second. * jroPlotter works directly with data objects instead of dictionaries * script "schain" was added to Signal Chain installer -### 2.2.4.1: +## 2.2.4.1: * jroIO_usrp.py is update to read Sandra's data * decimation in Spectra and RTI plots is always enabled. * time* window option added to GUI -### 2.2.4: +## 2.2.4: * jroproc_spectra_lags.py added to schainpy * Bug fixed in schainGUI: ProcUnit was created with the same id in some cases. * Bug fixed in jroHeaderIO: Header size validation. -### 2.2.3.1: +## 2.2.3.1: * Filtering block by time has been added. * Bug fixed plotting RTI, CoherenceMap and others using xmin and xmax parameters. The first day worked properly but the next days did not. -### 2.2.3: +## 2.2.3: * Bug fixed in GUI: Error getting(reading) Code value * Bug fixed in GUI: Flip option always needs channelList field * Bug fixed in jrodata: when one branch modified a value in "dataOut" (example: dataOut.code) this value was modified for every branch (because this was a reference). It was modified in data.copy() * Bug fixed in jroproc_voltage.profileSelector(): rangeList replaces to profileRangeList. -### 2.2.2: +## 2.2.2: * VoltageProc: ProfileSelector, Reshape, Decoder with nTxs!=1 and getblock=True was tested * Rawdata and testRawdata.py added to Signal Chain project -### 2.2.1: +## 2.2.1: * Bugs fixed in GUI * Views were improved in GUI * Support to MST* ISR experiments * Bug fixed getting noise using hyldebrant. (minimum number of points > 20%) * handleError added to jroplotter.py -### 2.2.0: +## 2.2.0: * GUI: use of external plotter * Compatible with matplotlib 1.5.0 -### 2.1.5: +## 2.1.5: * serializer module added to Signal Chain * jroplotter.py added to Signal Chain -### 2.1.4.2: +## 2.1.4.2: * A new Plotter Class was added * Project.start() does not accept filename as a parameter anymore -### 2.1.4.1: +## 2.1.4.1: * Send notifications when an error different to ValueError is detected -### 2.1.4: +## 2.1.4: * Sending error notifications to signal chain administrator * Login to email server added -### 2.1.3.3: +## 2.1.3.3: * Colored Button Icons were added to GUI -### 2.1.3.2: +## 2.1.3.2: * GUI: user interaction enhanced * controller_api.py: Safe access to ControllerThead -### 2.1.3.1: +## 2.1.3.1: * GUI: every icon were resized * jroproc_voltage.py: Print a message when "Read from code" option is selected and the code is not defined inside data file -### 2.1.3: +## 2.1.3: * jroplot_heispectra.py: SpectraHeisScope was not showing the right channels * jroproc_voltage.py: Bug fixed selecting profiles (self.nProfiles took a wrong value), Bug fixed selecting heights by block (selecting profiles instead heights) @@ -105,7 +114,7 @@ was modified for every branch (because this was a reference). It was modified in * jroIO_heispectra.py: Bug fixed in FitsReader. Using local Fits instance instead schainpy.mode.data.jrodata.Fits. * jroIO_heispectra.py: Channel index list does not exist. -### 2.1.2: +## 2.1.2: * jroutils_ftp.py: Bug fixed, Any error sending file stopped the Server Thread Server thread opens and closes remote server each time file list is sent * jroplot_spectra.py: Noise path was not being created when noise data is saved. diff --git a/Dockerfile b/Dockerfile index 7c27fc1..8545134 100644 --- a/Dockerfile +++ b/Dockerfile @@ -1,4 +1,4 @@ -FROM python:2.7-slim +FROM python:3.7-slim RUN apt-get clean && apt-get update && apt-get install -y --no-install-recommends \ git \ @@ -10,7 +10,7 @@ RUN apt-get clean && apt-get update && apt-get install -y --no-install-recommend libatlas-base-dev \ libssl-dev \ libhdf5-dev \ - && git clone --branch v2.3 --depth 1 \ + && git clone --branch v3.0-devel --depth 1 \ http://jro-dev.igp.gob.pe/rhodecode/schain \ && pip install numpy \ && cd schain \ diff --git a/README.md b/README.md index a04a8a8..6d25e2d 100644 --- a/README.md +++ b/README.md @@ -6,33 +6,29 @@ Signal Chain (SCh) is a radar data processing library developed using [Python](w ## Installation -Install system dependencies, clone the latest version from [here](http://jro-dev.igp.gob.pe/rhodecode/schain/) and install it as a normal python package. +Install system dependencies, clone the latest version from [here](http://jro-dev.igp.gob.pe/rhodecode/schain/) and install it as a normal python package, we strongly recommend to use Anaconda or a virtual environment for the installation. -### Linux based system +### Dependencies +- GCC (gcc or gfortran) +- Python.h (python-dev or python-devel) +- Python-TK (python-tk) +- HDF5 libraries (libhdf5-dev) + +### Linux based system (e.g. ubuntu) ``` -$ sudo apt-get install python-pip python-dev gfortran libpng-dev freetype* libblas-dev liblapack-dev libatlas-base-dev python-qt4 python-tk libssl-dev libhdf5-dev $ git clone http://jro-dev.igp.gob.pe/rhodecode/schain/ $ cd schain +$ git checkout `schain-branch` (optional) $ sudo pip install ./ - ``` ### MAC Os ``` $ brew install python -$ brew install cartr/qt4/pyqt $ git clone http://jro-dev.igp.gob.pe/rhodecode/schain/ $ cd schain -$ pip install ./ -``` - -**It is recommended to install schain in a virtual environment** -``` -$ virtualenv /path/to/virtual -$ source /path/to/virtual/bin/activate -(virtual) $ cd schain -(virtual) $ pip install ./ -(virtual) $ bash link_PyQt4.sh +$ git checkout `schain-branch` (optional) +$ sudo pip install ./ ``` ### Docker @@ -46,66 +42,75 @@ $ docker build -t schain . You can run a container using an xml file or a schain script also you need to mount a volume for the data input and for the output files/plots ``` $ docker run -it --rm --volume /path/to/host/data:/data schain xml /data/test.xml -$ docker run -it --rm --volume /path/to/host/data:/data --entrypoint=/bin/python schain /data/test.py +$ docker run -it --rm --volume /path/to/host/data:/data --entrypoint /urs/local/bin/python schain /data/test.py ``` +## CLI (command line interface) + +Signal Chain provides the following commands: + +- schainGUI: Open the GUI +- schain: Signal chain command line + + ## First Script -Read Spectra data (.pdata) - remove dc - plot spectra & RTI +Here you can find an script to read Spectra data (.pdata), remove dc and plot spectra & RTI -Import SCh and creating a project +First import SCh and creating a project ```python #!/usr/bin/python from schainpy.controller import Project -controller = Project() -controller.setup(id = '100', - name='test', - description='Basic experiment') - - +prj = Project() +prj.setup( + id = '100', + name='test', + description='Basic experiment' + ) ``` -Adding read unit and operations +Add read unit and operations ```python -read_unit = controller.addReadUnit(datatype='Spectra', - path='/path/to/pdata/', - startDate='2014/01/31', - endDate='2014/03/31', - startTime='00:00:00', - endTime='23:59:59', - online=0, - walk=0) - -proc_unit = controller.addProcUnit(datatype='Spectra', - inputId=read_unit.getId()) +read_unit = prj.addReadUnit( + datatype='Spectra', + path='/path/to/pdata/', + startDate='2014/01/31', + endDate='2014/03/31', + startTime='00:00:00', + endTime='23:59:59', + online=0, + walk=0 + ) + +proc_unit = prj.addProcUnit(datatype='Spectra', inputId=read_unit.getId()) op = proc_unit.addOperation(name='selectChannels') -op.addParameter(name='channelList', value='0,1', format='intlist') +op.addParameter(name='channelList', value='0,1') op = proc_unit.addOperation(name='selectHeights') -op.addParameter(name='minHei', value='80', format='float') -op.addParameter(name='maxHei', value='200', format='float') +op.addParameter(name='minHei', value='80') +op.addParameter(name='maxHei', value='200') op = proc_unit.addOperation(name='removeDC') ``` -Plotting data & start project +Plot data & start project ```python -op = proc_unit.addOperation(name='SpectraPlot', optype='other') -op.addParameter(name='id', value='1', format='int') -op.addParameter(name='wintitle', value='Spectra', format='str') +op = proc_unit.addOperation(name='SpectraPlot') +op.addParameter(name='id', value='1') +op.addParameter(name='wintitle', value='Spectra') -op = procUnitConfObj1.addOperation(name='RTIPlot', optype='other') -op.addParameter(name='id', value='2', format='int') -op.addParameter(name='wintitle', value='RTI', format='str') +op = procUnitConfObj1.addOperation(name='RTIPlot') +op.addParameter(name='id', value='2') +op.addParameter(name='wintitle', value='RTI') -controller.start() +prj.start() ``` @@ -115,13 +120,13 @@ Full script ```python #!/usr/bin/python -from schainpy.controller import Project +from schainpy.prj import Project -controller = Project() -controller.setup(id = '100', +prj = Project() +prj.setup(id = '100', name='test', description='Basic experiment') -read_unit = controller.addReadUnit(datatype='Spectra', +read_unit = prj.addReadUnit(datatype='Spectra', path='/path/to/pdata/', startDate='2014/01/31', endDate='2014/03/31', @@ -130,26 +135,24 @@ read_unit = controller.addReadUnit(datatype='Spectra', online=0, walk=0) -proc_unit = controller.addProcUnit(datatype='Spectra', +proc_unit = prj.addProcUnit(datatype='Spectra', inputId=read_unit.getId()) op = proc_unit.addOperation(name='selectChannels') -op.addParameter(name='channelList', value='0,1', format='intlist') +op.addParameter(name='channelList', value='0,1') op = proc_unit.addOperation(name='selectHeights') -op.addParameter(name='minHei', value='80', format='float') -op.addParameter(name='maxHei', value='200', format='float') +op.addParameter(name='minHei', value='80') +op.addParameter(name='maxHei', value='200') op = proc_unit.addOperation(name='removeDC') -op = proc_unit.addOperation(name='SpectraPlot', optype='other') -op.addParameter(name='id', value='6', format='int') +op = proc_unit.addOperation(name='SpectraPlot') op.addParameter(name='wintitle', value='Spectra', format='str') -op = procUnitConfObj1.addOperation(name='RTIPlot', optype='other') -op.addParameter(name='id', value='2', format='int') +op = procUnitConfObj1.addOperation(name='RTIPlot') op.addParameter(name='wintitle', value='RTI', format='str') -controller.start() +prj.start() ``` \ No newline at end of file diff --git a/link_PyQt4.sh b/link_PyQt4.sh deleted file mode 100755 index bc9f852..0000000 --- a/link_PyQt4.sh +++ /dev/null @@ -1,15 +0,0 @@ -#!/bin/bash -# This hook is run after a new virtualenv is activated. - -python_version=python$(python -c "import sys; print (str(sys.version_info[0])+'.'+str(sys.version_info[1]))") -var=( $(which -a $python_version) ) - -get_python_lib_cmd="from distutils.sysconfig import get_python_lib; print (get_python_lib())" -lib_virtualenv_path=$(python -c "$get_python_lib_cmd") -lib_system_path=$(${var[-1]} -c "$get_python_lib_cmd") -sip_path=$(ls $lib_system_path/sip*.so) - -echo "Linking Qt4..." -ln -s $lib_system_path/PyQt4 $lib_virtualenv_path/PyQt4 -echo "Linking SIP..." -ln -s $sip_path $lib_virtualenv_path/sip.so diff --git a/schainc/_noise.c b/schainc/_noise.c new file mode 100644 index 0000000..32100c8 --- /dev/null +++ b/schainc/_noise.c @@ -0,0 +1,82 @@ +#include +#include +#include + + +static PyObject *hildebrand_sekhon(PyObject *self, PyObject *args) { + double navg; + PyObject *data_obj, *data_array; + + if (!PyArg_ParseTuple(args, "Od", &data_obj, &navg)) { + return NULL; + } + + data_array = PyArray_FROM_OTF(data_obj, NPY_FLOAT64, NPY_IN_ARRAY); + + if (data_array == NULL) { + Py_XDECREF(data_array); + Py_XDECREF(data_obj); + return NULL; + } + double *sortdata = (double*)PyArray_DATA(data_array); + int lenOfData = (int)PyArray_SIZE(data_array) ; + double nums_min = lenOfData*0.2; + if (nums_min <= 5) nums_min = 5; + double sump = 0; + double sumq = 0; + int j = 0; + int cont = 1; + double rtest = 0; + while ((cont == 1) && (j < lenOfData)) { + sump = sump + sortdata[j]; + sumq = sumq + pow(sortdata[j], 2); + if (j > nums_min) { + rtest = (double)j/(j-1) + 1/navg; + if ((sumq*j) > (rtest*pow(sump, 2))) { + j = j - 1; + sump = sump - sortdata[j]; + sumq = sumq - pow(sortdata[j],2); + cont = 0; + } + } + j = j + 1; + } + + double lnoise = sump / j; + + Py_DECREF(data_array); + + return PyLong_FromLong(lnoise); + //return Py_BuildValue("d", lnoise); +} + + +static PyMethodDef noiseMethods[] = { + { "hildebrand_sekhon", hildebrand_sekhon, METH_VARARGS, "Get noise with hildebrand_sekhon algorithm" }, + { NULL, NULL, 0, NULL } +}; + +#if PY_MAJOR_VERSION >= 3 + +static struct PyModuleDef noisemodule = { + PyModuleDef_HEAD_INIT, + "_noise", + "Get noise with hildebrand_sekhon algorithm", + -1, + noiseMethods +}; + +#endif + +#if PY_MAJOR_VERSION >= 3 + PyMODINIT_FUNC PyInit__noise(void) { + Py_Initialize(); + import_array(); + return PyModule_Create(&noisemodule); + } +#else + PyMODINIT_FUNC init_noise() { + Py_InitModule("_noise", noiseMethods); + import_array(); + } +#endif diff --git a/schainpy/ROADMAP.md b/schainpy/ROADMAP.md deleted file mode 100644 index 80b3de6..0000000 --- a/schainpy/ROADMAP.md +++ /dev/null @@ -1,10 +0,0 @@ -ROADMAP FOR SCHAIN BRANCHES -=============================== - -### BRANCH - SCHAIN_MP -* Revisar si funciona con varios publishers. -* Revisar xRange y reinicialización de gráfico. -* Grabar cada spectra independientemente. -* Agregar kwargs al init -* Agregar gráficos restantes -* Presentación diff --git a/schainpy/__init__.py b/schainpy/__init__.py index 4654380..d51d41f 100644 --- a/schainpy/__init__.py +++ b/schainpy/__init__.py @@ -1,7 +1,7 @@ ''' -Created on Feb 7, 2012 +Created on Jul 3, 2018 @author $Author$ @version $Id$ ''' -__version__ = '2.3' +__version__ = '3.0' diff --git a/schainpy/admin.py b/schainpy/admin.py index ef2ed57..5eb45dc 100644 --- a/schainpy/admin.py +++ b/schainpy/admin.py @@ -11,8 +11,11 @@ import sys import time import traceback import smtplib -import ConfigParser -import StringIO +if sys.version[0] == '3': + from configparser import ConfigParser +else: + from ConfigParser import ConfigParser +import io from threading import Thread from multiprocessing import Process from email.mime.text import MIMEText @@ -21,7 +24,7 @@ from email.mime.multipart import MIMEMultipart import schainpy from schainpy.utils import log -from schainpy.model.graphics.jroplot_data import popup +from schainpy.model.graphics.jroplot_base import popup def get_path(): ''' @@ -65,7 +68,6 @@ class Alarm(Process): @staticmethod def send_email(**kwargs): notifier = SchainNotify() - print kwargs notifier.notify(**kwargs) @staticmethod @@ -144,10 +146,10 @@ class SchainConfigure(): return # create Parser using standard module ConfigParser - self.__parser = ConfigParser.ConfigParser() + self.__parser = ConfigParser() # read conf file into a StringIO with "[madrigal]\n" section heading prepended - strConfFile = StringIO.StringIO("[schain]\n" + self.__confFile.read()) + strConfFile = io.StringIO("[schain]\n" + self.__confFile.read()) # parse StringIO configuration file self.__parser.readfp(strConfFile) @@ -287,7 +289,7 @@ class SchainNotify: msg = MIMEMultipart() msg['Subject'] = subject - msg['From'] = "(Python SChain API): " + email_from + msg['From'] = "SChain API (v{}) <{}>".format(schainpy.__version__, email_from) msg['Reply-to'] = email_from msg['To'] = email_to @@ -327,7 +329,7 @@ class SchainNotify: smtp.login(self.__emailFromAddress, self.__emailPass) # Send the email - try: + try: smtp.sendmail(msg['From'], msg['To'], msg.as_string()) except: log.error('Could not send the email to {}'.format(msg['To']), 'System') @@ -355,7 +357,7 @@ class SchainNotify: if not self.__emailToAddress: return 0 - print "***** Sending alert to %s *****" %self.__emailToAddress + print("***** Sending alert to %s *****" %self.__emailToAddress) # set up message sent=self.sendEmail(email_from=self.__emailFromAddress, @@ -500,4 +502,4 @@ if __name__ == '__main__': test.sendAlert('This is a message from the python module SchainNotify', 'Test from SchainNotify') - print 'Hopefully message sent - check.' + print('Hopefully message sent - check.') \ No newline at end of file diff --git a/schainpy/cli/README.md b/schainpy/cli/README.md deleted file mode 100644 index 1cc9847..0000000 --- a/schainpy/cli/README.md +++ /dev/null @@ -1,9 +0,0 @@ -# schain - -Command Line Interface for SIGNAL CHAIN - jro - -# Usage - -To use it: - - $ schain-cli --help diff --git a/schainpy/cli/cli.py b/schainpy/cli/cli.py index 9480dfc..ef7089b 100644 --- a/schainpy/cli/cli.py +++ b/schainpy/cli/cli.py @@ -1,21 +1,81 @@ import click -import schainpy import subprocess import os import sys import glob -save_stdout = sys.stdout -sys.stdout = open('trash', 'w') -from multiprocessing import cpu_count +import schainpy from schainpy.controller import Project from schainpy.model import Operation, ProcessingUnit from schainpy.utils import log from importlib import import_module from pydoc import locate from fuzzywuzzy import process -from schainpy.utils import paramsFinder -import templates -sys.stdout = save_stdout +from schainpy.cli import templates +import inspect +try: + from queue import Queue +except: + from Queue import Queue + + +def getProcs(): + modules = dir(schainpy.model) + procs = check_module(modules, 'processing') + try: + procs.remove('ProcessingUnit') + except Exception as e: + pass + return procs + +def getOperations(): + module = dir(schainpy.model) + noProcs = [x for x in module if not x.endswith('Proc')] + operations = check_module(noProcs, 'operation') + try: + operations.remove('Operation') + operations.remove('Figure') + operations.remove('Plot') + except Exception as e: + pass + return operations + +def getArgs(op): + module = locate('schainpy.model.{}'.format(op)) + try: + obj = module(1, 2, 3, Queue()) + except: + obj = module() + + if hasattr(obj, '__attrs__'): + args = obj.__attrs__ + else: + if hasattr(obj, 'myrun'): + args = inspect.getfullargspec(obj.myrun).args + else: + args = inspect.getfullargspec(obj.run).args + + try: + args.remove('self') + except Exception as e: + pass + try: + args.remove('dataOut') + except Exception as e: + pass + return args + +def getDoc(obj): + module = locate('schainpy.model.{}'.format(obj)) + try: + obj = module(1, 2, 3, Queue()) + except: + obj = module() + return obj.__doc__ + +def getAll(): + modules = getOperations() + modules.extend(getProcs()) + return modules def print_version(ctx, param, value): @@ -25,21 +85,21 @@ def print_version(ctx, param, value): ctx.exit() -cliLogger = log.makelogger('schain cli') PREFIX = 'experiment' - @click.command() @click.option('--version', '-v', is_flag=True, callback=print_version, help='SChain version', type=str) @click.argument('command', default='run', required=True) @click.argument('nextcommand', default=None, required=False, type=str) def main(command, nextcommand, version): - """COMMAND LINE INTERFACE FOR SIGNAL CHAIN - JICAMARCA RADIO OBSERVATORY \n - Available commands.\n - --xml: runs a schain XML generated file\n - run: runs any python script starting 'experiment_'\n + """COMMAND LINE INTERFACE FOR SIGNAL CHAIN - JICAMARCA RADIO OBSERVATORY V3.0\n + Available commands:\n + xml: runs a schain XML generated file\n + run: runs any python script'\n generate: generates a template schain script\n - search: return avilable operations, procs or arguments of the give operation/proc\n""" + list: return a list of available procs and operations\n + search: return avilable operations, procs or arguments of the given + operation/proc\n""" if command == 'xml': runFromXML(nextcommand) elif command == 'generate': @@ -50,6 +110,8 @@ def main(command, nextcommand, version): runschain(nextcommand) elif command == 'search': search(nextcommand) + elif command == 'list': + cmdlist(nextcommand) else: log.error('Command {} is not defined'.format(command)) @@ -58,7 +120,8 @@ def check_module(possible, instance): def check(x): try: instancia = locate('schainpy.model.{}'.format(x)) - return isinstance(instancia(), instance) + ret = instancia.proc_type == instance + return ret except Exception as e: return False clean = clean_modules(possible) @@ -71,36 +134,35 @@ def clean_modules(module): noFullUpper = [x for x in noStartUnder if not x.isupper()] return noFullUpper - -def search(nextcommand): +def cmdlist(nextcommand): if nextcommand is None: - log.error('There is no Operation/ProcessingUnit to search') + log.error('Missing argument, available arguments: procs, operations', '') elif nextcommand == 'procs': - procs = paramsFinder.getProcs() + procs = getProcs() log.success( - 'Current ProcessingUnits are:\n\033[1m{}\033[0m'.format('\n'.join(procs))) - + 'Current ProcessingUnits are:\n {}'.format('\n '.join(procs)), '') elif nextcommand == 'operations': - operations = paramsFinder.getOperations() - log.success('Current Operations are:\n\033[1m{}\033[0m'.format( - '\n'.join(operations))) + operations = getOperations() + log.success('Current Operations are:\n {}'.format( + '\n '.join(operations)), '') + else: + log.error('Wrong argument', '') + +def search(nextcommand): + if nextcommand is None: + log.error('There is no Operation/ProcessingUnit to search', '') else: try: - args = paramsFinder.getArgs(nextcommand) - log.warning( - 'Use this feature with caution. It may not return all the allowed arguments') - if len(args) == 0: - log.success('{} has no arguments'.format(nextcommand)) - else: - log.success('Showing {} arguments:\n\033[1m{}\033[0m'.format( - nextcommand, '\n'.join(args))) + args = getArgs(nextcommand) + doc = getDoc(nextcommand) + log.success('{}\n{}\n\narguments:\n {}'.format( + nextcommand, doc, ', '.join(args)), '' + ) except Exception as e: - log.error('Module {} does not exists'.format(nextcommand)) - allModules = paramsFinder.getAll() - similar = process.extractOne(nextcommand, allModules)[0] - log.success('Showing {} instead'.format(similar)) - search(similar) - + log.error('Module `{}` does not exists'.format(nextcommand), '') + allModules = getAll() + similar = [t[0] for t in process.extract(nextcommand, allModules, limit=12) if t[1]>80] + log.success('Possible modules are: {}'.format(', '.join(similar)), '') def runschain(nextcommand): if nextcommand is None: @@ -121,16 +183,24 @@ def runschain(nextcommand): def basicInputs(): inputs = {} - inputs['desc'] = click.prompt( - 'Enter a description', default="A schain project", type=str) inputs['name'] = click.prompt( 'Name of the project', default="project", type=str) + inputs['desc'] = click.prompt( + 'Enter a description', default="A schain project", type=str) + inputs['multiprocess'] = click.prompt( + '''Select data type: + + - Voltage (*.r): [1] + - Spectra (*.pdata): [2] + - Voltage and Spectra (*.r): [3] + + -->''', type=int) inputs['path'] = click.prompt('Data path', default=os.getcwd( ), type=click.Path(exists=True, resolve_path=True)) inputs['startDate'] = click.prompt( 'Start date', default='1970/01/01', type=str) inputs['endDate'] = click.prompt( - 'End date', default='2017/12/31', type=str) + 'End date', default='2018/12/31', type=str) inputs['startHour'] = click.prompt( 'Start hour', default='00:00:00', type=str) inputs['endHour'] = click.prompt('End hour', default='23:59:59', type=str) @@ -140,13 +210,13 @@ def basicInputs(): def generate(): inputs = basicInputs() - inputs['multiprocess'] = click.confirm('Is this a multiprocess script?') - if inputs['multiprocess']: - inputs['nProcess'] = click.prompt( - 'How many process?', default=cpu_count(), type=int) - current = templates.multiprocess.format(**inputs) - else: - current = templates.basic.format(**inputs) + + if inputs['multiprocess'] == 1: + current = templates.voltage.format(**inputs) + elif inputs['multiprocess'] == 2: + current = templates.spectra.format(**inputs) + elif inputs['multiprocess'] == 3: + current = templates.voltagespectra.format(**inputs) scriptname = '{}_{}.py'.format(PREFIX, inputs['name']) script = open(scriptname, 'w') try: diff --git a/schainpy/cli/templates.py b/schainpy/cli/templates.py index 29ad580..a93fb19 100644 --- a/schainpy/cli/templates.py +++ b/schainpy/cli/templates.py @@ -1,38 +1,217 @@ -basic = '''from schainpy.controller import Project +voltage = '''import os, sys, time +from schainpy.controller import Project + + +def main(): + desc = "{desc}" + controller = Project() + controller.setup(id='200', name="{name}", description=desc) + + read_unit = controller.addReadUnit(datatype='Voltage', + path="{path}", + startDate="{startDate}", + endDate="{endDate}", + startTime="{startHour}", + endTime="{endHour}", + online=0, + verbose=1, + walk=0, + delay=180, + ) + + code = '[[1, 1, -1], [-1, -1, 1], [1, 1, -1], [1, 1, -1], [1, 1, -1], [-1, -1, 1], [1, 1, -1], [1, 1, -1], [1, 1, -1], [1, 1, -1], [-1, -1, 1], [1, 1, -1], [1, 1, -1], [1, 1, -1], [-1, -1, 1], [-1, -1, 1], [1, 1, -1], [-1, -1, 1], [1, 1, -1], [-1, -1, 1], [1, 1, -1], [1, 1, -1], [-1, -1, 1], [1, 1, -1], [-1, -1, 1], [-1, -1, 1], [1, 1, -1], [1, 1, -1], [1, 1, -1], [1, 1, -1], [1, 1, -1], [1, 1, -1], [-1, -1, 1], [1, 1, -1], [-1, -1, 1], [-1, -1, 1], [1, 1, -1], [-1, -1, 1], [1, 1, -1], [1, 1, -1], [1, 1, -1], [1, 1, -1], [-1, -1, 1], [1, 1, -1], [-1, -1, 1], [-1, -1, 1], [-1, -1, 1], [-1, -1, 1], [1, 1, -1], [1, 1, -1], [1, 1, -1], [-1, -1, 1], [1, 1, -1], [-1, -1, 1], [-1, -1, 1], [-1, -1, 1], [1, 1, -1], [1, 1, -1], [-1, -1, 1], [1, 1, -1], [-1, -1, 1], [1, 1, -1], [1, 1, -1], [1, 1, -1], [1, 1, -1], [1, 1, -1], [1, 1, -1], [-1, -1, 1], [1, 1, -1], [-1, -1, 1], [1, 1, -1], [-1, -1, 1], [-1, -1, 1], [-1, -1, 1], [-1, -1, 1], [1, 1, -1], [-1, -1, 1], [-1, -1, 1], [1, 1, -1], [-1, -1, 1], [1, 1, -1], [1, 1, -1], [1, 1, -1], [-1, -1, 1], [1, 1, -1], [1, 1, -1], [-1, -1, 1], [-1, -1, 1], [1, 1, -1], [-1, -1, 1], [-1, -1, 1], [-1, -1, 1], [-1, -1, 1], [-1, -1, 1], [1, 1, -1], [-1, -1, 1], [1, 1, -1], [1, 1, -1], [1, 1, -1], [1, 1, -1], [-1, -1, 1], [-1, -1, 1], [1, 1, -1], [1, 1, -1], [-1, -1, 1], [-1, -1, 1], [-1, -1, 1], [-1, -1, 1], [-1, -1, 1], [-1, -1, 1], [1, 1, -1], [1, 1, -1], [1, 1, -1], [1, 1, -1], [1, 1, -1], [-1, -1, 1], [-1, -1, 1], [1, 1, -1], [-1, -1, 1], [-1, -1, 1], [-1, -1, 1], [1, 1, -1], [-1, -1, 1], [1, 1, -1], [-1, -1, 1], [-1, -1, 1], [-1, -1, 1], [1, 1, -1]]' + nCode = '128' + nBaud = '3' + + + proc_voltage = controller.addProcUnit(name='VoltageProc', inputId=read_unit.getId()) + + op1 = proc_voltage.addOperation(name='selectChannels', optype='self') + op1.addParameter(name='channelList', value='0, 1, 2, 3', format='intlist') + + op2 = proc_voltage.addOperation(name='filterByHeights', optype='self') + op2.addParameter(name='window', value='4', format='int') + + op3 = proc_voltage.addOperation(name='ProfileSelector', optype='other') + op3.addParameter(name='profileRangeList', value='32, 159', format='intList') + + op4 = proc_voltage.addOperation(name='Decoder', optype='other') + op4.addParameter(name='code', value=code, format='intlist') + op4.addParameter(name='nCode', value=nCode, format='int') + op4.addParameter(name='nBaud', value=nBaud, format='int') + op4.addParameter(name='mode', value='0', format='int') + + op5 = proc_voltage.addOperation(name='Scope', optype='external') + op5.addParameter(name='id', value='30', format='int') + + + + + + controller.start() + +if __name__ == '__main__': + import time + start_time = time.time() + main() + print("--- %s seconds ---" % (time.time() - start_time)) -desc = "{desc}" -project = Project() -project.setup(id='200', name="{name}", description=desc) - -voltage_reader = project.addReadUnit(datatype='VoltageReader', - path="{path}", - startDate="{startDate}", - endDate="{endDate}", - startTime="{startHour}", - endTime="{endHour}", - online=0, - verbose=1, - walk=1, - ) - -voltage_proc = project.addProcUnit(datatype='VoltageProc', inputId=voltage_reader.getId()) - -profile = voltage_proc.addOperation(name='ProfileSelector', optype='other') -profile.addParameter(name='profileRangeList', value='120,183', format='intlist') - -rti = voltage_proc.addOperation(name='RTIPlot', optype='other') -rti.addParameter(name='wintitle', value='Jicamarca Radio Observatory', format='str') -rti.addParameter(name='showprofile', value='0', format='int') -rti.addParameter(name='xmin', value='0', format='int') -rti.addParameter(name='xmax', value='24', format='int') -rti.addParameter(name='figpath', value="{figpath}", format='str') -rti.addParameter(name='wr_period', value='5', format='int') -rti.addParameter(name='exp_code', value='22', format='int') - - -project.start() ''' + +spectra = '''import os, sys, time +from schainpy.controller import Project + + +def main(): + desc = "{desc}" + controller = Project() + controller.setup(id='300', name="{name}", description=desc) + + read_unit = controller.addReadUnit(datatype='Spectra', + path="{path}", + startDate="{startDate}", + endDate="{endDate}", + startTime="{startHour}", + endTime="{endHour}", + online=0, + verbose=1, + walk=0, + delay=180, + ) + + proc_spectra = controller.addProcUnit(datatype='Spectra', inputId=read_unit.getId()) + proc_spectra.addParameter(name='nFFTPoints', value='128', format='int') + proc_spectra.addParameter(name='nProfiles', value='128', format='int') + proc_spectra.addParameter(name='pairsList', value='(0, 1), (2, 3)', format='pairslist') + + op1 = proc_spectra.addOperation(name='IncohInt', optype='other') + op1.addParameter(name='n', value='4', format='int') + + op2 = proc_spectra.addOperation(name='CrossSpectraPlot', optype='external') + op2.addParameter(name='id', value='10', format='int') + op2.addParameter(name='zmin', value='10.0', format='float') + op2.addParameter(name='zmax', value='35.0', format='float') + + + op3 = proc_spectra.addOperation(name='RTIPlot', optype='external') + op3.addParameter(name='id', value='20', format='int') + op3.addParameter(name='wintitle', value='RTI', format='str') + op3.addParameter(name='xmin', value='0', format='float') + op3.addParameter(name='xmax', value='24', format='float') + op3.addParameter(name='zmin', value='12', format='int') + op3.addParameter(name='zmax', value='32', format='int') + op3.addParameter(name='showprofile', value='1', format='int') + op3.addParameter(name='timerange', value=str(24*60*60), format='int') + + op4 = proc_spectra.addOperation(name='CoherenceMap', optype='external') + op4.addParameter(name='id', value='30', format='int') + op4.addParameter(name='xmin', value='0.0', format='float') + op4.addParameter(name='xmax', value='24.0', format='float') + + + controller.start() + +if __name__ == '__main__': + import time + start_time = time.time() + main() + print("--- %s seconds ---" % (time.time() - start_time)) + +''' + +voltagespectra = '''import os, sys, time +from schainpy.controller import Project + + +def main(): + desc = "{desc}" + controller = Project() + controller.setup(id='400', name="{name}", description=desc) + + read_unit = controller.addReadUnit(datatype='Voltage', + path="{path}", + startDate="{startDate}", + endDate="{endDate}", + startTime="{startHour}", + endTime="{endHour}", + online=0, + verbose=1, + walk=0, + delay=180, + ) + + code = '[[1, 1, -1], [-1, -1, 1], [1, 1, -1], [1, 1, -1], [1, 1, -1], [-1, -1, 1], [1, 1, -1], [1, 1, -1], [1, 1, -1], [1, 1, -1], [-1, -1, 1], [1, 1, -1], [1, 1, -1], [1, 1, -1], [-1, -1, 1], [-1, -1, 1], [1, 1, -1], [-1, -1, 1], [1, 1, -1], [-1, -1, 1], [1, 1, -1], [1, 1, -1], [-1, -1, 1], [1, 1, -1], [-1, -1, 1], [-1, -1, 1], [1, 1, -1], [1, 1, -1], [1, 1, -1], [1, 1, -1], [1, 1, -1], [1, 1, -1], [-1, -1, 1], [1, 1, -1], [-1, -1, 1], [-1, -1, 1], [1, 1, -1], [-1, -1, 1], [1, 1, -1], [1, 1, -1], [1, 1, -1], [1, 1, -1], [-1, -1, 1], [1, 1, -1], [-1, -1, 1], [-1, -1, 1], [-1, -1, 1], [-1, -1, 1], [1, 1, -1], [1, 1, -1], [1, 1, -1], [-1, -1, 1], [1, 1, -1], [-1, -1, 1], [-1, -1, 1], [-1, -1, 1], [1, 1, -1], [1, 1, -1], [-1, -1, 1], [1, 1, -1], [-1, -1, 1], [1, 1, -1], [1, 1, -1], [1, 1, -1], [1, 1, -1], [1, 1, -1], [1, 1, -1], [-1, -1, 1], [1, 1, -1], [-1, -1, 1], [1, 1, -1], [-1, -1, 1], [-1, -1, 1], [-1, -1, 1], [-1, -1, 1], [1, 1, -1], [-1, -1, 1], [-1, -1, 1], [1, 1, -1], [-1, -1, 1], [1, 1, -1], [1, 1, -1], [1, 1, -1], [-1, -1, 1], [1, 1, -1], [1, 1, -1], [-1, -1, 1], [-1, -1, 1], [1, 1, -1], [-1, -1, 1], [-1, -1, 1], [-1, -1, 1], [-1, -1, 1], [-1, -1, 1], [1, 1, -1], [-1, -1, 1], [1, 1, -1], [1, 1, -1], [1, 1, -1], [1, 1, -1], [-1, -1, 1], [-1, -1, 1], [1, 1, -1], [1, 1, -1], [-1, -1, 1], [-1, -1, 1], [-1, -1, 1], [-1, -1, 1], [-1, -1, 1], [-1, -1, 1], [1, 1, -1], [1, 1, -1], [1, 1, -1], [1, 1, -1], [1, 1, -1], [-1, -1, 1], [-1, -1, 1], [1, 1, -1], [-1, -1, 1], [-1, -1, 1], [-1, -1, 1], [1, 1, -1], [-1, -1, 1], [1, 1, -1], [-1, -1, 1], [-1, -1, 1], [-1, -1, 1], [1, 1, -1]]' + nCode = '128' + nBaud = '3' + + + proc_voltage = controller.addProcUnit(name='VoltageProc', inputId=read_unit.getId()) + + op1 = proc_voltage.addOperation(name='selectChannels', optype='self') + op1.addParameter(name='channelList', value='0, 1, 2, 3', format='intlist') + + op2 = proc_voltage.addOperation(name='filterByHeights', optype='self') + op2.addParameter(name='window', value='4', format='int') + + op3 = proc_voltage.addOperation(name='ProfileSelector', optype='other') + op3.addParameter(name='profileRangeList', value='32, 159', format='intList') + + op4 = proc_voltage.addOperation(name='Decoder', optype='other') + op4.addParameter(name='code', value=code, format='intlist') + op4.addParameter(name='nCode', value=nCode, format='int') + op4.addParameter(name='nBaud', value=nBaud, format='int') + op4.addParameter(name='mode', value='0', format='int') + + + + proc_spectra = controller.addProcUnit(datatype='Spectra', inputId=proc_voltage.getId()) + proc_spectra.addParameter(name='nFFTPoints', value='128', format='int') + proc_spectra.addParameter(name='nProfiles', value='128', format='int') + proc_spectra.addParameter(name='pairsList', value='(0, 1), (2, 3)', format='pairslist') + + op5 = proc_spectra.addOperation(name='IncohInt', optype='other') + op5.addParameter(name='n', value='4', format='int') + + op6 = proc_spectra.addOperation(name='CrossSpectraPlot', optype='external') + op6.addParameter(name='id', value='10', format='int') + op6.addParameter(name='zmin', value='10.0', format='float') + op6.addParameter(name='zmax', value='35.0', format='float') + + + op7 = proc_spectra.addOperation(name='RTIPlot', optype='external') + op7.addParameter(name='id', value='20', format='int') + op7.addParameter(name='wintitle', value='RTI', format='str') + op7.addParameter(name='xmin', value='0', format='float') + op7.addParameter(name='xmax', value='24', format='float') + op7.addParameter(name='zmin', value='12', format='int') + op7.addParameter(name='zmax', value='32', format='int') + op7.addParameter(name='showprofile', value='1', format='int') + op7.addParameter(name='timerange', value=str(24*60*60), format='int') + + op8 = proc_spectra.addOperation(name='CoherenceMap', optype='external') + op8.addParameter(name='id', value='30', format='int') + op8.addParameter(name='xmin', value='0.0', format='float') + op8.addParameter(name='xmax', value='24.0', format='float') + + + controller.start() + +if __name__ == '__main__': + import time + start_time = time.time() + main() + print("--- %s seconds ---" % (time.time() - start_time)) + +''' + + + + + + + + multiprocess = '''from schainpy.controller import Project, MPProject from time import sleep desc = "{desc}" @@ -61,10 +240,10 @@ sleep(2) ################ # DATA EMITTER # ################ -project = Project() -project.setup(id='200', name="{name}", description=desc) +controller = Project() +controller.setup(id='200', name="{name}", description=desc) -spectra_reader = project.addReadUnit(datatype='SpectraReader', +spectra_reader = controller.addReadUnit(datatype='SpectraReader', path="{path}", startDate={startDate}, endDate={endDate}, @@ -75,16 +254,16 @@ spectra_reader = project.addReadUnit(datatype='SpectraReader', walk=1, ) -spectra_proc = project.addProcUnit(datatype='Spectra', inputId=spectra_reader.getId()) +spectra_proc = controller.addProcUnit(datatype='Spectra', inputId=spectra_reader.getId()) -parameters_proc = project.addProcUnit(datatype='ParametersProc', inputId=spectra_proc.getId()) +parameters_proc = controller.addProcUnit(datatype='ParametersProc', inputId=spectra_proc.getId()) moments = parameters_proc.addOperation(name='SpectralMoments', optype='other') publish = parameters_proc.addOperation(name='PublishData', optype='other') publish.addParameter(name='zeromq', value=1, format='int') publish.addParameter(name='verbose', value=0, format='bool') -MPProject(project, 16) +MPProject(controller, 16) ''' diff --git a/schainpy/controller.py b/schainpy/controller.py index ce97a95..3bbf5f4 100644 --- a/schainpy/controller.py +++ b/schainpy/controller.py @@ -1,569 +1,179 @@ ''' -Created on September , 2012 -@author: +Main routines to create a Signal Chain project ''' +import re import sys import ast import datetime import traceback -import math import time -from multiprocessing import Process, cpu_count +from multiprocessing import Process, Queue +from threading import Thread +from xml.etree.ElementTree import ElementTree, Element, SubElement -from xml.etree.ElementTree import ElementTree, Element, SubElement, tostring -from xml.dom import minidom - -import schainpy from schainpy.admin import Alarm, SchainWarning from schainpy.model import * from schainpy.utils import log -DTYPES = { - 'Voltage': '.r', - 'Spectra': '.pdata' -} - - -def MPProject(project, n=cpu_count()): - ''' - Project wrapper to run schain in n processes - ''' - - rconf = project.getReadUnitObj() - op = rconf.getOperationObj('run') - dt1 = op.getParameterValue('startDate') - dt2 = op.getParameterValue('endDate') - tm1 = op.getParameterValue('startTime') - tm2 = op.getParameterValue('endTime') - days = (dt2 - dt1).days - - for day in range(days + 1): - skip = 0 - cursor = 0 - processes = [] - dt = dt1 + datetime.timedelta(day) - dt_str = dt.strftime('%Y/%m/%d') - reader = JRODataReader() - paths, files = reader.searchFilesOffLine(path=rconf.path, - startDate=dt, - endDate=dt, - startTime=tm1, - endTime=tm2, - ext=DTYPES[rconf.datatype]) - nFiles = len(files) - if nFiles == 0: - continue - skip = int(math.ceil(nFiles / n)) - while nFiles > cursor * skip: - rconf.update(startDate=dt_str, endDate=dt_str, cursor=cursor, - skip=skip) - p = project.clone() - p.start() - processes.append(p) - cursor += 1 - - def beforeExit(exctype, value, trace): - for process in processes: - process.terminate() - process.join() - print traceback.print_tb(trace) - - sys.excepthook = beforeExit - - for process in processes: - process.join() - process.terminate() - - time.sleep(3) - - -class ParameterConf(): - - id = None - name = None - value = None - format = None - - __formated_value = None - - ELEMENTNAME = 'Parameter' - - def __init__(self): - - self.format = 'str' - - def getElementName(self): - - return self.ELEMENTNAME - - def getValue(self): - - value = self.value - format = self.format - - if self.__formated_value != None: - - return self.__formated_value - - if format == 'obj': - return value - - if format == 'str': - self.__formated_value = str(value) - return self.__formated_value - - if value == '': - raise ValueError, '%s: This parameter value is empty' % self.name - - if format == 'list': - strList = value.split(',') - - self.__formated_value = strList - - return self.__formated_value - - if format == 'intlist': - ''' - Example: - value = (0,1,2) - ''' - - new_value = ast.literal_eval(value) - - if type(new_value) not in (tuple, list): - new_value = [int(new_value)] - - self.__formated_value = new_value - - return self.__formated_value - - if format == 'floatlist': - ''' - Example: - value = (0.5, 1.4, 2.7) - ''' - - new_value = ast.literal_eval(value) - - if type(new_value) not in (tuple, list): - new_value = [float(new_value)] - - self.__formated_value = new_value - - return self.__formated_value - - if format == 'date': - strList = value.split('/') - intList = [int(x) for x in strList] - date = datetime.date(intList[0], intList[1], intList[2]) - - self.__formated_value = date - - return self.__formated_value - - if format == 'time': - strList = value.split(':') - intList = [int(x) for x in strList] - time = datetime.time(intList[0], intList[1], intList[2]) - - self.__formated_value = time - - return self.__formated_value - - if format == 'pairslist': - ''' - Example: - value = (0,1),(1,2) - ''' - - new_value = ast.literal_eval(value) - - if type(new_value) not in (tuple, list): - raise ValueError, '%s has to be a tuple or list of pairs' % value - - if type(new_value[0]) not in (tuple, list): - if len(new_value) != 2: - raise ValueError, '%s has to be a tuple or list of pairs' % value - new_value = [new_value] - - for thisPair in new_value: - if len(thisPair) != 2: - raise ValueError, '%s has to be a tuple or list of pairs' % value - - self.__formated_value = new_value - - return self.__formated_value - - if format == 'multilist': - ''' - Example: - value = (0,1,2),(3,4,5) - ''' - multiList = ast.literal_eval(value) - - if type(multiList[0]) == int: - multiList = ast.literal_eval('(' + value + ')') - - self.__formated_value = multiList - - return self.__formated_value - - if format == 'bool': - value = int(value) - - if format == 'int': - value = float(value) - - format_func = eval(format) - - self.__formated_value = format_func(value) - - return self.__formated_value - - def updateId(self, new_id): - - self.id = str(new_id) - - def setup(self, id, name, value, format='str'): - self.id = str(id) - self.name = name - if format == 'obj': - self.value = value - else: - self.value = str(value) - self.format = str.lower(format) - - self.getValue() - - return 1 - - def update(self, name, value, format='str'): - - self.name = name - self.value = str(value) - self.format = format - - def makeXml(self, opElement): - if self.name not in ('queue',): - parmElement = SubElement(opElement, self.ELEMENTNAME) - parmElement.set('id', str(self.id)) - parmElement.set('name', self.name) - parmElement.set('value', self.value) - parmElement.set('format', self.format) - def readXml(self, parmElement): - - self.id = parmElement.get('id') - self.name = parmElement.get('name') - self.value = parmElement.get('value') - self.format = str.lower(parmElement.get('format')) - - # Compatible with old signal chain version - if self.format == 'int' and self.name == 'idfigure': - self.name = 'id' - - def printattr(self): - - print 'Parameter[%s]: name = %s, value = %s, format = %s' % (self.id, self.name, self.value, self.format) - - -class OperationConf(): - - id = None - name = None - priority = None - type = None - - parmConfObjList = [] - - ELEMENTNAME = 'Operation' +class ConfBase(): def __init__(self): self.id = '0' self.name = None self.priority = None - self.type = 'self' + self.parameters = {} + self.object = None + self.operations = [] + + def getId(self): - def __getNewId(self): + return self.id + + def getNewId(self): - return int(self.id) * 10 + len(self.parmConfObjList) + 1 + return int(self.id) * 10 + len(self.operations) + 1 def updateId(self, new_id): self.id = str(new_id) n = 1 - for parmObj in self.parmConfObjList: - - idParm = str(int(new_id) * 10 + n) - parmObj.updateId(idParm) - + for conf in self.operations: + conf_id = str(int(new_id) * 10 + n) + conf.updateId(conf_id) n += 1 - def getElementName(self): - - return self.ELEMENTNAME - - def getParameterObjList(self): - - return self.parmConfObjList - - def getParameterObj(self, parameterName): - - for parmConfObj in self.parmConfObjList: - - if parmConfObj.name != parameterName: - continue - - return parmConfObj - - return None - - def getParameterObjfromValue(self, parameterValue): - - for parmConfObj in self.parmConfObjList: + def getKwargs(self): - if parmConfObj.getValue() != parameterValue: - continue + params = {} - return parmConfObj.getValue() + for key, value in self.parameters.items(): + if value not in (None, '', ' '): + params[key] = value + + return params - return None + def update(self, **kwargs): - def getParameterValue(self, parameterName): + for key, value in kwargs.items(): + self.addParameter(name=key, value=value) - parameterObj = self.getParameterObj(parameterName) + def addParameter(self, name, value, format=None): + ''' + ''' - # if not parameterObj: - # return None + if isinstance(value, str) and re.search(r'(\d+/\d+/\d+)', value): + self.parameters[name] = datetime.date(*[int(x) for x in value.split('/')]) + elif isinstance(value, str) and re.search(r'(\d+:\d+:\d+)', value): + self.parameters[name] = datetime.time(*[int(x) for x in value.split(':')]) + else: + try: + self.parameters[name] = ast.literal_eval(value) + except: + if isinstance(value, str) and ',' in value: + self.parameters[name] = value.split(',') + else: + self.parameters[name] = value + + def getParameters(self): + + params = {} + for key, value in self.parameters.items(): + s = type(value).__name__ + if s == 'date': + params[key] = value.strftime('%Y/%m/%d') + elif s == 'time': + params[key] = value.strftime('%H:%M:%S') + else: + params[key] = str(value) - value = parameterObj.getValue() + return params + + def makeXml(self, element): - return value + xml = SubElement(element, self.ELEMENTNAME) + for label in self.xml_labels: + xml.set(label, str(getattr(self, label))) + + for key, value in self.getParameters().items(): + xml_param = SubElement(xml, 'Parameter') + xml_param.set('name', key) + xml_param.set('value', value) + + for conf in self.operations: + conf.makeXml(xml) + + def __str__(self): - def getKwargs(self): + if self.ELEMENTNAME == 'Operation': + s = ' {}[id={}]\n'.format(self.name, self.id) + else: + s = '{}[id={}, inputId={}]\n'.format(self.name, self.id, self.inputId) - kwargs = {} + for key, value in self.parameters.items(): + if self.ELEMENTNAME == 'Operation': + s += ' {}: {}\n'.format(key, value) + else: + s += ' {}: {}\n'.format(key, value) + + for conf in self.operations: + s += str(conf) - for parmConfObj in self.parmConfObjList: - if self.name == 'run' and parmConfObj.name == 'datatype': - continue + return s - kwargs[parmConfObj.name] = parmConfObj.getValue() +class OperationConf(ConfBase): - return kwargs + ELEMENTNAME = 'Operation' + xml_labels = ['id', 'name'] - def setup(self, id, name, priority, type): + def setup(self, id, name, priority, project_id, err_queue): self.id = str(id) + self.project_id = project_id self.name = name - self.type = type - self.priority = priority - - self.parmConfObjList = [] - - def removeParameters(self): - - for obj in self.parmConfObjList: - del obj - - self.parmConfObjList = [] - - def addParameter(self, name, value, format='str'): - - if value is None: - return None - id = self.__getNewId() - - parmConfObj = ParameterConf() - if not parmConfObj.setup(id, name, value, format): - return None - - self.parmConfObjList.append(parmConfObj) - - return parmConfObj - - def changeParameter(self, name, value, format='str'): - - parmConfObj = self.getParameterObj(name) - parmConfObj.update(name, value, format) - - return parmConfObj - - def makeXml(self, procUnitElement): - - opElement = SubElement(procUnitElement, self.ELEMENTNAME) - opElement.set('id', str(self.id)) - opElement.set('name', self.name) - opElement.set('type', self.type) - opElement.set('priority', str(self.priority)) - - for parmConfObj in self.parmConfObjList: - parmConfObj.makeXml(opElement) - - def readXml(self, opElement): - - self.id = opElement.get('id') - self.name = opElement.get('name') - self.type = opElement.get('type') - self.priority = opElement.get('priority') + self.type = 'other' + self.err_queue = err_queue - # Compatible with old signal chain version - # Use of 'run' method instead 'init' - if self.type == 'self' and self.name == 'init': - self.name = 'run' + def readXml(self, element, project_id, err_queue): - self.parmConfObjList = [] + self.id = element.get('id') + self.name = element.get('name') + self.type = 'other' + self.project_id = str(project_id) + self.err_queue = err_queue - parmElementList = opElement.iter(ParameterConf().getElementName()) + for elm in element.iter('Parameter'): + self.addParameter(elm.get('name'), elm.get('value')) - for parmElement in parmElementList: - parmConfObj = ParameterConf() - parmConfObj.readXml(parmElement) + def createObject(self): - # Compatible with old signal chain version - # If an 'plot' OPERATION is found, changes name operation by the value of its type PARAMETER - if self.type != 'self' and self.name == 'Plot': - if parmConfObj.format == 'str' and parmConfObj.name == 'type': - self.name = parmConfObj.value - continue - - self.parmConfObjList.append(parmConfObj) - - def printattr(self): - - print '%s[%s]: name = %s, type = %s, priority = %s' % (self.ELEMENTNAME, - self.id, - self.name, - self.type, - self.priority) - - for parmConfObj in self.parmConfObjList: - parmConfObj.printattr() - - def createObject(self, plotter_queue=None): - - if self.type == 'self': - raise ValueError, 'This operation type cannot be created' - - if self.type == 'plotter': - if not plotter_queue: - raise ValueError, 'plotter_queue is not defined. Use:\nmyProject = Project()\nmyProject.setPlotterQueue(plotter_queue)' - - opObj = Plotter(self.name, plotter_queue) - - if self.type == 'external' or self.type == 'other': + className = eval(self.name) - className = eval(self.name) + if 'Plot' in self.name or 'Writer' in self.name or 'Send' in self.name or 'print' in self.name: kwargs = self.getKwargs() + opObj = className(self.id, self.id, self.project_id, self.err_queue, **kwargs) + opObj.start() + self.type = 'external' + else: + opObj = className() - opObj = className(**kwargs) - + self.object = opObj return opObj - -class ProcUnitConf(): - - id = None - name = None - datatype = None - inputId = None - parentId = None - - opConfObjList = [] - - procUnitObj = None - opObjList = [] +class ProcUnitConf(ConfBase): ELEMENTNAME = 'ProcUnit' + xml_labels = ['id', 'inputId', 'name'] - def __init__(self): - - self.id = None - self.datatype = None - self.name = None - self.inputId = None - - self.opConfObjList = [] - - self.procUnitObj = None - self.opObjDict = {} - - def __getPriority(self): - - return len(self.opConfObjList) + 1 - - def __getNewId(self): - - return int(self.id) * 10 + len(self.opConfObjList) + 1 - - def getElementName(self): - - return self.ELEMENTNAME - - def getId(self): - - return self.id - - def updateId(self, new_id, parentId=parentId): - - new_id = int(parentId) * 10 + (int(self.id) % 10) - new_inputId = int(parentId) * 10 + (int(self.inputId) % 10) - - # If this proc unit has not inputs - if self.inputId == '0': - new_inputId = 0 - - n = 1 - for opConfObj in self.opConfObjList: - - idOp = str(int(new_id) * 10 + n) - opConfObj.updateId(idOp) - - n += 1 - - self.parentId = str(parentId) - self.id = str(new_id) - self.inputId = str(new_inputId) - - def getInputId(self): - - return self.inputId - - def getOperationObjList(self): - - return self.opConfObjList - - def getOperationObj(self, name=None): - - for opConfObj in self.opConfObjList: - - if opConfObj.name != name: - continue - - return opConfObj - - return None - - def getOpObjfromParamValue(self, value=None): - - for opConfObj in self.opConfObjList: - if opConfObj.getParameterObjfromValue(parameterValue=value) != value: - continue - return opConfObj - return None - - def getProcUnitObj(self): - - return self.procUnitObj - - def setup(self, id, name, datatype, inputId, parentId=None): - - # Compatible with old signal chain version + def setup(self, project_id, id, name, datatype, inputId, err_queue): + ''' + ''' + if datatype == None and name == None: - raise ValueError, 'datatype or name should be defined' + raise ValueError('datatype or name should be defined') if name == None: if 'Proc' in datatype: @@ -575,167 +185,87 @@ class ProcUnitConf(): datatype = name.replace('Proc', '') self.id = str(id) + self.project_id = project_id self.name = name self.datatype = datatype self.inputId = inputId - self.parentId = parentId - - self.opConfObjList = [] - - self.addOperation(name='run', optype='self') + self.err_queue = err_queue + self.operations = [] + self.parameters = {} - def removeOperations(self): + def removeOperation(self, id): - for obj in self.opConfObjList: - del obj + i = [1 if x.id==id else 0 for x in self.operations] + self.operations.pop(i.index(1)) + + def getOperation(self, id): - self.opConfObjList = [] - self.addOperation(name='run') + for conf in self.operations: + if conf.id == id: + return conf - def addParameter(self, **kwargs): + def addOperation(self, name, optype='self'): ''' - Add parameters to 'run' operation ''' - opObj = self.opConfObjList[0] - - opObj.addParameter(**kwargs) - - return opObj - - def addOperation(self, name, optype='self'): - - id = self.__getNewId() - priority = self.__getPriority() - - opConfObj = OperationConf() - opConfObj.setup(id, name=name, priority=priority, type=optype) - - self.opConfObjList.append(opConfObj) - - return opConfObj - - def makeXml(self, projectElement): - - procUnitElement = SubElement(projectElement, self.ELEMENTNAME) - procUnitElement.set('id', str(self.id)) - procUnitElement.set('name', self.name) - procUnitElement.set('datatype', self.datatype) - procUnitElement.set('inputId', str(self.inputId)) - - for opConfObj in self.opConfObjList: - opConfObj.makeXml(procUnitElement) - - def readXml(self, upElement): - - self.id = upElement.get('id') - self.name = upElement.get('name') - self.datatype = upElement.get('datatype') - self.inputId = upElement.get('inputId') - - if self.ELEMENTNAME == 'ReadUnit': - self.datatype = self.datatype.replace('Reader', '') - - if self.ELEMENTNAME == 'ProcUnit': - self.datatype = self.datatype.replace('Proc', '') - - if self.inputId == 'None': - self.inputId = '0' - self.opConfObjList = [] + id = self.getNewId() + conf = OperationConf() + conf.setup(id, name=name, priority='0', project_id=self.project_id, err_queue=self.err_queue) + self.operations.append(conf) - opElementList = upElement.iter(OperationConf().getElementName()) + return conf - for opElement in opElementList: - opConfObj = OperationConf() - opConfObj.readXml(opElement) - self.opConfObjList.append(opConfObj) + def readXml(self, element, project_id, err_queue): - def printattr(self): - - print '%s[%s]: name = %s, datatype = %s, inputId = %s' % (self.ELEMENTNAME, - self.id, - self.name, - self.datatype, - self.inputId) - - for opConfObj in self.opConfObjList: - opConfObj.printattr() - - def getKwargs(self): - - opObj = self.opConfObjList[0] - kwargs = opObj.getKwargs() - - return kwargs + self.id = element.get('id') + self.name = element.get('name') + self.inputId = None if element.get('inputId') == 'None' else element.get('inputId') + self.datatype = element.get('datatype', self.name.replace(self.ELEMENTNAME.replace('Unit', ''), '')) + self.project_id = str(project_id) + self.err_queue = err_queue + self.operations = [] + self.parameters = {} + + for elm in element: + if elm.tag == 'Parameter': + self.addParameter(elm.get('name'), elm.get('value')) + elif elm.tag == 'Operation': + conf = OperationConf() + conf.readXml(elm, project_id, err_queue) + self.operations.append(conf) - def createObjects(self, plotter_queue=None): + def createObjects(self): + ''' + Instancia de unidades de procesamiento. + ''' className = eval(self.name) kwargs = self.getKwargs() - procUnitObj = className(**kwargs) - - for opConfObj in self.opConfObjList: - - if opConfObj.type == 'self' and self.name == 'run': - continue - elif opConfObj.type == 'self': - procUnitObj.addOperationKwargs( - opConfObj.id, **opConfObj.getKwargs()) - continue - - opObj = opConfObj.createObject(plotter_queue) - - self.opObjDict[opConfObj.id] = opObj - - procUnitObj.addOperation(opObj, opConfObj.id) - - self.procUnitObj = procUnitObj - - return procUnitObj + procUnitObj = className() + procUnitObj.name = self.name + log.success('creating process...', self.name) + + for conf in self.operations: + + opObj = conf.createObject() + + log.success('adding operation: {}, type:{}'.format( + conf.name, + conf.type), self.name) + + procUnitObj.addOperation(conf, opObj) + + self.object = procUnitObj def run(self): - - is_ok = False - - for opConfObj in self.opConfObjList: - - kwargs = {} - for parmConfObj in opConfObj.getParameterObjList(): - if opConfObj.name == 'run' and parmConfObj.name == 'datatype': - continue - - kwargs[parmConfObj.name] = parmConfObj.getValue() - - sts = self.procUnitObj.call(opType=opConfObj.type, - opName=opConfObj.name, - opId=opConfObj.id) - - is_ok = is_ok or sts - - return is_ok - - def close(self): - - for opConfObj in self.opConfObjList: - if opConfObj.type == 'self': - continue - - opObj = self.procUnitObj.getOperationObj(opConfObj.id) - opObj.close() - - self.procUnitObj.close() - - return + ''' + ''' + + return self.object.call(**self.getKwargs()) class ReadUnitConf(ProcUnitConf): - path = None - startDate = None - endDate = None - startTime = None - endTime = None - ELEMENTNAME = 'ReadUnit' def __init__(self): @@ -744,22 +274,14 @@ class ReadUnitConf(ProcUnitConf): self.datatype = None self.name = None self.inputId = None - - self.parentId = None - - self.opConfObjList = [] - self.opObjList = [] - - def getElementName(self): - - return self.ELEMENTNAME - - def setup(self, id, name, datatype, path='', startDate='', endDate='', - startTime='', endTime='', parentId=None, server=None, **kwargs): - - # Compatible with old signal chain version + self.operations = [] + self.parameters = {} + + def setup(self, project_id, id, name, datatype, err_queue, path='', startDate='', endDate='', + startTime='', endTime='', server=None, **kwargs): + if datatype == None and name == None: - raise ValueError, 'datatype or name should be defined' + raise ValueError('datatype or name should be defined') if name == None: if 'Reader' in datatype: name = datatype @@ -774,148 +296,41 @@ class ReadUnitConf(ProcUnitConf): name = '{}Reader'.format(name) self.id = id + self.project_id = project_id self.name = name self.datatype = datatype - if path != '': - self.path = os.path.abspath(path) - self.startDate = startDate - self.endDate = endDate - self.startTime = startTime - self.endTime = endTime - self.inputId = '0' - self.parentId = parentId - self.server = server - self.addRunOperation(**kwargs) - - def update(self, **kwargs): - - if 'datatype' in kwargs: - datatype = kwargs.pop('datatype') - if 'Reader' in datatype: - self.name = datatype - else: - self.name = '%sReader' % (datatype) - self.datatype = self.name.replace('Reader', '') - - attrs = ('path', 'startDate', 'endDate', - 'startTime', 'endTime', 'parentId') - - for attr in attrs: - if attr in kwargs: - setattr(self, attr, kwargs.pop(attr)) - - self.inputId = '0' - self.updateRunOperation(**kwargs) - - def removeOperations(self): - - for obj in self.opConfObjList: - del obj - - self.opConfObjList = [] - - def addRunOperation(self, **kwargs): - - opObj = self.addOperation(name='run', optype='self') - - if self.server is None: - opObj.addParameter( - name='datatype', value=self.datatype, format='str') - opObj.addParameter(name='path', value=self.path, format='str') - opObj.addParameter( - name='startDate', value=self.startDate, format='date') - opObj.addParameter( - name='endDate', value=self.endDate, format='date') - opObj.addParameter( - name='startTime', value=self.startTime, format='time') - opObj.addParameter( - name='endTime', value=self.endTime, format='time') - - for key, value in kwargs.items(): - opObj.addParameter(name=key, value=value, - format=type(value).__name__) - else: - opObj.addParameter(name='server', value=self.server, format='str') - - return opObj - - def updateRunOperation(self, **kwargs): - - opObj = self.getOperationObj(name='run') - opObj.removeParameters() - - opObj.addParameter(name='datatype', value=self.datatype, format='str') - opObj.addParameter(name='path', value=self.path, format='str') - opObj.addParameter( - name='startDate', value=self.startDate, format='date') - opObj.addParameter(name='endDate', value=self.endDate, format='date') - opObj.addParameter( - name='startTime', value=self.startTime, format='time') - opObj.addParameter(name='endTime', value=self.endTime, format='time') + self.err_queue = err_queue + + self.addParameter(name='path', value=path) + self.addParameter(name='startDate', value=startDate) + self.addParameter(name='endDate', value=endDate) + self.addParameter(name='startTime', value=startTime) + self.addParameter(name='endTime', value=endTime) for key, value in kwargs.items(): - opObj.addParameter(name=key, value=value, - format=type(value).__name__) - - return opObj - - def readXml(self, upElement): - - self.id = upElement.get('id') - self.name = upElement.get('name') - self.datatype = upElement.get('datatype') - self.inputId = upElement.get('inputId') - - if self.ELEMENTNAME == 'ReadUnit': - self.datatype = self.datatype.replace('Reader', '') - - if self.inputId == 'None': - self.inputId = '0' - - self.opConfObjList = [] - - opElementList = upElement.iter(OperationConf().getElementName()) - - for opElement in opElementList: - opConfObj = OperationConf() - opConfObj.readXml(opElement) - self.opConfObjList.append(opConfObj) - - if opConfObj.name == 'run': - self.path = opConfObj.getParameterValue('path') - self.startDate = opConfObj.getParameterValue('startDate') - self.endDate = opConfObj.getParameterValue('endDate') - self.startTime = opConfObj.getParameterValue('startTime') - self.endTime = opConfObj.getParameterValue('endTime') + self.addParameter(name=key, value=value) class Project(Process): - id = None - # name = None - description = None - filename = None - - procUnitConfObjDict = None - ELEMENTNAME = 'Project' - plotterQueue = None - - def __init__(self, plotter_queue=None): + def __init__(self): Process.__init__(self) - self.id = None + self.id = None + self.filename = None self.description = None self.email = None - self.alarm = None - self.plotterQueue = plotter_queue - self.procUnitConfObjDict = {} + self.alarm = [] + self.configurations = {} + # self.err_queue = Queue() + self.err_queue = None + self.started = False - def __getNewId(self): - - idList = self.procUnitConfObjDict.keys() + def getNewId(self): + idList = list(self.configurations.keys()) id = int(self.id) * 10 while True: @@ -928,45 +343,34 @@ class Project(Process): return str(id) - def getElementName(self): - - return self.ELEMENTNAME - - def getId(self): - - return self.id - def updateId(self, new_id): self.id = str(new_id) - keyList = self.procUnitConfObjDict.keys() + keyList = list(self.configurations.keys()) keyList.sort() n = 1 - newProcUnitConfObjDict = {} + new_confs = {} for procKey in keyList: - procUnitConfObj = self.procUnitConfObjDict[procKey] + conf = self.configurations[procKey] idProcUnit = str(int(self.id) * 10 + n) - procUnitConfObj.updateId(idProcUnit, parentId=self.id) - newProcUnitConfObjDict[idProcUnit] = procUnitConfObj + conf.updateId(idProcUnit) + new_confs[idProcUnit] = conf n += 1 - self.procUnitConfObjDict = newProcUnitConfObjDict + self.configurations = new_confs - def setup(self, id, name='', description='', email=None, alarm=[]): + def setup(self, id=1, name='', description='', email=None, alarm=[]): - print - print '*' * 60 - print ' Starting SIGNAL CHAIN PROCESSING v%s ' % schainpy.__version__ - print '*' * 60 - print self.id = str(id) - self.description = description + self.description = description self.email = email self.alarm = alarm + if name: + self.name = '{} ({})'.format(Process.__name__, name) def update(self, **kwargs): @@ -976,88 +380,85 @@ class Project(Process): def clone(self): p = Project() - p.procUnitConfObjDict = self.procUnitConfObjDict + p.id = self.id + p.name = self.name + p.description = self.description + p.configurations = self.configurations.copy() + return p def addReadUnit(self, id=None, datatype=None, name=None, **kwargs): + ''' + ''' + if id is None: - idReadUnit = self.__getNewId() + idReadUnit = self.getNewId() else: idReadUnit = str(id) - readUnitConfObj = ReadUnitConf() - readUnitConfObj.setup(idReadUnit, name, datatype, - parentId=self.id, **kwargs) - - self.procUnitConfObjDict[readUnitConfObj.getId()] = readUnitConfObj - - return readUnitConfObj - - def addProcUnit(self, inputId='0', datatype=None, name=None): + conf = ReadUnitConf() + conf.setup(self.id, idReadUnit, name, datatype, self.err_queue, **kwargs) + self.configurations[conf.id] = conf + + return conf - idProcUnit = self.__getNewId() + def addProcUnit(self, id=None, inputId='0', datatype=None, name=None): - procUnitConfObj = ProcUnitConf() - procUnitConfObj.setup(idProcUnit, name, datatype, - inputId, parentId=self.id) + ''' + ''' - self.procUnitConfObjDict[procUnitConfObj.getId()] = procUnitConfObj + if id is None: + idProcUnit = self.getNewId() + else: + idProcUnit = id + + conf = ProcUnitConf() + conf.setup(self.id, idProcUnit, name, datatype, inputId, self.err_queue) + self.configurations[conf.id] = conf - return procUnitConfObj + return conf def removeProcUnit(self, id): - if id in self.procUnitConfObjDict.keys(): - self.procUnitConfObjDict.pop(id) - - def getReadUnitId(self): - - readUnitConfObj = self.getReadUnitObj() + if id in self.configurations: + self.configurations.pop(id) - return readUnitConfObj.id + def getReadUnit(self): - def getReadUnitObj(self): - - for obj in self.procUnitConfObjDict.values(): - if obj.getElementName() == 'ReadUnit': + for obj in list(self.configurations.values()): + if obj.ELEMENTNAME == 'ReadUnit': return obj return None - def getProcUnitObj(self, id=None, name=None): - - if id != None: - return self.procUnitConfObjDict[id] - - if name != None: - return self.getProcUnitObjByName(name) + def getProcUnit(self, id): - return None - - def getProcUnitObjByName(self, name): + return self.configurations[id] - for obj in self.procUnitConfObjDict.values(): - if obj.name == name: - return obj + def getUnits(self): - return None + keys = list(self.configurations) + keys.sort() - def procUnitItems(self): + for key in keys: + yield self.configurations[key] - return self.procUnitConfObjDict.items() + def updateUnit(self, id, **kwargs): + conf = self.configurations[id].update(**kwargs) + def makeXml(self): - projectElement = Element('Project') - projectElement.set('id', str(self.id)) - projectElement.set('name', self.name) - projectElement.set('description', self.description) + xml = Element('Project') + xml.set('id', str(self.id)) + xml.set('name', self.name) + xml.set('description', self.description) - for procUnitConfObj in self.procUnitConfObjDict.values(): - procUnitConfObj.makeXml(projectElement) + for conf in self.configurations.values(): + conf.makeXml(xml) - self.projectElement = projectElement + self.xml = xml def writeXml(self, filename=None): @@ -1068,275 +469,180 @@ class Project(Process): filename = 'schain.xml' if not filename: - print 'filename has not been defined. Use setFilename(filename) for do it.' + print('filename has not been defined. Use setFilename(filename) for do it.') return 0 abs_file = os.path.abspath(filename) if not os.access(os.path.dirname(abs_file), os.W_OK): - print 'No write permission on %s' % os.path.dirname(abs_file) + print('No write permission on %s' % os.path.dirname(abs_file)) return 0 if os.path.isfile(abs_file) and not(os.access(abs_file, os.W_OK)): - print 'File %s already exists and it could not be overwriten' % abs_file + print('File %s already exists and it could not be overwriten' % abs_file) return 0 self.makeXml() - ElementTree(self.projectElement).write(abs_file, method='xml') + ElementTree(self.xml).write(abs_file, method='xml') self.filename = abs_file return 1 - def readXml(self, filename=None): - - if not filename: - print 'filename is not defined' - return 0 + def readXml(self, filename): abs_file = os.path.abspath(filename) - if not os.path.isfile(abs_file): - print '%s file does not exist' % abs_file - return 0 - - self.projectElement = None - self.procUnitConfObjDict = {} + self.configurations = {} try: - self.projectElement = ElementTree().parse(abs_file) + self.xml = ElementTree().parse(abs_file) except: - print 'Error reading %s, verify file format' % filename + log.error('Error reading %s, verify file format' % filename) return 0 - self.project = self.projectElement.tag - - self.id = self.projectElement.get('id') - self.name = self.projectElement.get('name') - self.description = self.projectElement.get('description') - - readUnitElementList = self.projectElement.iter( - ReadUnitConf().getElementName()) - - for readUnitElement in readUnitElementList: - readUnitConfObj = ReadUnitConf() - readUnitConfObj.readXml(readUnitElement) - - if readUnitConfObj.parentId == None: - readUnitConfObj.parentId = self.id - - self.procUnitConfObjDict[readUnitConfObj.getId()] = readUnitConfObj - - procUnitElementList = self.projectElement.iter( - ProcUnitConf().getElementName()) - - for procUnitElement in procUnitElementList: - procUnitConfObj = ProcUnitConf() - procUnitConfObj.readXml(procUnitElement) - - if procUnitConfObj.parentId == None: - procUnitConfObj.parentId = self.id - - self.procUnitConfObjDict[procUnitConfObj.getId()] = procUnitConfObj + self.id = self.xml.get('id') + self.name = self.xml.get('name') + self.description = self.xml.get('description') + + for element in self.xml: + if element.tag == 'ReadUnit': + conf = ReadUnitConf() + conf.readXml(element, self.id, self.err_queue) + self.configurations[conf.id] = conf + elif element.tag == 'ProcUnit': + conf = ProcUnitConf() + input_proc = self.configurations[element.get('inputId')] + conf.readXml(element, self.id, self.err_queue) + self.configurations[conf.id] = conf self.filename = abs_file - + return 1 - def printattr(self): - - print 'Project[%s]: name = %s, description = %s' % (self.id, - self.name, - self.description) - - for procUnitConfObj in self.procUnitConfObjDict.values(): - procUnitConfObj.printattr() - - def createObjects(self): - - for procUnitConfObj in self.procUnitConfObjDict.values(): - procUnitConfObj.createObjects(self.plotterQueue) + def __str__(self): - def __connect(self, objIN, thisObj): - - thisObj.setInput(objIN.getOutputObj()) - - def connectObjects(self): - - for thisPUConfObj in self.procUnitConfObjDict.values(): + text = '\nProject[id=%s, name=%s, description=%s]\n\n' % ( + self.id, + self.name, + self.description, + ) - inputId = thisPUConfObj.getInputId() + for conf in self.configurations.values(): + text += '{}'.format(conf) - if int(inputId) == 0: - continue + return text - # Get input object - puConfINObj = self.procUnitConfObjDict[inputId] - puObjIN = puConfINObj.getProcUnitObj() + def createObjects(self): - # Get current object - thisPUObj = thisPUConfObj.getProcUnitObj() + keys = list(self.configurations.keys()) + keys.sort() + for key in keys: + conf = self.configurations[key] + conf.createObjects() + if conf.inputId is not None: + conf.object.setInput(self.configurations[conf.inputId].object) - self.__connect(puObjIN, thisPUObj) + def monitor(self): - def __handleError(self, procUnitConfObj, modes=None, stdout=True): + t = Thread(target=self._monitor, args=(self.err_queue, self.ctx)) + t.start() + + def _monitor(self, queue, ctx): import socket - - if modes is None: - modes = self.alarm - if not self.alarm: - modes = [] - - err = traceback.format_exception(sys.exc_info()[0], - sys.exc_info()[1], - sys.exc_info()[2]) + procs = 0 + err_msg = '' - log.error('{}'.format(err[-1]), procUnitConfObj.name) + while True: + msg = queue.get() + if '#_start_#' in msg: + procs += 1 + elif '#_end_#' in msg: + procs -=1 + else: + err_msg = msg + + if procs == 0 or 'Traceback' in err_msg: + break + time.sleep(0.1) + + if '|' in err_msg: + name, err = err_msg.split('|') + if 'SchainWarning' in err: + log.warning(err.split('SchainWarning:')[-1].split('\n')[0].strip(), name) + elif 'SchainError' in err: + log.error(err.split('SchainError:')[-1].split('\n')[0].strip(), name) + else: + log.error(err, name) + else: + name, err = self.name, err_msg + + time.sleep(1) + + ctx.term() message = ''.join(err) - if stdout: - sys.stderr.write(message) - - subject = 'SChain v%s: Error running %s\n' % ( - schainpy.__version__, procUnitConfObj.name) - - subtitle = '%s: %s\n' % ( - procUnitConfObj.getElementName(), procUnitConfObj.name) - subtitle += 'Hostname: %s\n' % socket.gethostbyname( - socket.gethostname()) - subtitle += 'Working directory: %s\n' % os.path.abspath('./') - subtitle += 'Configuration file: %s\n' % self.filename - subtitle += 'Time: %s\n' % str(datetime.datetime.now()) - - readUnitConfObj = self.getReadUnitObj() - if readUnitConfObj: - subtitle += '\nInput parameters:\n' - subtitle += '[Data path = %s]\n' % readUnitConfObj.path - subtitle += '[Data type = %s]\n' % readUnitConfObj.datatype - subtitle += '[Start date = %s]\n' % readUnitConfObj.startDate - subtitle += '[End date = %s]\n' % readUnitConfObj.endDate - subtitle += '[Start time = %s]\n' % readUnitConfObj.startTime - subtitle += '[End time = %s]\n' % readUnitConfObj.endTime - - a = Alarm( - modes=modes, - email=self.email, - message=message, - subject=subject, - subtitle=subtitle, - filename=self.filename - ) - - return a - - def isPaused(self): - return 0 - - def isStopped(self): - return 0 - - def runController(self): - ''' - returns 0 when this process has been stopped, 1 otherwise - ''' - - if self.isPaused(): - print 'Process suspended' - - while True: - time.sleep(0.1) - - if not self.isPaused(): - break - - if self.isStopped(): - break - - print 'Process reinitialized' - - if self.isStopped(): - print 'Process stopped' - return 0 - - return 1 + if err_msg: + subject = 'SChain v%s: Error running %s\n' % ( + schainpy.__version__, self.name) + + subtitle = 'Hostname: %s\n' % socket.gethostbyname( + socket.gethostname()) + subtitle += 'Working directory: %s\n' % os.path.abspath('./') + subtitle += 'Configuration file: %s\n' % self.filename + subtitle += 'Time: %s\n' % str(datetime.datetime.now()) + + readUnitConfObj = self.getReadUnit() + if readUnitConfObj: + subtitle += '\nInput parameters:\n' + subtitle += '[Data path = %s]\n' % readUnitConfObj.parameters['path'] + subtitle += '[Start date = %s]\n' % readUnitConfObj.parameters['startDate'] + subtitle += '[End date = %s]\n' % readUnitConfObj.parameters['endDate'] + subtitle += '[Start time = %s]\n' % readUnitConfObj.parameters['startTime'] + subtitle += '[End time = %s]\n' % readUnitConfObj.parameters['endTime'] + + a = Alarm( + modes=self.alarm, + email=self.email, + message=message, + subject=subject, + subtitle=subtitle, + filename=self.filename + ) + + a.start() def setFilename(self, filename): self.filename = filename - def setPlotterQueue(self, plotter_queue): - - raise NotImplementedError, 'Use schainpy.controller_api.ControllerThread instead Project class' - - def getPlotterQueue(self): - - raise NotImplementedError, 'Use schainpy.controller_api.ControllerThread instead Project class' - - def useExternalPlotter(self): - - raise NotImplementedError, 'Use schainpy.controller_api.ControllerThread instead Project class' + def runProcs(self): + err = False + n = len(self.configurations) + + while not err: + for conf in self.getUnits(): + ok = conf.run() + if ok is 'Error': + n -= 1 + continue + elif not ok: + break + if n == 0: + err = True + def run(self): - log.success('Starting {}'.format(self.name), tag='') - self.start_time = time.time() + log.success('\nStarting Project {} [id={}]'.format(self.name, self.id), tag='') + self.started = True + self.start_time = time.time() self.createObjects() - self.connectObjects() - - keyList = self.procUnitConfObjDict.keys() - keyList.sort() - - err = None - - while(True): - - is_ok = False - - for procKey in keyList: - - procUnitConfObj = self.procUnitConfObjDict[procKey] - - try: - sts = procUnitConfObj.run() - is_ok = is_ok or sts - except SchainWarning: - err = self.__handleError(procUnitConfObj, modes=[2, 3], stdout=False) - is_ok = False - break - except KeyboardInterrupt: - is_ok = False - break - except ValueError, e: - time.sleep(0.5) - err = self.__handleError(procUnitConfObj) - is_ok = False - break - except: - time.sleep(0.5) - err = self.__handleError(procUnitConfObj) - is_ok = False - break - - # If every process unit finished so end process - if not(is_ok): - break - - if not self.runController(): - break - - # Closing every process - for procKey in keyList: - procUnitConfObj = self.procUnitConfObjDict[procKey] - procUnitConfObj.close() - - if err is not None: - err.start() - # err.join() - - log.success('{} finished (time: {}s)'.format( + self.runProcs() + log.success('{} Done (Time: {:4.2f}s)'.format( self.name, - time.time()-self.start_time)) + time.time()-self.start_time), '') diff --git a/schainpy/controller_api.py b/schainpy/controller_api.py deleted file mode 100644 index 2d1775e..0000000 --- a/schainpy/controller_api.py +++ /dev/null @@ -1,179 +0,0 @@ -import threading -from Queue import Queue - -from schainpy.controller import Project -from schainpy.model.graphics.jroplotter import PlotManager - -class ControllerThread(threading.Thread, Project): - - def __init__(self, plotter_queue=None): - - threading.Thread.__init__(self) - Project.__init__(self, plotter_queue) - - self.setDaemon(True) - - self.lock = threading.Lock() - self.control = { 'stop':False, 'pause':False } - - def __del__(self): - - self.control['stop'] = True - - def stop(self): - - self.lock.acquire() - - self.control['stop'] = True - - self.lock.release() - - def pause(self): - - self.lock.acquire() - - self.control['pause'] = not(self.control['pause']) - paused = self.control['pause'] - - self.lock.release() - - return paused - - def isPaused(self): - - self.lock.acquire() - paused = self.control['pause'] - self.lock.release() - - return paused - - def isStopped(self): - - self.lock.acquire() - stopped = self.control['stop'] - self.lock.release() - - return stopped - - def run(self): - self.control['stop'] = False - self.control['pause'] = False - - self.writeXml() - - self.createObjects() - self.connectObjects() - Project.run(self) - - def isRunning(self): - - return self.is_alive() - - def isFinished(self): - - return not self.is_alive() - - def setPlotters(self): - - plotterList = PlotManager.plotterList - - for thisPUConfObj in self.procUnitConfObjDict.values(): - - inputId = thisPUConfObj.getInputId() - - if int(inputId) == 0: - continue - - for thisOpObj in thisPUConfObj.getOperationObjList(): - - if thisOpObj.type == "self": - continue - - if thisOpObj.name in plotterList: - thisOpObj.type = "other" - - def setPlotterQueue(self, plotter_queue): - - self.plotterQueue = plotter_queue - - def getPlotterQueue(self): - - return self.plotterQueue - - def useExternalPlotter(self): - - self.plotterQueue = Queue(10) - self.setPlotters() - - plotManagerObj = PlotManager(self.plotterQueue) - plotManagerObj.setController(self) - - return plotManagerObj - -# from PyQt4 import QtCore -# from PyQt4.QtCore import SIGNAL -# -# class ControllerQThread(QtCore.QThread, Project): -# -# def __init__(self, filename): -# -# QtCore.QThread.__init__(self) -# Project.__init__(self) -# -# self.filename = filename -# -# self.lock = threading.Lock() -# self.control = {'stop':False, 'pause':False} -# -# def __del__(self): -# -# self.control['stop'] = True -# self.wait() -# -# def stop(self): -# -# self.lock.acquire() -# -# self.control['stop'] = True -# -# self.lock.release() -# -# def pause(self): -# -# self.lock.acquire() -# -# self.control['pause'] = not(self.control['pause']) -# paused = self.control['pause'] -# -# self.lock.release() -# -# return paused -# -# def isPaused(self): -# -# self.lock.acquire() -# paused = self.control['pause'] -# self.lock.release() -# -# return paused -# -# def isStopped(self): -# -# self.lock.acquire() -# stopped = self.control['stop'] -# self.lock.release() -# -# return stopped -# -# def run(self): -# -# self.control['stop'] = False -# self.control['pause'] = False -# -# self.readXml(self.filename) -# self.createObjects() -# self.connectObjects() -# self.emit( SIGNAL( "jobStarted( PyQt_PyObject )" ), 1) -# Project.run(self) -# self.emit( SIGNAL( "jobFinished( PyQt_PyObject )" ), 1) -# diff --git a/schainpy/gui/__init__.py b/schainpy/gui/__init__.py index 6eac6ef..e69de29 100644 --- a/schainpy/gui/__init__.py +++ b/schainpy/gui/__init__.py @@ -1 +0,0 @@ -from viewcontroller import * \ No newline at end of file diff --git a/schainpy/gui/figures/.bash_logout b/schainpy/gui/figures/.bash_logout deleted file mode 100644 index de4f5f7..0000000 --- a/schainpy/gui/figures/.bash_logout +++ /dev/null @@ -1,7 +0,0 @@ -# ~/.bash_logout: executed by bash(1) when login shell exits. - -# when leaving the console clear the screen to increase privacy - -if [ "$SHLVL" = 1 ]; then - [ -x /usr/bin/clear_console ] && /usr/bin/clear_console -q -fi diff --git a/schainpy/gui/figures/__init__.py b/schainpy/gui/figures/__init__.py deleted file mode 100644 index e69de29..0000000 --- a/schainpy/gui/figures/__init__.py +++ /dev/null diff --git a/schainpy/gui/figures/branch.png b/schainpy/gui/figures/branch.png deleted file mode 100644 index f7a68f1ef9a7616c9068b0611842780e91f01c5c..0000000000000000000000000000000000000000 GIT binary patch literal 0 Hc$@8?tx|+>e7D!cXTANSS^LR>~GoGN9-mSG3ymbG_RZ&VGOR>NNhS`_5Gh13s;|ab`|@+ClXdFY3H= zYtMCBrMo?@DayMyxwu(-hhMf{tj5-_oQu8xow;fK;_`xk2WQtDc3k(dsqs3;mam=m zzU52KD$ecXn0F{T+&?-#;mR+YjOAO+Ura83EBrxI$HMuy;ImV=ez_=}eCS*hXZGJa z;NzV`f1(bs7@Vv0EBz>-GJk#KpT)O6F#g{8Sg|S2(&nXF0r$K%!P@5Bsp4Xp%M^<{ zPnE{`-pSonlzR4V*_^ zF{#3Qnm_+u=`-`ym#vDszQX2<aX40H^Q=#@$p^i*dfa~WtANJ?^<_AE83e~43w)qUHx3v IIVCg!0EM8%#sB~S diff --git a/schainpy/gui/figures/close.png b/schainpy/gui/figures/close.png deleted file mode 100644 index 49a888419aec1337de0d806ebdd8938ba2bdec70..0000000000000000000000000000000000000000 GIT binary patch literal 0 Hc$@jZo;Ptn zCXmHIkf|&R4IyB#0V{?TVB}i{jYrlKFhZGJi~tAw6M>QMSHeh`-k%WOQ5?*h=%Q!K z;?VSVAj}ac7#^&rr-$WGgD@mJ`=4~Vhl3sA@z@w7lF#QO_}dXIPB0Q}Wo3m#nIX;0 zOyvkuu8_$i3rv|@gC9wL<*}o2137dykIrK1t@vaL>lhCQgZ;$(u%Gv2u>Vtt$^EHH zt`8|7vyo^73i)>=9z7`N?^!D+KWO;>^04wE7+VgF%wuufS*%d}o)x?vfk6(Yk!WP9 z5R3dP;(y31DKQ5)bedf2iXuE3i~PUXFZIa3$p6t>~f!SYJK9tsaOLbT%m03eZI zx5wQZ^hhL2>h)OD+&MM4;U4?^C+|R>D?uEVO-Gr*GB$yqh<2}ze-UBir0!)@ApO_OAn~svf<_J9f;&DX&|-yuAD!HCH2F(F&f+?K+Z4Q|b-*{7x!$rCFiQ z1U$*=Xnlz(^HG_(WWg|g|FYI!GI-0Q5F-p#ozJt$MIF!~m=iYnrjNe+{B@OusyJHk zX) zbAmHr4Thq}lEkMv+2_ANkDu_^xD@eZW3_$Vs;e++8ZhpAX0}P-ntRf9U{I2NrkJv^ z!F#+xZJ*J>2RzmAH<=>KY)nDU#{;`j>#VLpPu}FoM$4aQG1~C+(zJ0FA84k z3tBW8uPU}XLs#q$9h^m%)22vlFW*L~*_2Jq*g$Du3?)WZ%6Z+oSU-^P(%G@rJ;iti zs=akQv14HLU_B?>ihI))A+ipt?MuH;L)z}pT@{_Pz~#->B&|pKZJ#NlJ)X?U$zh_1 z>MhrbO4>(9M+5Rm3*coS>RBMva|nYrc5`!^MINqXw+xWgu_4;2lmJVk!t^MGIc((o zjo_+^iZ8^`qmJz6mX;P)aeKxa91eGuQm$0XBFu9~H7POLMH*E4uAt%V#?sfTiNwmw z%F~ecKqye;&tk{&%lVPHrUT2Nw+HqEo$D`(sn!g=qLIuG+@WGuW)tC?EP=UH>Q?_YYlA+2b$mcMzYPq zFzym1O1|2wx$q{n3-+$kg%S-daq_`))bW$T=~*{V`j$;QHgt$Ge1eP`T7?Nt|8kS0 z{9eETZKZT1p|OlxaFx;JDCRJ_J<*3tom;4A&mEJ3F+bRp)0`?9eWoDWZ0ZD>g}0zVi5d zYj}b;D7`?P2ui*muK>|D4mBLR6jf-YU$-mcv%=^LcCo_;Gsu_Z6Nt;HY0CAtCzR%x zXjtEy(~Pnvvx2zl&1pN^>eQM1h<&D151D!VcQ__uweQ|Z*IDDq5!}a9qldFF?;ol( zsH!fVm3Qpsz@5p8iVw-9vo)bEx|O9k%NnZ7#plko0hNyz6=MFGmh90^o-Ces$AnlE zaA|KV{bYAcNLiYdr}4vk`wTM0tF&gLUM=klJoWU|qoe~c0iq+x{4QviIPNN*czw}#~S&b>{I$+&2XV4O^8&WSoO@rHz{Xy_T9j=XvTi!yvGT;7Vh zKlsqx(`9Jnk5X9!VQAvDyM46ELOGLmCXbaLtK|@S^TBL$qjGh4d+J!_l@1H_!nsbz zN&{4BR`;}rE-;=Qx%Z?+y@Nlx5xSk?bNNkWfg<7JH}jUoM{Oe=%T`3fbhSZD=Xa&5 W64}fGO~+{ES3ub7Y*%6v5dUvWeYQ zuXdw=zWQoPfNC7e=8Zw(rEca60Nkei+kk-VT*e!K8N$%i&-9U&w1cO+pq-^Oql-8_7ygXB2>g^<1(|8)y-a{LS8_e74<^pP%yvZs#|hq$1a zpfD$ll!Jpq*2mFVT3Hm7Ab$k#`HhV4v;>uxtLFeKp|` z`yK~vTdx<2?ul24!mCh9+8n;mPfSHl!VI$voOA|8ZOpBfV;ETWwa;BRkn)yn_-VqP z@n#`Zp9tMa| zDA!_2H!}Xx{%g}Z7p9}j3U52Aa^lGf)n8mKV-9Hc1%_>ytAvI{#>#|b%yvcY6)fYY z3$&B%%0`YqWFmF9b>Y(2hH?JzlINsmAxo(EO5(!D3~n<-?lxJ)j4}WGMPkF zhAbYj|9mdg-LUv*y@yzszZ58}cW^!WkZ4p}si(|m>C+f_Ak`Zv0WHrHKa)ud)mgC) ze3#6%p+HdD7MJ%`K{I3qfhG$vA!&Q8mtivYHsYM;xm=4K9y?Bvm*X>G^OnFYm2ur? z)$Vh1ZrNIhx9yAG4AYM+0WJ`Gy;|#OE(Id{T1Ii+lz?&#po-ghxNxfHtbo(k8p-3L z0e(^^`k#rga4i`OaqyLVC!HErBNWhUkC%t*K-Q2(n$l#gB?(NReX9yNk?_WmPvn<$ z$*yk&$m`CYk&4leIdeYDqmRglD;HJ!W8{^}wsELhlGS&d>W_9%s~ztN)?By4b_VA` zN?)7<(#>p&sTFJZZGKk{*f=w}lIQ`fvqq{uF{aSUf8FcNUbm)eb#rA0s+*_6!u@c( zEeXt(aTB@ zOfGA~(?-~h8rc0WGY(*w;D;ZzIfxsF`B%joySsANh4YY+^%q+hz2%;kRjQ;u-SnIo z&(L-<=o0?>q_hdbyY0=cg^`Nf8FRfz3B4IKtWHr?2K93#P(e%Chybt`+Sv|A~b$!dylMNzg$71Jzukz<=e2Jkvj55;VMb@v5BheCWgeg zBk$SfjdlBp%=J&`o#>n{HP_&0_;$H_N8dW+5#1>%{`$J49$Z0&d5eL^5e`(jjF?$s zw=DU@jVU$)S)R+P4JvlgSO6t^kq$YBq){A9j!8zz)HEmVV#9Fdw?iW2ojZpUnYhICoNBue#sY`L8>qB&un>^Q28<;Z3ZFlj+^V1RE+3G4oH^}mENQ4{Knt3 zvg;C9kkM?I=v&yDY9K<`XG$4xyOZ}KXDjLBBGcRGUW7&OQ;4pu`S5%Mj+CkxC#8gN=shexl96yDE4cdaBxICLJ-nZe7e9QE@3QesfHFhCk>#~d)65ucXBe`D=>6w1T^iJAwiHt2;mgB` zoLhPpEAU9JG1Kos-HcD>uXwPLNsz1_W4vT&L=B48bhC3$I!ab#w3JkTimD^_ASbMu zEPXJY?EHkfX_Ioz6a+@MV|)C`E~eK~R;#guWgRW%;^k^4ZffCIeU!*{e4X}(Uar2$ z#Rn$ljJfL@G6xgq4!lD{UE82ZVj|_BgXxP}dkb2RB5C9TYuB3nJ#obc#x@BkYjlf8 zJ8CO|OS2370il;I;7G>tra+Td;l=6kHpzhfM%BKetixs?*>$VWJ+^14$Ko(KC}-MH z;(_$9I``(iq)>o6ougTlX`&ET>3z6DEYe*G*7w6yi|-x-!4JXLTmYRnQCF!A5?$ap z1{yvGz>87U-1WL%D@i(ARdG2(9-t~L-P+<1nE7@luJy6bH6!+^><<@)PB!_zVuhv; zJNwkp;|&6|<;XgiSfKNZI?<^HPb2x!iIPtZ%-VPYh5lzZq+et!9#+lz$N1T3iC_HK z-I1G0*sGpFC1^-$k8<3eA8aOf(RUSkb*DA_L20?i&EvkFL8{XtpbG#1$tnMKUanlQO|h)d4NUN2 z%0(00d*VhB01M%N+)oeAQ{9o>m0n2P`!lV^E9kB0+^JQPD6HOEM4VdUX-|E>jKbP1 z(An+x*TTd77y_tL3id_s=~fG^5!nk2K^A#k6=_@q+OU!(vi>^Fq9p^um|Nnz*u3%G z{Q#cm%!v-OZED{foBD(?8EZ#L z0EWh(BoI>I!L(e1tLUBLKD>DN{g;;PT2ns9*Ewyj{2>FcNNRa|-|Qf_chMi1=h^d9 z=8(g+T@*syiJpRC3|q}n=0omz<92O7N~|%p>>!}}+8AsH_iYD@i%-iu>i6~N+0MC7 zrwP`XPuAXr$zBf9U;Z2o7QBFZ<=5+w@1)MKsFmN6A;#IRzr3G)jSKsT3OsJ~K@B81 z9HAz{!ArRb6JY^$t7-1|fyeoy&@*!jWKRW38uQDY{E$3N8vCWwj z>{qLCTeas=^b@1in?`^qjOT444mM-Z1dUv~HU*AGg=O;6dWIkGMlcZTUWZNd`KLb~ z7_+%6E4JKXhcu6TJ3ka?eAw)(%}PI9q1>MH25<7wOJ~SHwD1P=(3HbYTvb`2Icu?b zyp4ac0aJMFwFdA_Zda}&SOB%7>6zmi(S0lXiWButZA^)2XBT7yUD;V}@*&_(oEt*< zEU(&Uf#YlRJJas;c%r>KW;Y&17S0>psa{S=1c)Usmqig+ewgiT^O;qm*WkO$`_@<} zO?lU8lk474MEOyKKDx;)A&nGFTVNb^SeSGgQ+8)k@1<|!r?mEBpli@a0dA;^lU3aK znP7h1DN84{qB!r*K?bZ*y6m^>BPE_+{$-x-Gd@$QWSLGrM|!$&nEe<1kH;yCygjU)`sie^!&?vzaaSr{l;%g zF+V?31HpyZ8B09jx7Rb87b3<$F0-d6irQuqJ`q+z!YZvjI`cuk&F|%f~1x;VdR7hC>yM8B#_uYfq*l6!*1r7XE zKTEpRRd}Bn9T-#Ly+kkQU;RDdE6AhOd#+{~D>#yidM|R0DFOZ%TP{lAwh+71AOO^X zIhx&D_!7@kZ4T$3CQ{Qc28CdVUcZ6xHkR{L)wOIRdc=PeFt+MC;xFEcb*Px+j+ki=)X>}}b_^e<> zA%SiEqc8{%MU%Uns-DAlY9W{2zgaX{WX&gCqE>?)=6Akw35T*0x#cbtDV3xX!JV7+ z5aQh*q(`UdKwBHTBY`J+<9cA&YF);=va zY9dg6B{;2O4VixXeb4c^sFd*L`jP+g;lv*P5ZR@m)A)DnL{Fy(Rk7l3G?y2TbL@xU zhRMD~yl-(o?~rSrF*HvE2mZd(#d(IdoJFw(kyURK5roQ;QziH3Ni~^DnB1=S0m0N2 z(jtmv78kqyX$5l!)UB()4q$_paN5Ul)H_KP-oC+Z#OIjfs*vM7ar&|1iF|AuXZf=) zQJ+ED(`}bBY;+vMv?SEyx_w4+nNPD`(d-cO*YH69JbOezg4hm-I+kD~ZC|rE^E?43 zfNRwWpMPH4FAyI=!%t{_hH>Z?Kb7H$u(_VL#J$&AkxPep*=7teP~@ZfS9Q)!L{1dtrZWwE1}0AN|bpwlTtD@$3pKAkRm zux}4ME{T$zolBv2KkVu4QR{0&JjxmDhYBDNJi-~M_LsJtLga$KHKJu!-YFgF`*tSgnY=>_j*57s!LxCS|-BGGXc#zIDG_S-OJdBx(@bv zry3g>Oj=vd`MG7^G`6wUoV2xmC%P0890>a|vtY1<&)3@ysJD2MG>Y7}Do3RiL!`w6 zfLgzbAvvRGXO)Aau>+|*s*krlgBy(wkKi6>qLZUz02MhT+bhLrCNCUJj*jBQ+b26B z|Maj|$|^Gk3I6HAJLQO^`j9~aH;xH@iQbR4M&jWh2s*??u#7Kg)OtZgthHYlYBDn5n73r|LI;&*%k`3uLD8PAAg$MD!(4ye+@ zk8=}vj!5KZpHK8D5s%IKZzfLs=eCp$VpJ~}EZQ3LH?|61WwrN6YRRrq+U1Lps1 zf7OEdpJG0Biuv3s9`1nodu({P_r2XT0N4~jay=YK)tWitMPwW`TIcF->G#Ov)=#>m zMeweX{ry>wN`0<_LlQ;e^}K&%ODs9iqXFl=4U01-T*OtP(xM5^5ryxQsvFvFN0Gtv zMXn3>YwHYO&C=(ueM=~?w^{KIJLSLU&;I^hd`{P+jdu`K-kB}GKAyMyB7EsXE2}#5 znxpxqg|meSZH3lcTa2A)(dz1|psuct{sg$+XK93>goa-E_uac&@4Y<9JJ)HDgJsSk z59j6PrUe{1l1L2**ilwi=E`7HmrqYmLvz1={yfCr-{1Ud7(z#9-eGg(H-{tKtE;Oi zKhGpOWoS^b-xvGInoLF8*NNF*BisF8^zfL320ibe0;3!i!a9tP9U zs7IktFW1*Yo2XX2GiQ9v^0S7Ahh-6|w_hxOyekk0Mm~IKY_F-wGCVslVJ#zDCQU+Pgq~3sl0A6aUXPHH{TS2zBvzKzmViFRTY8o2Qu=#E0 zIt%sBH;9yoH^gFoXjd``3WWyJ{7G|gbHmA%m6E6fCgxairRIx^wq~@7iVA6cef`mk z1vvgi{MA5Ht)>PID%;Mqn1e>4ie}S}hc2XcQXaf{bFEx1m$Ofw7U@~MJ!noETJgA_zz9T(NnBf7J6~r3 z`?e9E$BO1~1O|qNk8e9hO{nYp%?(CH$P$*tPj>iPNDvWkkl z{(}{)*L*u1PF!AYTG}zSk^m73g(Ffae`v0&3*Xk>?q#B;0|kjE7!1aK9IkXY@xz$} zLj(eGRlXnA(@*Me9%?F(E@$5UWkxj8uQ61&IVdRT$My9t#9=P?yTx?H8o5j;L9}vX ze!gB)(+l*_OC+x>qKR$h6Q=ZF8ckeVyAOWBWe=fepsA^;HCiMII;T)B$XPdeT6;^& zvToO`gpP~;leJs zc{qiX4O>JhoU)^L~Mftwx_dRiBxescLRsI`(A9Lu2$ld1|j@_=4x8 zd&q38XP(!O@`YW_#g!J4vHV}|q(nx_J&X{i9cGaaE8D$WT3X(Yjg1+~zFD4#(47kt znmoR~c|+qOyk%U-^h33;D~q zz>Ph33S4{2OL9(O_1a9Zf}v2Me5Egwai#Po=9nVv*zWGChAaIJs-K$V=Ih$vLeKaE DB~=*Y diff --git a/schainpy/gui/figures/open.png b/schainpy/gui/figures/open.png deleted file mode 100644 index 04030f97175183f8e2aeea4b8e62c1a4a7786277..0000000000000000000000000000000000000000 GIT binary patch literal 0 Hc$@NSwWJ?9znhg z3{`3j3=J&|48MRv4KElNN(~qoUL`OvSj}Ky5HFasE6@fg!CBxDSY3;nDA{o-C@9zzrKDK}xwt{K19`Se86_nJR{Hwo z<>h+i#(Mch>H3D2mX`VkM*2oZx=P7{r ziAnjTCALaRP&E($v8W_74PjGWG1OZ?59)(t^bPe4^x#An4GAd< zJOkrpQ^)r7Yl7+XZsda>VlpHzYajE0N+lz9!Yj=H}x+~^oSgDoTzLcw# z@Asa!TmI_a!3X>Q6|%Lhei45=JAw0&dgFWc(iF+o!^CJ)_3^X69jM7 zUAs81`Su6*oeQ+3vsR01@}>wGf1Q(Py=m9_h>haj?GKE%{GHJlUck;~rlj8^{@5U< zLH|OWNAc03xhwmNbJw@#Z(uFa>#fz6cAVSlnW=rVeDNn2hLa7)dc$HHKYtLN#j&z{ z;_I!oY<`V)cV1Mln8L8_q80n=hr)Z9kH_2zm3tnxgJp99gIueZW!A@%MdFK7W+i7! z?XFH(nEKzS)YL@CP&M3frrPEE;`$pMWmxo|F#qsfc9X&9`sAuBw`VP0x!X#dZG-f; zgSr-s$(MFc4pG{&eoD>TD}p;Buhvzjd@q~b$bW(F)IYmZ(I%}EZ(NTvvn`X3H~4TS z@QROwskQyx9xJZAcMEy1+&G`7yVs;&)^%x}*4c(_WsTnx*seq{l&M_$ZN{ZN`=eaU z`rPLkQpY=A1lk@nUpVDa4p;NZf;z+53_?rApX%<=a14CAH065rgYawHbAxr#49>+F zeb^%B6aTDCck|4-5+CGzoa=tf;>ozPWp?Kh&B($v>p$6A^exPOARVYFxnbhwUDi^o z=A|uU*;p%7o%X}@MfJ+RC(riHH(s!cb8*(j=NoUWvokwfqjb5sw0yU$-}T(CN1y&5 zya>RXmR-vAH z;6G!&g5<&|m%M^~+qMXFPY%i{nfz<>GXIwwv*M?o`*7mbeY<*wuK4S(PX6AZ0m|W? Lu6{1-oD!M<4c5`* diff --git a/schainpy/gui/figures/pause.png b/schainpy/gui/figures/pause.png deleted file mode 100644 index 366bb9423145e19ae0974c2a5660d5e8e675d2e8..0000000000000000000000000000000000000000 GIT binary patch literal 0 Hc$@h?X&sHg; zq@=(~U%$M(T(8_%FTW^V-_X+1Qs2Nx-^fT8s6w~6GOr}DLN~8i8Da>`9GBGMsb3~lr=#9(gGM{^9sr}|K@+Hu+F!-LI^%jb}KBm)CWx~Gd{h=pKksJC~rr^vc! z^N+bT3l}M6x^pZNbS!!3mf2LOZQRk(&1l}c_8-H+R_nwAXEV%MuJ03YpLx$iDzoJ3 z4ndQSW3G>zJ37|?Fx&b4-1pGsmoAq$-gUoQyzlee^?q}+JLC_)c=>Xox3{<0^kvJI zWzCy6&q}z{<-#VN{f{lC#+dUcr_D+`^0usf^XAR={XIQ3kF9Eaf844l`H}abDY2)& zzxdV+ISbysd-m*MUh)0w*HD$oC*OyyTz&efkffv}QQ#H7_H>;Pk0eE?id{WSAP~K7XEGSXrt0?fdt&YuD;> zJ`*~l{Nvy8BS&1O%<$rJ`N;4_eu=yTTj#oUdcq5g%+13aIvbW{Eo)#63ZKl#$It(^ zz#^w{f;7vug}>h)=-#a%&9fr2!DsjG-DjUYOPe`)vhsl@t4l5pa;514e6Q-t%hf#^ zgx0QEbK?X5gG!G1>Ss1?+&Js^@8A2^uUp4;`}Xa%+qRia-c@L0XV=x<&c0N)d4g>Y zYmTRz=Mjeu>CeJcgc>{7@Uk`k_i}mJ@^SX0NkSK|Tv6GX$dYKfX6@R}lP5hlZr;3@ z>Eouodv)2^*a}aP0^^N$nvnYOpFejp zwQsC?-YU6f-MUo?H$0ZwzU}Vr=AC~&-B>uuE93QY#lkdtbo+cl}l znz6Y+Wz3E8FZ6EVr_wL2X zOVqWry(wWj$Wql18ydQ_OP*0ALr1LmMnPG5`Ba|?*|H1Xi;Iibn^avo=F$B9`}gVq zRp%F+?{_RL2sn3TT65{CI)_e%3Km~ypZc=0Z`0!^7ae%IeE!P=2M**HyhvHjB=NiR w*G#1h9!I4=6PZkGY;FJcGOV-@{FVMoeg)SRg_^J@pu(QP)78&qol`;+0PD(~e*gdg diff --git a/schainpy/gui/figures/pause_blue.png b/schainpy/gui/figures/pause_blue.png deleted file mode 100644 index 2221e854815f2e0f55c19d3b831a6557592b09e5..0000000000000000000000000000000000000000 GIT binary patch literal 0 Hc$@DDLE!tv_-i{e2@^D|>;5SdIs0zhPBq+aAPJv=cGFhC#>fIbX> z!F2fuT@nRH_KDKPk<@lczVev)lYEK61adGQ2NC2x7x3X^V<>bFvunGJ$iV^scEXYN zWbp+8f)l_%PapWkEua@vk+#HOe|~2HKiU8V{IB+P7T`a_>?#HJq@pEIz_({ZOIWvR zxq(0;ZkA>yjz+SfPM$%^PV(CU5hrpUmT8E?1F$Jdv0j^|mnB^19rCEM>Uq_+pPo^j z+ka7-Go`jOse_%=qx9y-G8hvNY-LRbLswT5o~)hhU1P)E#5KHr(Yw&A+(HTrc#;bK z+}lLD(!4QtZ1zEQh1+^N1`x^6i0{U<*~!V6A53zXl8@sK)y8QB^yJ3Gz<)?UKbH)5 zuG1&YzLLGnt-=oLwv8^j7koEbGP-Km0dA=s#$Eq2k3cWC_8+4GiYb|CJv#~UcU9va zId304FSc;!G-Ydi$VGMQEoZV12=s&MR?YQZx{lQ% z`!RR2Qq5Rv#t(_DK%rBV6Xa+z*P2Vb7%(Va$}4i<=W&OQm*iHI@X+K^e7+(voqHGw zF0@-Ncpu%Wu`g|{KT=w)Jpvb&Gd{tlZdWeAjLbGsstxOB`j+YsFd;qRw>NK|Vc|s1 zYw)%?_U0?sYw&Tk_eL2AsT;NoCdiJlUqK}_i{nxQSQhZ)=A`{ z^#jo>HoRnXfAwlt`SeFyPAOY1#Z{q{`%0xTOgrTdhNLU2E^4{==v!GytB+tbPc218 zYnf;8?oC=k>U2g?I#X$?qGqP?zE<`${bI=9DtK(u&$*a|n@KOGS>+A9nE1OczG|s* z((?6zhbp`l7ne||7{zICp^ES6^j% zKXt_Yh)iebxT5bh+=#nxZZf=I!GWe4#F$BVQ@>t31_vi;GI^ZE+j1Rk4zcO1Fk+I= z=i|3&hSagag0yQQvTqMo%}r%62QlfIh=C%uWFfKJFjnYsf4pkE)flTTTcroS?p|3` zI3w&@^6n%qC=s{W)swxJ9|Edt7~G!^dq3x$01u;s@m;}wD+`adq;u^sA)fk-w0xvv zF%9ft6!F_if1$EZwu05KrC16_HQNIpWSZsQ!qyKT616(i747QIB_wMobm(l55GM5K z>2tU4H8`+X>oo*oC81{`({qkh?Np7)Qkx31HI)Yp&HFPd79&XC8#*;(Rdfn!5%d~& z$xB9Bwb^~`9B*-%&>a|BCZfqiH2+>1ew{qms(yq%4w>I4>bycMSC?$0U}>q(OG~Vu z-KEUA40~#`T{eu4 zy)nO{)X)*J=BM-2W9NPD_{-pd#8bH?b>`03BaRu+z!o}) zR~phA5;Ib$`Int=qEcg4sgToE@PvMuNSS)dnZ>JWeHSy4(E%pqEDG%DOb?P<+=S?) zByto3QHPBTXJ?sMbt80^m{F}u?T>9cs2Vi+1k=|dyp!9wvAUGKjByhk&K&;G_mg%3 o{VFdgUKJZTuI!a17F)C<+%n+Bv>bi-yWqdJG(T%re#)Er58YPAXaE2J diff --git a/schainpy/gui/figures/pause_green.png b/schainpy/gui/figures/pause_green.png deleted file mode 100644 index 1da39df10109da36ef37d7099df24913182b5176..0000000000000000000000000000000000000000 GIT binary patch literal 0 Hc$@h?X&sHg; zq@=(~U%$M(T(8_%FTW^V-_X+1Qs2Nx-^fT8s6w~6GOr}DLN~8i8Da>`9GBGMsb3~lr=#9(gGM{^9sr}|K@+Hu+F!-LI^%jb}KBm)CWil>WXh=pKn=-q6YP?7&< z-?&dY+wK`H9NGLL_<)w1XFx!9^CH13EJw1s5*!8oH)XlLI@Z48in`zz$DDYRjnm8) zdTDNq$~d@Xp=svDRVad!{p)@3KIv@~wtAhIvrNK*EM!sJHzS6e?Sm;iIjbJ#*&RACcSkzz+3MIRl{I8lAHJT=m6o2=2x8}H{|x?uQ*cjJ#)9#4P1%>Q`m#TiCE-$tv({s;G? zR+Vi#`|{VLJ9XbT9)8h(;Gnpw-)j1`WHK+Fo1FOcMA=!XlKDE@mwZvJcE0&D z=X}z7tMJVK{D0GTILx}zYWtrpuD0^`hu5*WdxMe;^-SNs|Fhm=zU<*g&s;_3zP>SS zm$ad6wLQAsF@eFI>KE4U`f~nl@j-A6p0ZUw87}SNq9AVt(>& z#h8nsb6*^unvfPq-cfrejmqj!#?Or+EvG{hc61xeNu7~5Z|c>K3ca0{A`$r(k?X|F8J_V* zYMc+9qFY<!YH6*?UAV6_b))K3w`ynpaF+AEw-wK*P3l?t-$`|~jqG3NTxrn$b1ujb40Sx%cS3 z{0+{or`O2eJ@Bq@kFt(x%b`afKFl!w^FP)6+G#8P-EXA7Tz$jw)~d_AS)}E^=@XqR z>N;Cr>1^m1D%@xJM_|g6A3`s?w=OU}cu%4F-kXbtHrhI0?@ravT=+pc_txt(1!6nr z9WI_}w#Iapx!&cA-t!Cn^Bx`~PkV&p4tO!R4T|wl3g5E2B$B!n!|Ye4wJ+)78&qol`;+0G+PH Ay8r+H diff --git a/schainpy/gui/figures/pause_red.png b/schainpy/gui/figures/pause_red.png deleted file mode 100644 index 04f578e59069a9b7e6344e7a9dbfa252bb0582a2..0000000000000000000000000000000000000000 GIT binary patch literal 0 Hc$@h?X&sHg; zq@=(~U%$M(T(8_%FTW^V-_X+1Qs2Nx-^fT8s6w~6GOr}DLN~8i8Da>`9GBGMsb3~lr=#9(gGM{^9sr}|K@+Hu+F!-LI^%jb}KBm)DBho_5Uh=pKnXl#Z|sK~!( z=khMi3ea4e?J2SsnauY_p7D?RRBo&FXKT{dXj#EbZOLImX7fds+GK%-s3>&gZ$s=kHtm z;1^7o;mWr@B4r=@u9yYJmZvsl*$4Q1V~=?5d(Ys$n$5p`evt?6vbG$u#lQc)dmH=uPKngD&nNZ=&G>Bh^V4QUAoq5`QQ9_Po1sDEGR;brM&|S2D zIm_u!pDJ1R9Ga$D{ZRO6QOVD(3qP1_&}Y0}aLh3Dp8kQ`Px>~MxN~5AUWr7 z>C36%e}xY?2*(JSZd_V#vu|I8*Q6y0Wz+sV3gyxk{TikyX%iEXP&ct@XXU1%#U~c1 z|J-q*_`Sw)%T)^d4t1+~hKL03l>2L3*7QvNw&T)$b*uRJgCm}{7_u%E3jMacd(7qF|?d8JuJ!!Ugs{dm%m)Y*}qx}0j|JvP|C8+EtRDAi~vbm4G*|8K~ zm(8)$tGs?7=N3zO@%n4a4{u%D)ptrZ{V>~>x3@MQ^>Fw6xH$OmBL%Ai`vRtfu0OOO zNAK3g8T~u%B*d9A^+-?IE1GhAXC?2_yw~r%I9}mRs5;+_7LObuT^)W}&fo{5zP}%E!FlbH8qRVTv0c)9NSp zqz)L)+iSCQ4UdSLe%@uZ8Fhc|uSrl>i@y<_KIiN!UFOsB&YjV}OJ@ds{4>>2-V^gTe~DWM4flGv?F diff --git a/schainpy/gui/figures/pause_yellow.png b/schainpy/gui/figures/pause_yellow.png deleted file mode 100644 index 5e6d7384ab1bb05d76b6d40410d9eb8b5f0f103f..0000000000000000000000000000000000000000 GIT binary patch literal 0 Hc$@h?X&sHg; zq@=(~U%$M(T(8_%FTW^V-_X+1Qs2Nx-^fT8s6w~6GOr}DLN~8i8Da>`9GBGMsb3~lr=#9(gGM{^9sr}|K@+Hu+F!-LI^%jb}KBm)DBtfz}(h=pKmsBcb4sK~!} zvo&QCGc+VjV;j6Xx$i0(9+dFakY0UIs8>7usH?E=1cAkw$vukiLYjOH|23@MWw2E7 zn1ai#BO;;~(NfSAX}jIP=5Zb_}VT`C#;On6P%uh;jwo6=r?V2Rt7 zUKT&`zJJ%1b)6oq%Ifddd3-gWxLRGs>hB5v-a?z{Kc5xWEU8jxZf^Y`(s$~OwbI4- zt|t~c?vs_z@!nk>r?N+{>{suNc}FIS?U;7?y2;g+_=mMWza~dr-_yP0#?CYsk0gtK z?q((O(aRTIcy!I>+5O2MxMj{AP-KdKULd1?^3ClntF<$pBwr1kt>M?MqSX|Bnek ztzLh_C;D0+zj5Scy4`uYe8wZaiLK?F^WqYItUs~!uu`#q$L!nM@Ai8~a_wQ>b*^6J zY~dUJ9}&%m{@&<*yZftslE*KR)iukma@WL(H{SYg^N=|#^V&}Pqm^Q5f2|knoATo5 z#wEuRkINm|TVa>5^LNVU3&|R`E7+p)RjpE~oHUOK(_yTqd2lE-+PXj;ZM42bQ->G&y)?xC-jb&1_pMmv4MEVaYMU z!X*aadV?W+|gO@Yj>_bUgE#DHf>en7C(Kx%f)3}UiSh*)26>m zcrW3?ZQk6Q9Py-G9>-Vt`J6hPm(vEfgcgy9J6EA4L`H*^T%`8(#qv#^%-dOQX%(17GUxn}0x@c~3 z`h?f_9Z5XQXJ1~Qt1iNSuW#y`n&55n6>>@$dz8NYS^fG+m#ya(ufmlr@sD&RdaRSLeUi{<;E(*5-V#FP4015u6qCo_N-NJG3Fx$IOD{@;P?Tc+C8s$WvL zU6r0B(8?UY{Ju?4{1={`rP6F$zHjxt8Q9hz-}K^`Z}?2B<=gMq3Z!!`;=f#PvWfrI Z|6TcAS7*EIy$C9EJzf1=);T3K0RSM(j+_7h diff --git a/schainpy/gui/figures/pausered.png b/schainpy/gui/figures/pausered.png deleted file mode 100644 index 1fbf3428438f6fa5f5c39d0f969645c074feca5b..0000000000000000000000000000000000000000 GIT binary patch literal 0 Hc$@@n2KS>z6@Ei4F)4)FoPjQ=!(doENLP;zwNsA zCCYNe*h$wCw`99fw~!?LGwI&@KmG1`p6{G*dEf7R=X~dRPJ)#MT7XxY7XSbV7-LX2 z%vFLV+?>q0yUza%b73Rfp!ES&L$a$(gC`K>BjCI$OIpt`!c5Dgdv22*Du z)XCui6dYAOfGmHI%cHxI!|2Dc(m@&1k|0udiTW|oH3(m_BD4ID~z_$Q*n zp+r_4aAP9Q2X9aCqWB`A2*~%Y@6ndQ1U&OXS=aU#{C|CiX2;+OKE9N%CHUe#AC*bZU0N`kfG3vanj)`2zrI%-1qG>Yb!F$!D-7;9#9VIq+VO3@ zHx(OC;g2K4w>0A1ogc-RCS>*IIRZCnQ+ZxU;sC z-=G;4J=$1Y#?F49%Mpq@0sH7WR6|`&^Hr1Nip-W$>KLxK+4h~fP`_)}UNiA5cx_N) zG|E}DjI*5m2w67IBn38*lrJ$Uy|-nKVPEn~fsLgH6$fdm&9w{}%x&y%zPjbMQ~bL? z;#il(=vD4ybV{H?*fs0IJln4Sh?YkFAvL?F#}0(Als$hKn=N##P$fswzkB)80HI9wcFB+XuBJW=Ibn& z`5=6<$7Vvmq;K53;z1*NXK~G+D7g9nyZXKi>}~4wB&A7g<~r!abt!}7pfiVZKHUbVp2)LxqofCFc6%u_Yb_krDiv zrSW}Fr(Z6COJ!8}LeyHKvy}`j>IL^A&Paj)I77ky)zz0g39I@7>6=sOCzpv{@m_64 zrgk#(RG3tXM*e{Qa8I!Xd<@Mx|~gc9ULboXBf# zSQ(J_)@3-8ElBy?`?%FV+Kf6w{IFK7<(yViu*7GoNF#YVG%^?2XwDZ&6Lg1E3qK1L z9+gVVZFQneD;@PHe&^gM`ZA5G!e9jZ-&U{(niPncwKU3n;#Nq_%XkjWsa?t2B11Of ztM2vX#39U?!1x~d*@;!p^KV3bOa{*|dWxS`VfPkxUbLlRlBI6X&rZ5E zpJ@eRW@#wniZZq*dX*Yj4rx2XPK;tfJ5|%A>YSb=c<^TPfPIsc@^spuC)Y0}%LR_3 z!aD^rtDDjLzC_@caq*6nMb9lt(3pU}pM*9fYYhQYM4KE*8uOW;^J?X`%KD9)8jVaM zeidsf=(qd?uQqNP}mRj25NzHAgq721sL6j$>v! zcild|-3XJ+4!vX~1kZU#8Aqc@{@_y#AI{Gc2`=rl-f&k3xwriajrMMjH{`ic743Jj zGmNiv7XH3lS}nFXO((dvhNkZ*XH4H!PDm$zKHab{w`m<=@?2hFPUJ?^ZtCGK4GQ<#=IWDQi$wiq3 zC7Jno3Lt)BQhsTPt&$Q{4Fo{lnOgw2D6bgmE1>`MD-sLz4fPE4v1uyFOiRPA3nT;4 zW{1#c1GL1-xhOTUB)=#mKR?IL5u{$hGdD3kH7GSPrP4+pT?LY6kdapYMVYBUzk+;Z zXK163AqMlkKAQI-VXF@fTstnHO|XcvbSYs)*pS#oe+|6(6bVV0^E=YfN=6mh)GnU51jCq%~ zyh}5W)S8*!s;GKpiQ{&`w+ga~v(E9psg&V}bXt+ez`O9;FaHZm@*m!L&i}}&Yci+K zxy5F_7EE&Q-|xD=Y0cUx7q*>RW#J*{&C@@<+muX@0`5V z>(Jf1Z;!sMuP%(9tN-V3*_@E{O_3S}bp;hKs@l^&t!;n4_gkFV+L_Duw7kkBQ>KDcc}kVZTN9#)oI-;%27b|80vj zS|+r2e_mC`tpdNSi`%yp9$w_oIiG2=*b@bD(}2|~cdQ;Bn6l(z!}A|a%^Z8@as()R zzjj6^*{QPMYr@`bC$|gk=D*JKOj!RxgACsu!Q6>qh1D4c)TTsx>voIo$mUYftDSqt z!ac(>mB)3Tv9IY*%a?G@td0yo&Mu{n1=O?+$VA`#~Q$DL;mHNty8Y}O=_s$Qv z#IWPR`O=avEpabon~GA)l*0-h%1!i{v2lCfgcheBmByR0MStq9bZqewXPG?D;K5t< z@HcDc3C6!m*)!+bD{rTa?wu;{_ubm^c{9KNx)>kM(`%!?6c+vNoVvQ<}^Z>zaNfXaDC9o5W|hmTRtc^kg=A7b>wWIBV;aqc^1& z1{}yqTDRTk^{b=XqxavEEevFj)=Zj{;3wS4;<`Zdgwea|a7DF-P^}BwUVUuYt+s_p z;0W_^O`ac)8+bOJaMa6k$@MxS?3PsIYH*3id^YQEqQZ2^4a&17 z*1p%boigu)w%Gm(2BnA5Usab!#jpAI^VGWCe*ZV7Z@aetxKhyXCEFD@xz={dNG|&K zB_W(u{M%eJ-qjromfYENHhRsLDYLpROx@ztZ+3OPr~41--}Rz)_Iq|4y?>TAbFpiV zYpA35+K2}YDWS(2mu^|ByDIax{$9?p8Mq+js30 zv^U+fB(Zkd?9cPNp9deDVJs#vG4+V@@%1@|>yLjid%9L4-i{}{c*6M)H-Bxb=sU1- zb7IIxb@4S{d5@XzS1)>Y#=R~3)Q<(Zb^gzuyZ)>GxUBu7y-q!I*vW}beSg)zgDMYC LS3j3^P6GK4GQ<#=IWDQi$wiq3 zC7Jno3Lt)BQhsTPt&$Q{4Fo{lnOgw2D6bgmE1>`MD-sLz4fPE4v1uyFOiRPA3nT;4 zW{1#c1GL1-xhOTUB)=#mKR?IL5u{$hGdD3kH7GSPrP4+pT?LY6kdapYMVYBUzk+;Z zXK163AqMlkKAQI-VXF@fTstnHO|XcvbSYsr@BIc_x}uT|$}<-+Cdbq!XBljY%PVfv#PSa3o`E$M5_ru{p zlNV*@W+#^ZZ*{K8yubLtDnZS&nj5NxUVf>p_@nw(zII-{y+Massl?|rCeG4^cfU*Z zu4$#)@I+4%cz0&6(Uyfv9P2)0PDu=~=zX^6S>%5OM_JcZ3R}N^bK6+(%XrO;!yGNb z8V?`YXhm)JPMmn~tW{*>;*8v$&#i4{Gw$4-!Nr?-m+z>9o=Lzt=0C;B+JAO(ilkmX z5>*zJuKFl_Yvnb;A0a8%{e-oh~xo2(sY2 z!J1_7`KeCcpCIjUnzR4it)FU%&*Iq4*Ij=!&84?QXLgFn^zJR?2Oev$UpgVSKx|p*qJ0;YU$C*B6U{u{ zusiS%-ywF{KRbQ5S)W^R%T=Oz;k!@wLwu&0OFZ1(_o;f>V&^;R%$X;4Jv(~if9)Uf X3u+&D)Al@?6kl3g3JFc>W-XD}=4WGOw`G2>wL69>AwQznmVLG1j*w!CctQ$k(Hcc+ zMdG$3?LN#>epK@FR!hQlDaC4c>(_mrd(ZQn_dVylpZ9al=l$o+CJ{ZfwT!huAdt3~ zC!Va_SF8Ldb>-S7jCEFSOPFMj?V#GOb>qrJli^8afHkW#$5l1^@;ciDa=@2$mg!9={K1MemG{iBxu0@ndZ;$p6*8&w~8VFkh7-ze&YvVURzb4XZ_H zHK&3=;A>uZoS&m!FohFcNHJ2B&C~kN(wHVZYwzW9SA~tB0;cZ`%iM23czZ5PuktRB z2U`f;2;+UW&|14=k)%l$6L{NJ`ChN;l8|RFNTSffD?6qiOmT*<&tpE7&ko(bs9U;N zE-RJEyJj)6#FYLKbUqRM`moW`?khy->u3+AUxslfsCDHf$%X^iR>sR{b16WwDPt5m z_?Em5xVa(Hf8+aEh)qAjJ;#1;l^{7u@W=C>2r@-fX&8V;rOWv(MtA3XJ?cAtJ3ccZ zt(kJeSR2$g*D@&atLw@7&VvcsK;`vWO%>8H{gd6Xukz;o?jR7!A$v^qr#gP zTj6Yvq@k7BpZ!gFSual;Kpr4;!n+a(Mg4BAxcbHNyve9$+hw)mb58KOBfDm9%4ar6 z4V+9(FZ)IqLJUo@1Mpf#ee(#gvmvHfe>~NvRXP*DY#r>cL><)L6LZZ(XIXX#0>Sc69jtToc4&En9$*8Mh z`%m{ZmRhdJcHUlR)Koew?zO?bqU8I=^`I{0^xW!W9cvs~yl;q{plaI{&7^2Hg!_m9 zl+R?-uKbdk2IDY#s8B;}0(w>d_H6&dV)Z@JkykFR9JoZSXk`bdJx(~V^!WT6s$oVa z2Aq&L2*F$QobHsHU9Fe5zW|dQ)h|#68?`(ou&nmNDDSDt6jvT!24b=WSk7n15Qj%4 zxZYS?5{_9B$Mti7jAwGw+nlnhdz+=p`;VvQDiX>l4rZcSF)7AX53DW_YKn$X2gU1= z4T;;@{DOJU^ysCfeS$O8ln&yP4lX{S$Q*LRI`p%-&#JaR#kXL|A=jD^o)XiJ=fpa} zr|X4vDVcpM#05I-REsUfK%0+ZaFo)6NYi|jlquTdkfnYsy(G9XOMY>WL83E??fwve z+;tj*wd|`M!!SlUp>vi=yVbm90y=N#e)BJykHsDYJjq!7+U=CN{XHkPZaQ`C!~Xd} z|E4zgh~l!{mhnfPDkeANKd#GO*WwmoGF-#D)mUTBFU#44mDiqX61DQhN%l{Ib5aWn zrZ19&6w3l<(R1|H3ZI(v#5|hwRMPz||y1tq37uDmho^ zy1jcSKxcDE#q*Jcfwp&6hNx$*A{^OkJm?{t@V4Ci?-$&cTo(K`CoQ0`>J(|So<}Ek zu5(liY$7i>Y`Orp9k-tb-@Yo_l#RWI)(!etLhSP9yyqXPIZ^-!WW(T&kGTO)Y&W0F z7_H$y#_@e;44zIzR%r)DvbMf0{#_TJ9KtPbd#br01zE-1aI~`Rn=tWaq7AW9nx#Vj zqud`e+{f*CGx<`1A%_UlpFcaYdXiHqLv8e2BjKzpTN`g>fjU}d*B6Gu1!i~!S*Bi6 YEZH(9T3t?w1ymn_mm3jZyFE1hAL0|tplTohVo^zE8p5W$VyL%(9@GcP=o{)8=)=th84nV*@-NCv1=0jt^d7Gf5F6r zchWODW_HX>SeL(`bLGZ~GiONsV3?J$;+Yahlm3FH9S`=^-QO9UeBQ_Z`<&uDC9yxt zcdy>P>))z%?<}>o|L0KN(KgHO*(`s~t!)dpOnvBAApM5b z-r>yuCHW$jou=npr@Vf-`aYjrqqalf{hR7Nxu*Bn;~IMvrdso?Yi4#xT3O@I7X5NR zbNa!nF7@62mZr(xNa&wp+JMy%}Qatul+?%lDy`$c>l3P-ZJ2>4;ELZilUv;d0 z_IIZcqjCk49phq|yDdBC%+T$B&C=l{yGC1Z!vwvi=Lc4cYA&_*JTtHLC&L_>%^HFi zj=g^VI7!e?HQ`B`cAtJ!i1SK8gI1^K!JJFtw&;3A9o*TvbJh8^r(bSad+WMUf>!Vo znckObH>Yx(xc6Hp=6T!aj`a1phNn%L#OBLO+^*z&abtOPBGZksIp@3;`VQ2(Ej^KU zC5u2+7RDCdC)c&zzbMhNu126lX zU4G{o?teO)kkGSEMd*WdkxgWZp?0p>ZgTe~DWM4f;Xc`} diff --git a/schainpy/gui/figures/start.png b/schainpy/gui/figures/start.png deleted file mode 100644 index 2820368e8596875c15fe786090cbcf7690d489b5..0000000000000000000000000000000000000000 GIT binary patch literal 0 Hc$@MzCV_|S* zE^l&Yo9;Xs0007CNklTJ^-NtaC6FHZ?MqQLnc#`bzHl-0uUx?E;eD6A=A4EMr4#!e*wIqo4I{U@WYAkU9nMX9MZcO%5JaVx zX^-3&hx-=H5h@=|mEU!}Wn4?2Jr^XPR5)?uC(GK>w?AJ=_ge9X!!wsn>i3MghaS76 zE)ir>iMGFV)Ly^;RZ>ix4DyD4b$wE7`OGE~shWnScW^RTOeDSW+;c9d$)pmw1Aq9! zf&-*#>Y6Uf^~RGpc4|PblnBTj+SB#Ghkis*)8~>?aV&|Wg&}nW0Z%;fuc!X8>r;!4 zq*5;_#L+B@2m;_YTmE+9yz}}*5Cl<_#f>oZyGjK}W$F@{bLL$^Pi0#WHI&( zi!YtZmhR3GD$KO2&ED{_SIs!1Pb#RCj@|Q6L$(`6?sN$im~K}KaW>cRycfMJ_|>)@ zN9vNLaQ$umjP9QGB$3^sv#I6A>F zp>zsN521_Df}4V2Fc_NdAAoYkT5nw!W+w1JCNm5L04x>@!O}s5(oX~0Mn*<}77{=r zHH8RGMkI~t8=*;KsBV&c6Xm{sp=X#TaQGHx(>5D1g985TL}P5p z5()%FCxA9W3;510q8C+BczO^;*jdDHs*MKzSNk>#@SkBel>%E*O~Gj3`?HyX9ctAH zAdmzLhqZ7ul%cqJIS}3Ut#=Fxx_ThCyqI41nCN-+l!BW6f&3ACy?kb-zUiH#j)jeJ zaVby=S#l=ULZExsn(U|hJ~puUUPY-{PC0nTn7XXHM!7lorWfR96>DiIe4JC+nj^lH zrSqvaTLZfL{dAIjyK8U07hzo59x`I#s_Pl}%djnwy)=YE}FzU6N|5 z&Krw5^wx_V@usQ!X~Fulsx}%5P}}82p0NtsrLIRzUP`kDC1Lgh#&j(86 zBqbuG3DXCfeqAogP(oSX>aCetEsS$9i+<&6o?K)vT}3y=<;wO?UoA*5R9tWQ1lq&v$A`l&T?$5B1fsy zht?wplREa9e!+$?f2gujwm`~+oc*Z!ZUvkvphD$|a&1r2A2erd&xkt!B}2-%A4dez z-pf=u!O52%EeCZ=A)b&5P3wppG`G((TI*=qhCAjr|J=cYI`P*!+*IoLg8dZ_f>YyN z6l+}^_*2y-K0ZSiN0YI@av?gWpyY&;sw+%kPx?N+fyT5UpCVADLn-{~A-|G3rDkIn zqv^j*Vl&D$+>JD(EiLk5K5GP>>*iXe>^(c*8)T=L z@iwUq{NZs`5@{hf`$0e5!_GL>b7%6xk@UNYD|a>wv6r54ye(e!)Amz?SW{4w-j*sA z3h$-sV;mli9CSo+CTZwIn@Oif7oJ6}?ig<8*O_i#cN*Y4S;VEKB#bQSDfZR{H1v7R z0h3d<_7?WBRvOFv;if$MMI!M|O(c8Vo%j=jh>>DHEGA_v7X1C1J6ulf$KMuT3}5qf zmW>=XV0#`OIG!11Q@(;L`BM3(Ltc*3IOoG;&xaK|-=lq(A48VOGwnZ1B)!A~$%s+F($;AKw diff --git a/schainpy/gui/figures/start_green.png b/schainpy/gui/figures/start_green.png deleted file mode 100644 index a3b4e7ed408ab6ee04c8191df2ae23d16cd0263c..0000000000000000000000000000000000000000 GIT binary patch literal 0 Hc$@h?X&sHg; zq@=(~U%$M(T(8_%FTW^V-_X+1Qs2Nx-^fT8s6w~6GOr}DLN~8i8Da>`9GBGMsb3~lr=#9(gGM{^9sr}|K@+Hu+F!-LI^%jb}KBm)D>1Wy;o5DUTF(6`yqsUrXH zR!?4*y;*H;^hJ}&yi-rPgr&b`Dcz=VtMS&Uf+#^5jz{qZb5HpE4VYRd?C4nd+sOKP zOms<4pmrdybMC^%jq7UC?eBd&c<^P*lPmERwG4NkFYo)z>3#0Re1_nA zr#}~*{A^>Ia_;>m^}~}oe*OC~#p-v#jpL89m8L#V-*;4d@BXhFvbnBIJM4E)?efm4 zZ2k<~+kQwaHrghUvQg*MrRmfAtor9WMRUs31Wvk~^kW)B*P4{)883c`o9z73&A;H< zQ3pebRF63c8^zvcTK!7C%^~g)5qslv{-dqax@FvvTUnO*QC_Mes7rY zS9V#lVoFP3s37BXCHJTg-uGO8tPGV7)J?Z8KVQil&a`Kl`8B(*Z`MUje$$=ltgrQY zSEI)Yxpmw7RN4GzUtY2PX7k&NcXJf9eogXn3-v$vO|Hj&mA%Q`3xWsKJFbSCzVBn( z-ZgK9NX!Ju?sJ>H7E8@;5;WV;IP;3@oin~la+MNaC&qHT5cTTk`n$xnxwNA_I*OS` zH0G7Cq}6z+a%X=tMPN(L1hny7P|a{L(7BX4jPaQ1-&th84-d z3C6j6*N%p)Wc|vuRYQ6^cl+@;X@eUL2P@6LC`hjeJfD&!5&q0iIaya*{mGgq$}8F< zW*`0bCaC}Po*0#Ox&NP@eA=|o#_X=?)Z5kD-=z1?7F>B}*|SG2A>DJ$1s%8~U)D*3pz>`D!ur)}xkY_RsR z?H}EiOE}f*Z2_om^=oxc+AMvE6(R=B=$3?!CAA%UhSW1VpqLh-h|9EGi3T<9eg^ds@RhiTBHz(?3jO()~Mi;^u()8>R@< z1d8mi*R*TZtGuhs8E<17z!&xGjqaxO?Ebk&>JnGVud8)Ec_!2=(%kanvPFTXZ!Rz` z?p$_BxFqe7@wu16%}YQ0T=W0pkMiT4&;ObCuUx;BF}He~$(G}*qoS7jE?vCiN9Su3 sO;yul+v^|7uocQI>x#3{tqb_icsQr%@67#Hb)aJ2)78&qol`;+07t#acmMzZ diff --git a/schainpy/gui/figures/start_red.png b/schainpy/gui/figures/start_red.png deleted file mode 100644 index 3d8f9c1fda5fc7222d144413026f42b427cec38d..0000000000000000000000000000000000000000 GIT binary patch literal 0 Hc$@h?X&sHg; zq@=(~U%$M(T(8_%FTW^V-_X+1Qs2Nx-^fT8s6w~6GOr}DLN~8i8Da>`9GBGMsb3~lr=#9(gGM{^9sr}|K@+Hu+F!-LI^%jb}KBm)CWs;7%%h=pKmXlzbMsKCE> zd*5z55ViKISCqlUUnw;P3p+2ea~dj&Zb;bDVyAJ?MX=UzqDilvhsoM$F7rc74t`K@ zye!dSfAPvOb;qS#OC7j!xBXw6+4%O_yGw8P?*86)f6ngjyGl20u0A(s=KC{eX3pID z!CvUVG7p`imnZ%SW)w7}EIGCL>DvSMFaG1mz4x%~lftCJzw%$1&cz)F_mmHaXMPhG z|3AU|>*w8arDo<=ET>N8hzaMqY;1n_{Qi9f(lh6aHNG!rl-#Ph{rX--QMcEh9e!WU zz1@CE-M_P^w=<=9Q`^5tF-gg(t@?f^ran_=JIAzkVy`U|W-pER6U6XzSViOKuHZy}LR zQIZk!rkWEokp-(4cpon@VV%BMf<8}fr}c^z$;57hiR9eI?`laW(q;>C;H zqBgUqUNpa_^M`HAkq5s5W?c^Yk@ukT*tNOyrX4*jyXn&R%`Xa^w?aB-}v5?ahM|#Z-6}zp=Jghi9&7F*Nqo00TIXNh1MtH09 zNIk}QE1iGGs=x2Q5V?K%R21@NrN7R3tD#$`;v1zCs+8k%Z5PX$tXpBx8)nNjT)xwk zkiziu`mKp?-?f=na0Y2?2<6?CdN)#8bhecAmfFN{wMTqC;(}Y8JwruzPP+H;)Ys4r zv*g&!Y+2T?=HA)7ezAqM__S$^n{}-xZqO>xeV29T@zZ-!IrD|CD__<&TfXLrV2m_N zm{wrv7mZcX<%=KvJDb2IsWUfWU+r~QR^Lsxen*_U*E-9(MT$-T*QC&_3ZL^@vJ(6G zcABz1UpYNs>vG;s-Lr{(p)=nv&+|?@`AI!pt@p1$*8WTP)LTOZuc@w13c6%n?ltSz zORHONs>8zeSu5A21(x!e%NQ9f`^PNq8S(wrLQqO;HEa&nXW`sO< zb8z>r4sTn3QsC8Ze}=0c*H|3Cd0JC+irbRar*7SHw6JyxU^N$Pk6tVs_rgm}S$Vz8 zDZ%HSk9FstnldNv#;yF9Q!X_+U20sIH_Puz<$_ETnYe_aQ^m|ycYkj4+aAuF@#W~k zK)WZ_C0?Fgx^h+N2RH-#6j%89wb_6Av@pgpW9R0N8?K-J(f?tKpQfzFt=~smrGrwu zYL>axcYpr;C|1UV-?XLr*W(YF(v~}yT?~=<`@C>&&Xb(G2kxkU+m>&fd#W$#W4FDj x@zc5Q{;KlVRLj1wXV`Em(#mJ`wvKiGS#Jh%dp+Ce5C$r{Jzf1=);T3K0RR$Lq_F@1 diff --git a/schainpy/gui/figures/startred.png b/schainpy/gui/figures/startred.png deleted file mode 100644 index 3d8f9c1fda5fc7222d144413026f42b427cec38d..0000000000000000000000000000000000000000 GIT binary patch literal 0 Hc$@h?X&sHg; zq@=(~U%$M(T(8_%FTW^V-_X+1Qs2Nx-^fT8s6w~6GOr}DLN~8i8Da>`9GBGMsb3~lr=#9(gGM{^9sr}|K@+Hu+F!-LI^%jb}KBm)CWs;7%%h=pKmXlzbMsKCE> zd*5z55ViKISCqlUUnw;P3p+2ea~dj&Zb;bDVyAJ?MX=UzqDilvhsoM$F7rc74t`K@ zye!dSfAPvOb;qS#OC7j!xBXw6+4%O_yGw8P?*86)f6ngjyGl20u0A(s=KC{eX3pID z!CvUVG7p`imnZ%SW)w7}EIGCL>DvSMFaG1mz4x%~lftCJzw%$1&cz)F_mmHaXMPhG z|3AU|>*w8arDo<=ET>N8hzaMqY;1n_{Qi9f(lh6aHNG!rl-#Ph{rX--QMcEh9e!WU zz1@CE-M_P^w=<=9Q`^5tF-gg(t@?f^ran_=JIAzkVy`U|W-pER6U6XzSViOKuHZy}LR zQIZk!rkWEokp-(4cpon@VV%BMf<8}fr}c^z$;57hiR9eI?`laW(q;>C;H zqBgUqUNpa_^M`HAkq5s5W?c^Yk@ukT*tNOyrX4*jyXn&R%`Xa^w?aB-}v5?ahM|#Z-6}zp=Jghi9&7F*Nqo00TIXNh1MtH09 zNIk}QE1iGGs=x2Q5V?K%R21@NrN7R3tD#$`;v1zCs+8k%Z5PX$tXpBx8)nNjT)xwk zkiziu`mKp?-?f=na0Y2?2<6?CdN)#8bhecAmfFN{wMTqC;(}Y8JwruzPP+H;)Ys4r zv*g&!Y+2T?=HA)7ezAqM__S$^n{}-xZqO>xeV29T@zZ-!IrD|CD__<&TfXLrV2m_N zm{wrv7mZcX<%=KvJDb2IsWUfWU+r~QR^Lsxen*_U*E-9(MT$-T*QC&_3ZL^@vJ(6G zcABz1UpYNs>vG;s-Lr{(p)=nv&+|?@`AI!pt@p1$*8WTP)LTOZuc@w13c6%n?ltSz zORHONs>8zeSu5A21(x!e%NQ9f`^PNq8S(wrLQqO;HEa&nXW`sO< zb8z>r4sTn3QsC8Ze}=0c*H|3Cd0JC+irbRar*7SHw6JyxU^N$Pk6tVs_rgm}S$Vz8 zDZ%HSk9FstnldNv#;yF9Q!X_+U20sIH_Puz<$_ETnYe_aQ^m|ycYkj4+aAuF@#W~k zK)WZ_C0?Fgx^h+N2RH-#6j%89wb_6Av@pgpW9R0N8?K-J(f?tKpQfzFt=~smrGrwu zYL>axcYpr;C|1UV-?XLr*W(YF(v~}yT?~=<`@C>&&Xb(G2kxkU+m>&fd#W$#W4FDj x@zc5Q{;KlVRLj1wXV`Em(#mJ`wvKiGS#Jh%dp+Ce5C$r{Jzf1=);T3K0RR$Lq_F@1 diff --git a/schainpy/gui/figures/stop.png b/schainpy/gui/figures/stop.png deleted file mode 100644 index 4ecbdf6edaf819a5f1f7d1a850f6fe0c0662d252..0000000000000000000000000000000000000000 GIT binary patch literal 0 Hc$@MzCV_|S* zE^l&Yo9;Xs0007CNklGm1hbzUmeXm>6QkBUhppZK=^s{?9(l^~-qlf+OOYUmNSztD z<2TJ-n9K6zNtai()#SE~3=E7+lu{pSDZS}aXWeW$n=FmZcuqy8G%+-A;y*c3iHe$r zOCGb7m5b+Qj1>}zv5}`v4CR8VD5$8q;!tHVDHn2=)esa8J@&+je>~AQ5G7Knm*mQY ztW+Whi2ii!*s*^-Yg!_LASjiRazFS>6p=`zD(a@yRipxl4g}?XQtl;s#tH>$>YCav zYHO+sN`=CqM7fuhyNRBGwj2$G#FRp6N<$FkM*ff}cf&ia_-NF<>V0h&3?!lmg5Vj2 zp)dTX-V1Z-!|^u<$=Z=iW~7=LE@*0KsGBk|@{2Fkh4=3K6e3o?nyjZb%xkErNI@Yt z_N&jG%eJ;R2oYh?X&sHg; zq@=(~U%$M(T(8_%FTW^V-_X+1Qs2Nx-^fT8s6w~6GOr}DLN~8i8Da>`9GBGMsb3~lr=#9(gGM{^9sr}|K@+Hu+F!-LI^%jb}KBm)DBlc$Sgh=pKpsBcD0sK~!} zv&~Pc@a5LMiBA%j3Mx(1%4-Qax=u<|D%M5os;{}2$V#zG0pgrOL4qf@DRf=xU7**a z7q#}NnyW9jrixjoZs&w!$1eOiTVix_&&40Mb9tn|0Vn3%q#mprw_~( z(q4am_rI*$&+B_?&1OxM)||LS|9kJ%MJJxDIx%zNls7+0o@wnm#y7QgYD}feY`>|? z)$L#Wt8+}-ET;V+zj1QFq7O?~R$o2sH_I=37xBuA_TU@2RXW#N;T)Ss|PPGf3(z-Px?eC|kubSt#m#Kb`&J?V% zsg?Yp`7-K@a;g6Q&quaxySw#?eus4*qvOYiTXoJYWqz}=?7_tgAM$M6n~G|b54^se zR8jq&*(1zwX~XSZtP&MDum6?rE@l_KanSiH`}eIXUoWKpO8PkC;4bwaNf(d5X-imR zR&_~iX4tGfrt;1r`64X`94?AKKIZe`&OA5vWNrPO8FHCcw*>C*zWgXkyp-8<@!DGn z)4KnfvT4d4^;~m6DC5CTgO%o=W0G$Q>YZ7)Ey+hiLsrgryvhLQ6YO|B4mv{7Ov)p{qrxUA}#SwM*!J5(? zjDeDCCb?*`o%NZ+$8xSYe{0O+?-N=TSRQo7#m~$t+RU={vTE?c_^QtzR(^Z&;DL}) z)bEvxOn08&<`$ZA`@l_~9(K-Eo8p=Sj_0QZFKXiIRnoRJG%r4=Bl&)JZCPg37VG9~ zR<8wvF5mulNSx7bb>izq`(B;uJ++l#^PWv|HxqA4@0@G+>`>6ZMQ1mbtz07w& zy;q+GU*0k!<;Pu^wAYvQzFJFrUAe!dGcT`n*Y~du>+@%ZHHA%bRsC`D z#nRSLE_S|)@*IQxs{dB${C&Rh z>%!FwWp@k9>TmicpEz^VBAh?X&sHg; zq@=(~U%$M(T(8_%FTW^V-_X+1Qs2Nx-^fT8s6w~6GOr}DLN~8i8Da>`9GBGMsb3~lr=#9(gGM{^9sr}|K@+Hu+F!-LI^%jb}KBm)CWoTrOph=pKp=>L3?P=Wt% z?&U@IY3nkr&79pKk*gxuz44faQmfYdq%&82%^f33@k?|ZZ7{@gROGCxiC zPxmF3-?;*j(_EMOo85Egp5!q#l>5ZP(wN@sFH3H*UGi8ic+5z9-i(hDLNA)lK09Yr z{mc7OVq$am?zV3a`%1U^O5btOwOOmNeBIgjll{JaY3Ju!)e6QxW)REYtk2Ay(YdRC zas7v!|5t9ld|cvuc7nThh;P@YlD!q3Zu)hO%;{D2_bgk)lW$L7T@Vnrq2Rv3J?UA$ z8m4_IJ;*h)_E|$@_{KG^e%y|2eEvKy{w3}aY;L~Hc`{R{qLih1`SSh^twPeGMn7*Y zcVJ&~n2TMByLrO)sl8L(S*?CPcfV3E^gCI{PR*#Y?wR%**7&t8!n?O`Jkeb@rCQc&n#bJ_dY2E%?>hgH z)zh-+-=VZUUv50Jy)E@7EwCV0o00A4lQ$fjw3?S(J`lyyE&qt!M(mGkj?caJw`>&ILmfv zQ;uKki3?`a_C^}E%4;y635?>r%_*NzedS)=&b`+Uwr6Demf78SktdRHa{I}h4Jvot z%!(!ynIA}7>2gSVwwA)acctfcO)}#TOS$&KY@zPf)u$6@OxgJ-CW=Yx;jB&_p3U_- zrEgeNgwCeRwpjf-#<h}jVJ|dK$(?v~ zL6RWzhUe3=uODKlUKqaaBi9-k@xm7i6MgqFUKMe9IU#2AY{R3*D>ePaEq_Pqc)zWV z5R?tmUHL)b;z`Dn>l`Hx@iR|u^JwM|;_>=5M|ZOhi)HrB`Lhc@h^4Ksu{Ybf;@P|0 zb$5kc7NsNwe8~T}cE=@T926T--|UC2KCu-fX0Of1csW33{c9 z#{0JaoTu9tBYAD}t2h36w1yDjt+dhMlfc>+>W>LY3Cj< zzjIYk`N-}F`wz+Qe>BaX$P_rMx_Ktc>X^!)QtvH`HlY8_0__%63hRIN&t z$oHB)sqN4c!|Zn{3ty&PdN^b0<*C;$-8>Six?sLPZ_s_tJz~kO$)Uy5uBpd*NqKc1 ziEvn@$)?H~Z*w=8H|lB5(wm>d`sI&qzO;1T&&S5IBLB_Z^h!;0zsJ6B6){VfKCgD_4<$U|rKpDr23?0jt%D=LIx+de)oN&M6phDZz)z4*}Q$iB}oVlrY diff --git a/schainpy/gui/figures/stop_red.png b/schainpy/gui/figures/stop_red.png deleted file mode 100644 index 0bda47e2902052c0dc258c3b0cc73b177ee5a2ed..0000000000000000000000000000000000000000 GIT binary patch literal 0 Hc$@h?X&sHg; zq@=(~U%$M(T(8_%FTW^V-_X+1Qs2Nx-^fT8s6w~6GOr}DLN~8i8Da>`9GBGMsb3~lr=#9(gGM{^9sr}|K@+Hu+F!-LI^%jb}KBm)DBkEe@ch=pKlXmmzQs7UPh zGiMi{-D0-U_mWCf>6U;A8e*)jEx#_hCFi(G-{N1WWEb=&U`oK3!;jke1z0~eF8Jac zc+%Zb#I;GoOf)*zAU7a5=*;08K9dZS(`J1+SDSO)&bZvj)75;A<@Yz6zuWzOv-!Fd z`)U^k=cyJlOLaf9UYNu%X`|+Mu`;ImW3LZ?shr3CYH`Hz_o*+Lh3-16n6zy2y0JzI9$kB{N|v3*ku-+1o2xV=38k^6(E z9}nwy{4rd*QHgK9@&gX-wsoQUpMKBivAZlikA?5(!CXsk)2+MaH;6yKU*(>X=^1O& zxky7tMCH@pM;uL0^Y1@jI8pA6r=9-G>2ZShwwUv+(@%N*hj~N3kFhSBuFFETum6QU z8gwmxYW4fc<$vED+%0$5UHQ;A*Lmsd>c9ih#xo6Oi~j$!h2`ZJRp+k@uSI?4+v>$* zXQh=MbX`7q?&NiD_ZjYN|M`&lZANZjvzB_|fdwb_{$pG8zVBen-qyX3Hm|(VDQJBv zwY+DJbCSRKwXZg8!QsmjW0nMERJ?ldHr%eTV7k_sm&>jM{{B@Xa4qzwso9%;V70GR#$QN$e*A?lJ=sb+;Q?pzDFQeM`%f0Z`Za<;A7 z_cpt0d7tHxxqp4fSF7uVvAW&KKW4mZWEbS)`S8_TT;3}_E`P~PKfi^q-to;a)!ApY zW8J-LbN)Yz)?u6<_4K*J-w)rS{WTT_bnH=@<6P8`CX!+_o$u9Q?(VGpJ$vHorYf3z zlV{929aA{xj`y@KmFeNp>sf;T{`1#7Tv-?ACinKF@i|$u=s2JMhHECwofkSy>tlYS zptEbp%)3d`w9koUZu!{asLp4*{LQxtnbYS~dc7vS51kg|q}gfI(^|PkW3BV)S$qDS zP`3SfYQfZ#(@w_uOEJ6)nl`KIb8g@p_Lq{J9?MUiwPL?~IY>fg%E$2dWMA`d)7Sgo zw>T+!AuzT4rG5YXlVL^LmD$^q;uF1GEOY8OkEkc)I$ztaD0e0svZ1r=$P? diff --git a/schainpy/gui/figures/stopred.png b/schainpy/gui/figures/stopred.png deleted file mode 100644 index 845e10b56b0782150701c1857dc236a95fcf4e44..0000000000000000000000000000000000000000 GIT binary patch literal 0 Hc$@!lvI6-E$sR$z z3=CCj3=9n|3=F@3LJcn%7)lKo7+xhXFj&oCU=S~uvn$XBD8X6a5n0T@z~m3YjNd0( zOkiMOjLZy)D2ed(u}aR*)k{ptPfFFR$Sq(10hlpi<;HsXMd|v6mX?DWr0I{efGYw%=UNO{LKo9DJWb_U74D{hTKN}crUGpSnQv!k zqmLm5bBjKjV<0}&hkDhH%SIm_Y<65ehtwm1k+#m$#WBP}us8H?_mohPx_kYR*CW&K zh@PIQqskv=!pAH_Fy6^UVx%c}R=Tu0Z=-@Yu`StlY-|y0tK(pJ8FV-K( zetEb09Iw3XvMWpLl8+th4LlXGGNeaR)^zGUJL4OrDjxg)9eY`PVB1@V*sJjchZ?dM z^lbWQb7A7|4w@xU&+;UQ`#{PW5tz8XljMKNY$ougz7uPfM z@s-!5ov~0hS*D|~D=1KgX;G=7GEX^q2d;>nbQcFh1lFzjb2U z0#&P!_kM5hDV3>CK0o`+Kj!~m7@}FPOzs!SI{D64UZ?W+*`*#TOnN^9PK7>x9Pm?c z#!Al@osZTSGKOkwC>Oc2%ZgnuTda50k43Hcn}4l)TzTQBXGm4mf_;{JI?+tQNvGW;+*@_y%=urff4$~Z%nV(9 zE~cJq*Vb=*y7AY=uhQ`!B$JzJZ@sabFeQV>d2=p%|8s+&H4$9jJH_68$h;zGp|IhK zd+38>$%)US{F9^4c&o%bT3ePWR4(*g*P+L4GtVlkIS=@*GWgjVO2z#1@``9P%1?L` zy=fVb#+$p9yz9Ov7`^nKbl1b>k^-A?r--(@-h=7_71pmYo#!^PHW(GyeEhTC!lmT3 zQlG!d;?s@OxDJ0l6rBFXGPK8Qo&DOxj;*G``KxH!S9Ozw*ukzI8Enueraj zRrkJImFbOmlE#eJuWe z#w?4=f1B4H;0=m@bLHf`Ba^k~yJhX1k*RNYfA02T^)q4n0u&EREqLSATRB0x_twFg z%UT!z{vG(^<#XdB)~RQoC|{XgyFS^4|4)i8v3uCHNm9zp{MNU-D(Bp|vo!mn ztWP_Z-BQ<4ZE@OoX6e*TZ_Ezo94a_e_GJ4iUss3Cvh7_<&XseB+8H=L4}G(wyGHBE z^t<&5H@@&sFgf~NPO1IR&Fs8wd#i&T>=~b(IVOBJbA1!{{BIL}{faK1EAi(~cc-*Ik{rx=l$$E&-ruzy}hkIe(yfsb#1}A8xsyEM%}q3|58w4 zCX?CmYx!08yqAVmpL7X|o1!^2XFl7@)9KymyXyDw%~^6@mvO>u;S2jd*E6QuFeX3n RG;RY`0-mmZF6*2UngHrL<6Qs% diff --git a/schainpy/gui/figures/tools.py b/schainpy/gui/figures/tools.py deleted file mode 100644 index 0b5e891..0000000 --- a/schainpy/gui/figures/tools.py +++ /dev/null @@ -1,15 +0,0 @@ -import os, sys - -def get_path (): - """Borrowed from wxglade.py""" - try: - root = __file__ - if os.path.islink (root): - root = os.path.realpath (root) - - return os.path.dirname (os.path.abspath (root)) - except: - print "I'm sorry, but something is wrong." - print "There is no __file__ variable. Please contact the author." - return "" - \ No newline at end of file diff --git a/schainpy/gui/figures/w.jpg b/schainpy/gui/figures/w.jpg deleted file mode 100644 index aa878552f0e19afc59e86847265981d09d4c34e2..0000000000000000000000000000000000000000 GIT binary patch literal 0 Hc$@W+$BJ8w_rgg7A(O-a0?#X-Tjia z*V;Sly>rh!=f3xS_s87@^)W_wb#+yBj~dlggNGjvKY^zTGV(G23=9CkJpKR=OMsQM zhqVO&$jdVV$N&Jq0oX8bzqEmC{N3;%s|D){%@DEV8vUhPefAfZdot5M9 z7k~v|vr+z_(1CsA^XGLa%s;3fmQd!cTwELk+1TuySznvko0zkj+S{^uymnyYU}a|m zgvC4@UYml=T_{YQ+U|Ev2zym5TX2ox!~jVuVyw%ia$tPz#^17N~#o+_D<#$+^pQJ z>@1IpyKz!J@^CV<5LA+qu(5NpadNUeQm{CC z+PS>;V6k(i`ip~C$y`TorWzk2?#!C#~< z))xO0(qBD)lKvVcsA}zD{zga2+Sc68`EgzmN?vyM-@9~w>*5mNq@?)8Kv3THwWYbX zwV8{R2qnkwZF1(;mR2tRAZ2@0i|tR#{JQ+7p+zbF4gZFJ!@uF*@Nf7x{ND%v3f0W* z9^*6j$3X017LWvx5fPCP5s;CPkWf&NQPFWR(9zJ)39)gW;F1!NlaUgUkWf&w(os+{ zQ<0F+^D!{9b8vHWlhX-^@N){Ya&dG1VgiGLf`X2QPJn?y!1H9@+scWSCG` z9XJ?D02T`d4h!a?6Ci(#XAxljfIq`m00tHg9sv;v83h&XkpTJ>fQ5mBgN27fK!AsT z?Dcu<2jH;~u%B~CAmV^tBT+iya{9;RAXB}l`hur6a!Ac(;uL^_`V60dkcftsj-G*$ zn}?T=UqDb&N?Jx%PF_J>LsLszM_13(%-q7#%G$=+#nsK-!_(_sU{G*KXjpiB!u!Oe zp>_2QjZI%WI=i}idi%bOj*U-DPW_mkSz2CMU0dJS+}b`m zJ~=%*zqq`*{-qZT00;9I{D*pBJ?aGu4-W^A^h+-oSodE9SnvqXIS{cWKuE70aVR!b$@V}+mKNb6zUNZnX9L(e5!C?Vn zz=ep`Q5kn>X|3u2`pYH#oc6Nb{bogGdCD&ZuivwOk9WUiTnrOi)f;cynSTHhRV*I> zGOY*TYygr7SY9pAJ^-0SDtC7Y4?tMn&=KG5;V9%_H)~0+C!FggdX`w{;z@`61JG6t z31Z7ww!d1+e*hMz#V%7P^0;9VHaybqJ*>qp&QM>BX0Bg&@z&QW*~2Qq{~cYoja0CY~aS`7eZo~?hAZ{YVC*55me6WrT}wQU4H zc0$Y^fMu(bw@MFyFn89?4c!BPqmLW7>`!p-{&$k*zmW(O3;?G8$uw`&*8Mn|7jYhd ze~LD;ewp=8t?R709;L8+oFe)+DO7%+0vH`e1xJU5k$b{&TzV{W9{_pW`&e_Rk2H$E+ILU(6f>=BQ2|%nl!9!DxGlvlTGu3$>&e>=U5s7$9G4$ZdJ3^RsEmo&%+<4ZP=op?wBIk`o|%eckCik$+7!tx+mtfx zah=A4Bf}A3QR!DEBTvN9qaHPq(5pjXLKf2?F+A`t5o@4KUi z1u4q^S{%1xrE%Czeb0!E_pCUr!;YyyYB&ht4niywXfawHztZ0btG zm-+#S1FJm%mG1W%H!7FM@9rBbkF!xuQbBTpqhn8hEwtKtTBfxXjP>IjxFGSz$&gxb z$*h118R8o-8x~m1sxIl(=yOE=`js{EMA!LJs+Tnl^^;KD)yOh+xkBEd1pr2zFv%`; zuni0;(&K)1h^Sk@i0=yt!OQb>E=em%q^+y2iy=1qV8==R-B`ov$K2NS@q&(&;mpcf zQ|ME6NGqpJp3e8wuGJ|$(7139A_^LVhJni$VO4|(`=@QUR2S}J`SvWeQTY2q{m%)$ zkj+hyy5GEZX_z@HUtpk(@aoPdi2f-na&3|i9o}CZJ`-YC(J@bQW|o$l@iHZwQ$M4f zbXA0h-uHV{hNAePoCtnS7$?t!=j25zI6)7BV^l9TmD)eD2XgRO^*hRAzR~J6V zfh9|s#A(_nQln)!=yt^HE^2hm59LVe-N{T;?zvHyfqE01o;ZCRSp61HZ-5g^Oc$w& zSeH;7rLG*lS(LQFxOF#*iCosW{7SuR(V_unZ;z!jg4kzt^^B^;%F63)s$B{R_9w1q zmI)+LovY#R)T^2z)FL#;5)I8INj_Eed}tgp(gR+3E7fPoXWT;Njb9x0=AlO-$~;-p z#h=pF26;HG<7zj?#=|*q)t8h`Ic?8$Vwk0R>q^u0Gmkd#8~%h@gbHbl?JwCX6^XtXL24Btfo_-_)JsBQiBz@n*@xj6SKlWa09R z`Y6k$2A!`d6AJcpw3_^+$MiNZ zmF9*Tctza6r`xhrnVez;DyCM)HjWCZEG<#Z#z~e@XjSJu3V^=;ovG^hzNz1=ti7#+ zsfB|g`zd*lZuO-8F5iI{9&NR|F6<#PNy)9Wm`rWvjIDg(ivVzvivSYEq;FN;D}(K= zbJqtz>N9e^2Zb?1?`;ox5TK9WRJ!#nJ|BdZ7X4~5Fi37t?)D?GlGYYa@##)P8n_^Z zpe#13KyFt?uIMs&aCvRlP{^$j8rH957ghVI2Wbw|<73jb&(Qf@)SRH6?o4+@kVdhw z+;%(UT|F#&5X1&E> z(o5s|t(D%@xoyIwvRF(FrVwQhihzmc8tP($&nvM=1bASX zaf?>{4|b*+P0)n?_Y7AmIplhIxbD|HcU!?-2P_q4C-1x4@!l8*JQ>z3$NaWFU&(ra zr?tFq#NLhRhHm`in;*XBIBQ-jbgT^CWc?>sa;!N>Rz3PG&B9*0J3P6Y&%D~UT85p( zj#SoE+tpjY;Y?9UZIE1KNh?}T(zrTH#lZ$m=M1e9J7On-x4!OKId-sv98HJ;c{w1H zI&hwAkvh5GJwq@b&%YsH&C#tQtkoS;f}yPxzGKlXpk=Uuj){rkJZx=Wcr|&v$zc?= zJ3NSoDIw5dnaPwhvR42d%Zv8N4mT`|Wu1UO&qh1_>n+-tpBvki8=v$K53Hx42hit+ z|1Yt#_0i*gehfe=9%GB$TF~F$pxG$5k6o0L1rLDX!EMR;UvJ2lM|8IlddYX*ZO273 zke+n%KTbwU3iP{iHm9-S*{773PW)S; zu~oN0O45MS@vX$+S#u@&6lV`32`4u*ve-+lyNqq47c|iVaZeRB^jIg2BJB=(xJL9tcoOAWR^MQITJBe8TolqS&`@OjVPRzeV~K1jF|O& zpgg+3rEMX2?&m#i?%_tDyY0lS#C_UFrIR?e?^Rxa2>6qx27xB35Wu0wgS^!c{X!+Q zj{WVFpkaw!%Z%dNCqq1TV?FmIE#?|+$lo2FNi#aXB9@6q=L{wbZcxGZ5d30MGf`GP zibB;CF{smxH{JlhGWk*To1O^l9_e8ZDCTQSImJ)Yv2X*eXM`^v0Nt9g$~Wb9)J!z1 z!Tk3PLt@#bXUh-3Q?~sBVOg5_>iTJyX>1sZP3pCnZ83$jC^U5?Q>ofE$vAG~`Wp8; zXMK&le&qfJIyWm&Dt(y@Xo3g)ij2VGvq=eaKWdhY{R03h{TaZ&2_C#>RjHBs@8xx7!|cEe*j$B58{UJDWNW@`5!6m%cG0R*~p`K-)l+YQ)prc zX+tMyniVaWZ7tXv_ZqysAwi%cx4Nb$A$E4h@SUCx@8*Y8j&7NvE+w7Jl+@@GsAN=_$6b(o zQSizUX1vCMY3QDCQ6=9a^+2)(fy~$;(n~H+ta#(FJV!P)E%~*<*|y>P_gYk(JVw|* zHmCXe9Z5{f0-iBlg%89mlZUL+^<{c;O!e2h^{g^(M5)UEoL9xln!~wrHfZ~btDa+o zoxQw5DmIPR{Q8N-`ZenQxgc`0&(bu_apvXMI}YwQS>Ls#(T@x|t2-$Bh|xNQ`8(UKaCOu?-b3VD<&LlA{Zy0N;aA-FB8k@S+B|Ef|4xx;f^XSj6(J3R z+Jh~=191GeJ4jNWnveEICVApJ6tpR4zGLH6&0RkL*IizC(1Mn9MZ9LLE}}?;C;;xb zoCbN!EvXEr@mpIblMA5_jt=N<#lHQ{dh!9lZWa)-1*cHh1u9|@R5bg^@K?BZwmuzm zy%B($$$b?hYOu-Pe%`k)TD!c(XH79aQ`J^CUXHyzYMz9Gs5zChK|WfpE%MU?qyAm( zrrZ}PW7*b~R~Y8h3D&R-+g^c0S{Wi|{dmC$bDa>jEj7JY(%#k>-!YmPGz7C$r5zS= z_BR@8pu@ZG`$YP<1liO=xo*25sP13OP517}ya~ytQ0$rHi55Z67}>mLW3 zx#69!AER2B{|O?}T--*@sN6YZJpksp5eHYqwOQ9oV)y;hfXcGU)iWsD^#x>2<=0V( z{~t#s%ipp@@>XKQKOY?___i33<~8>GHzJFn`ADpxjoVTM)s55_-YEGV zE{nIkO}gDXlLc*yZRoRtzKWd^_RY@_s?R@iqWb9ylKoLIZFL|A8iqdsK!l&MJ8 zu!p`i8eQ61olSUj=@UlbO@v7^7#4UP9ieMf6?2CW;d|G4rBa!+={zy6(ak}ov_s`) zo&!utrq-QMh$~-(ETq#}kL>wG-Uv6-=ZkJr$tp;5kDr*}ii3?o!Kr|cyx0=U(KqV{ zz*}WBwvR1O!RtxKEf{V|BbXe=#;vE>GA0jlwo`W~lR%r2!+b%JHap2RI53_tOtd6S zLO_TK{i#_P| z*vLy-@NopUqFISLN9T=8d)^&t(zYd%G~=u^VGd5-SYr(pKb&RqO>KFf?|P9&&ACsq zzmjq$=_Q3=X)*aAIrn{SpTpMX%%PzT>i%EKm+%>} z6&so|I}GIuemnryx2DaR&p+4}aDK-0^8?5;?7)lIrek|`l7&W+&J=gk%_04ugQ?Vo z!=y0Vye z1+)w`_@>wt3j#WH73Iti72sv9oc!(NI=$#bACj74j^3wFJ=RpHcCNKux#wlktR_OY zoM);rSWG>Pv|rhGOloHb91vie)xn+wuZ>&)~sY1UpE z7=1#DXc{9z4JT}t2`La3yqmj&)z6!a@`#Dtvpigk?#@HKt>$kGJr~VwsV_J6Nb5*i zpdKA4djOatl7c}K#J5b$BAUv+=R=Nt_%7uN7)CDxuBYG0G6r9htsIsnuIQQXHy-YQ zyuEj{lU3rX6^_gAM>BhVOyyhqvJanGC6*s7YafslnLcSBE0yli+4kphi^~8h3L${{ zOg!-5Y@S(t&fI<~^77p+IQHmC1hahwiwg^W0yS57M~bP2OZ;OAA9n)(i&;@Zhl`T|aiJuV$u+1h8&o zTJN2=MNSP&?Vfk~hKrrs?4J=6xUz_PGfcUtKbIi~%`_i;!RJ`Gm7Us8^q;GEx>3A< zMAm}fWs7$je=J8;wPJ|blJ-@UH%qShjkJ&_vkIJ8`{KrA)4Xfj64g;bQeYcTMjSq$ z$EJ#=`n9iX;*t6bQxc-JkFiHsBwb$qw=7pjS6UYxcZ^_LJgCqv^gMgPxyE(&hp)js z4^&J157VXAAqp3 z!=^^rzGywNQKt8oVIMPuzSc-%bzTHYkO`hlovs{Jq@sn|QRwEx8S)apWBB>b(6(%` zW7#OI$?!uw)68@g+YtKCXH=)Kxq)m#ElJsf9UZNt)nm12OxCs(o?7g$GTG%#`^x!7 zT{dL2s_MSLlh3nVYJTZikv3bOZP@jw2%_@fTb4>vcAISuhRfQoXM}(Z$1%a*d|9@n zX=^4aiRH;&MvBNXLe#HN_k@X;>6;~TB-lP+jN4$l>=U@(dQEA%4k1rS6dyx=@ z4~2)vzOT=tJGw-P>W^q+&nts6e67sHzAi2q;QQUGIOf`@EDe})9~wL(JX|QA1yj%AzTN4{7rPn*Ff9SknN)fSpSf<=B_*+F||+GosHjUzv1+v zV)4agM4n9*rNLkwpHUad25?7QboQ}a<{&e(B%G~rTGazjmj{hcsox#$35cRh#Jjsm zCIU=F+`HcRWg1@A8RsB6E6TYWJE2;7ZZlpw7CSN@Ru)P4UHHl4!zVX06k2PKA>~v_ zm*s$TAuDl(c*m4`xVt>`mABr#0xOftm5C1^L~(uqaP~fv@=ii~1b*Z^M=HjM z6_dUL>wz4bZ?LB~8S=Nb!$mu6+aG{T#e2NOWij;WZ7+v@hJ$1#DajnUpM&(tIy0R5+;dBNo0T8wy>?y0m7A)WpZ*uAh-d{iV?I|ARr zmNb>t`th=r&(K7(VCiqwsheH0nxHpbWL+QWa~l=@_tUz?kN3awx;|OAp8rVestf-u ziC}q$`s8}O?VBGRskJqdp265tkc>1TF3@_a1`Bb0i>+8%9 zySV7B0{a#!uhgk>CJpuYx?Xvma_^#98xBux5DL}C8g=-06(kQILOl`2c|uS0(sx@O?%j z#3y?;Vz+w2>T+3V*OMe^J+0 z$a%^b39!@WCfM$G%ZfDfsS4wCS4*v+M{**UU}U%PA#G9B4e{6UztR9*N_HGJ^*u|ri)ZL{B@oLnNCv$Y>aw?G-XmXR@du3a0gALR^NNKhN0{`L08iCCKFOiCF6CI%Ml)UP z%SLnpL;aE08V#RFi>xr%G|L^=AKE!Vu0!oGefM|Qf!N)(XM+E?=_9Lm)_Ww38{l3h>yckk7U4ZGURLAg}jRy><9>{$)(vOj$d?z8cp9)e8BlhgX|v zM&zPe|MHQAOqTPy(RS8r>}H2f|J5H3I++0}u00~5&?-RIsnO8VUg@)xK@9rP z?^>uY|9ty%&HPg*vrpYy!Hz-^~TU5nKfd2c!F zyr#Bodj-_5ev#>?PeGO^P{#BCAX^b7BsY#xuzFQT6`hu&`G_kzt}3C&Dn)%ULZbQ+ z9xP&@PMg^vKhlU9TdrXX};tU**v60O!v-V|>XtQcdpg*j5FC@lzo+8ZWDrCbz!@5N? z7@5S-Hz#pC>r&sc^Y?HBG(G@CRfsw3gSG|r@q_^fsGU#6Pbo{S-D~7|4l<*$XvDal zzCg!h@ScQrNE$mZfSGR0U;28~ewekqb3^3ViEb#VfXbCOXC3b~ae?de^W zL($)1TWW3QNU%E61+m)?=gZz?~tvn-60vc3s!QxDYtm&3}O8!Ptq zv7&KIP`EH`#}xkt4?I@$4Bnh$-ZTAx8{&9zijymC`TF#Wv)n9x02W0nfe1nESML;*XmnDSR6OYs<- zH1BOq6m3u8PfHu%j}fhV{3YGtD1T;%Y4SxzPM%Q66Ur#WEpEwsmPmWrP1Z>bFz4BS z!^HJU&?hm8r6&alPjO!%cQ$kUFtASZXLw2x%jF*prBC6(utBk{J`#^=66dFmG+lrm z(xU?HfG@5y%z(oA2ugmyI#|e0pFLBOyr@O#EzRqq`MH$ms`&~d4yswJOc0Z#J~+{a zB4hXUXk+$rgyxxiC5*0#z9@N4u7J|adB(xWHkRpbl!iBxA5@XxaI=looJQN7ryLbh z1M9E?s9Ic|WDg$z$dzWfnniumYjzpcCRjjoX*>r|O;bC$a2`Yg%U^diFsNJ8B zRHd*o8!>tRwlCJVp?p4XEKAQoZ5@tJCo1UUD~h|dnlMD6=#}F<8FgyT1%9ClgNC6C zFSadpqnROB@_lMvA*NQpF#?kVR%)jh! z2``qSThA|lyx~>hbF*IVC>K8e=83O83C@UX6(nO;4`&H7jCmB7x__!m`+gp+(ZFiZCq`Lc2KzLk9j2|@C zBHJfG>qm>j>N%iLQ9eAOC+Nh90qE!E=zbPkX9Vrc_@gPxEBsmH$9}`GtRddH@3d%USTy=jU5!$hc832FQ!}6~b-Vgxg#@r?ckR$Wx=g zY>pduEp;5g{rJ);&wt7kF6Nv6liboih$VWfL9zqY1K^qu+)IsbOQM(bo;^AewY>oX z(O^I9EG1O0<&3dZfT1v6H#dNax|-SwQBlYk=8OlgWexgc-qsPfe+MV{Uc>K$+Q_>t zIf>Ik*!6Iw!ZhI#{TE{@l+xH|?#9`^w0`7rHX-NxagPTFeKN?_l2~_UYF!39Ez^xQ z@G~(*S&T})ww_37*MO8>#3pp|G#%2bJaqp6R9=qBe9p_d3jgr{{Cok4LS~s8!3`-; zy`LX>JmoVzp0xZ??zu_b<$7*|-arm%#JWG=cM-1FJOK6FG1sVnj?%0DT-OvSZ!fG? z=-5cYe%nPmI)~lbd6k>({n_I~S%OTyD|)+0BI{oIl4ax_f!aol<8i+?4oH=FM_Nd4 z{b^5CteVJOx_ql*F$n31%TG*DU};W(=0`Ps*#oe*M}q09PkxK6a+D385fc2kUH5X= z;Uk@z{OOK{;_&dnp5|H_^+%Ut>d)9Nj`ibYwc*p768ZbYyTa(GsxbN5YJ$V0?xqf+ z)4AVGYr3sf=(r=xYY!cj7Wh{|#w3{Zo84*aK1ewtTeoquW8nZy7wL)oyLi~s(}EQt z6{OJ*XbH-DFF6@60L;d`?=LtBNKV%?{5%SZsT%`eOPPgC{^WfX;Gd!M zJXo5(LierXQZ^<}KQm@qa|uq*0{%hyw9ngbusw?5OlhKGhq3&EccG%k;ZHiBpgg(y zzzjLqgRyXVuPyKZpj-rYnm?||t{Z02Kw(U%D3*mL8ATQIJdj)tTO+8aw^4M|6SSwP z=i~H^$1W&;M@@1?4AVf|GDNr5BScG=zocPzV1_9L66*Ujs;({c4PFa+Zu0#z{6M7J z?^@&+r|NJ8xX|xKonpvxv+A8%=sxh!Jn}GZYts{|abvUlLpyr3)LJd=Tk2H4J9yg% z;GjutiR#xDqOyMgd?uV!s)R@OZANqZ?qKlKt zCuiBlQfYI@SJ}J=U|44H{wD7pTZH&NfcF8om?uft6*Zrm$gNfha{*1M{SskEQ_dXT z34=Cu9@cu71s5b|@n0%6NdMErf_@7vY#{%vGw2OXeP9dW;WhPRp`pHh8(fQF>EUPD zAz0jV+c@y3F%en&+b?jRUuxoipJ$F{7Z5NVu+iTZObTPxPIWn+UX;F@gQ2TolOZ_6 zz2j1`7#$dve+sOH0xRfFN>Z<_bNG1k=R$hTnfft0&3!KKm@;?AcT~Dv?uVX&4e{$6 zQ<8~?V^~$b1cRA9en2}^OqX28@;}(hv+ywFw4cq08htlu?Bni3mYItttt00+Pe1Ad z4Lqf?zR^t8k6-E(_uWO%nvoDr2oThC=Fr*QyJxyAmCq#D55%uK=Ib5J4da-5+px70 znOe2H7^*kFcK?nw3xZIPH#i-?U})e!DrHRQIA@CY&7hB({mJvBXR}GiQ|n@fR0GaL zWziQ&*g|riDolKfD2TkkbF#El=h8$rnw+E`Jsh~<8AWT)_C^=G7f~(h(qWG6 zb9Oc_k&^pS_dF*Z-97w;_qB*xJYI~dn1UGU$;D+&_J^r&;b$8MD4{YPM^5->JpJHB z{IdwKu0{3Ob7+|M>Ww@b6$E;`gfvDQZ9eQKl9I2v8Vl@$-(A*FyNhA)1Vv50q&$p7 zx!JkWPPP%HndB*z3oa=W`np}s&CZT0K?no^31e{lg@)o)n+aE;uzu&y`Xi>y92Kf}!^cdU(&H{ZD)JiD_5QZy8eGD4 zf#M$BIHW3t&Ig9XB`SDl1LaZgtC*#~UV*V9nQ$T#E~hMW4LYpR!+8j?jhKV^VElTr z&3oyDH)a}+Y}CTR`y!FFWF&)pvn9#Pv-xAsju%y0p*kB>YjSwH$#4YiE2owG3B{rl zrOSk)K3bl5E4BlZv~z92;ylsHN^Yu!O9(!L=&tr)%~>CkJ!FI#+k%C;uSuqTmDLxN;IGf8rBtLhz?)*u zGcAuzKR@(M<8aQjeyrS6%`htC?MP&!Wo2T#k*BQk7{+8K59tdtzeddXS5-k+DpW}R z27H~QGJ~TtiUjsc%^w~BTVn1elp7WO6FGFCZUACUZ`2N2X1X$hC36pw_wb5x~Xt;B6db zrK4WMNsZ6UE)7H~vdl8~j?{G_I5@|8!aJt!bWXZl*-oT;C|H;9R+`!@c1_kziK;e_ zDWE&ikdSygs#Qjp=Otg%hY7MeQH4;EWe4r;7`zcBuV$99FmOkD3_7$;4)?+s)>Rz7 z2%?`>I!||$tU8|yeYkbbq4d2sSqfXnGlAgUEJh=GYZf<~E~!&;>&0S|FL_W89anIQ zVQfi@)0!Za@Z^D)S};BZBe?D=eiF+B^C@RJyV=lgCHJg zMzb9vAI$6#ZO(Dx0pW$$Kv@}e?SyS^jBxy7v%OJrxn^1?15qtf=TkpfbYf!PG8t)? z0lZ{_Xq@OQ`r=e#oLJkv&XCNVVFELiaNrQ49Qv;bx(VoHAU;-sdB52m-vV+Y%A9VcQ(JkT;G(>B1 zAbjnU&XGMM{pRQ|!JxnNe|> zlA@}Qi3ZkJT#B!EjBV37fpE-0X-(y_G%FUYKC*^FHtu`Osv!S$+6(x%7If@W@&3UY0TyN1Zim4d;WK z&En!cD_`HdS{uw^$+t%5nUI?;b*7`=6Z>OEQo;Lu7TRx8ZK>Qc##^7AI?=M0ZWeNGI=Z5Iul8Y$$UgUV6^vAR6Y2i;r>z1o z&DDrLM`}B0z-LZq*#OywC6)o8BOJKq4mUSyP!9cF$F2u@fMMKo7#SJCbRugv znp8e}M`Y{qoabNB6{<^b%y6cLPt>zh3fTJ#4ZC5!$H~L-cJvs;D3q6ZW|?Ts##MG5#VzARf3)BXGlre=;? z`PaKJw~YDzGix)}dr2%`Ik8Y?=6k6CALnq_#q;RQuMD~Pyg~A2af!+(eKQUYzO@6& zisr2vkg9wPqP{GlH*xWTq>USksao`L&4o>Jk_O+S2G61tDhYcKY7RgPfr7lbxxi*s z#wc@iJ6ZNq0Jd1x5zI3Hjm{AHhO$P#WryA^u1!d&(I=W%<>_o`pKBz&&TMtq8$0`z z{b^iGT0%lxfR9bN1ykxXE(1O5RD%}5T+M03~6lg+KW2hT6W2m1PU=}thzEWP8Tm~D}d2;A?ou_2P z#vhf;h*Ph$qV#Cpf(aJ}yK7$+W~HpVUkmpa_d9x+E}Ji|k$!|Z$g13!`BZVh`)zq* zHJl`)v`ttWw9EolYMWS35}>}}yxWX_(QPk1BoPe`P3hqHf&4Nf`&#RLC4CW1$w*{s zWVuT5qyXU%_csPd{Y;0%f_#cB^#IaNLK4c@A76ywPRn*p->FG;nada=Q;fzpRk_R; zrkvgl1#gUBbbNo?==W3Tf(s_GW2JFwZY;$noLNalOlu zNVlBgS&;P|@c^qdXKLIJ-v|mv<5QQwIS}r0iZKrG)~Am1RrQw`=t$FFu~LMR6czdi zY>I%tI7bP1^Gsdemw7*RB{4VvKU*HA(VwWd)pv1>I_|Q5Q}vOoT@LA`W#mSIkbx-K zsLLfY4~E|L1hr3lXIF;f7(cwf4qsDxCc@#V!~Cn}J}X|{eKL39HTj{+p9s!a6>Ue9 zlh)kDI7%LgI-WIhLr$E&JEkP=*WoV7wRQ@Ewr&iH`h%|o@oquO2dL9J^Ad%#tR+5N zYZW}m?-GU<2!Zfpp_ITeVSg+>l06AO(ZrJC&3fKU$}GjtsoWWK$E$IJxFlzEV=dC< zn?+L1Hv%oSpI3zT^5@zzyiRjaz358$nZ`3@4yx}fK-D#ktxbwx9YgC?Ax$hNog3{O z54*9hrdQWn$&<+-{Z9BOyzjv@nmY}aC4^f#qDT|mEnFI&_DjrPxX}I5OIN*odA6KA zHe{QdGKXhcpkL`VlGfF!`ZC*Lf_8MK(yzvG?V>)N#ux3e&DDUZUOPQeKJ(?Jt$N)k zZKB&ls%*2$cfKy(>ST6~$z>!lrv4Q=T@fuo*jrjxy|hC=|MfGP60Y~vt5N#-)4o-W zN%zSq?B3V~3FO$)adJ8AQc2O{8q^K*BpPKF!?ZP*n+mT6t^>*{aN8{9SbQ} z4OQ>eObBZ8x%+0|UHT_DIxyUombpaa_PT)1$P1yuiwN z$r)p?Qb1jUzCmzn(>%6Oa^*eYLVesHCHmD#Z`o3It}Co>L$vmQ^(%jZ`7CnJ%St^S z!U{HX^Wceg3!1P^$1RYNlA}oA(}MhP)qUIkZlBV`!v0wcxseW`kJ1j3SUvrdrE|J| zN5U7+J$vC@%T@MpAg<;_v$!0Qy$Uezq61362vm)J3T5A5$mtJs%Si)QugaYT!E$JJ zH+u0tHZ?Qd$dn#KE1=@YIedpNm~I)3#YuS91Xbn)SH6bOk7vY6XEb2E?Am}`!i%Mn(Z{hpOouZ7i-Xn2Z46=RsL&1cca9gh|fCV}KfGC38J?NMotmSoP{R^VJ0k=9MNaWyW( zl#2{Uls89TZzXRn*tuL!=7?_wzG~^D zTWHFgw#XGJrn2^~T~ChhQxA>bGPJN|6>Uk-L8D^sNDfb8OVeH+W!l9^dZ z%JRC#dH`T6Wm5?C)um(zFtW!|VG$${?$WDDdt9;9hxc(W&xi$9IeOgD*!d~=2Skn6 zo;SwocdBTM0PCgGS8;WAC|B2H9Wzk#06rkDcM~kQS!?D8gK?2uA+l`%Z5Ri6_XCat zujgOMo=-pLyNV?m3%1OSDtud}yb0Cnw>=x$i79{i z(T$P&CVAgJCm1n>nkFqM5)=7s0BdgW1kK}>8ha73YP{U)$yl3Bo4q*z#mAPae)9zs ziILZM_VF0t!9?ANW#@WVPYq!Vo-U($W({piY8X;cPdyK9o1h!kj_gI`UvEieJCi@f zi)LATpQ3NVr4fXA%putER10=4~D*+A=A=C&<3ykuF7z{%@avR1gpEq1cn+P}7yG;?rnA+#RqqwC z<;mE4%hS=*H~9Ayiw*pjiPcklUkUQVhRK-=0I*0)J#QzN`Yh^kdKca}9n`%IqRPrT z-0KqStbZz5*7TKn@mxn&UMVuaDO1)odcGu^wm~+2f0S0IpSS|+y&P{Mbu2kWvsOL~?$-)U#e+OB*Ij@O8BQB*8GOHzGj8;U=l*Gu= zv4$&Iy`(kBNI4Bm$jm7rDrc)aBP^MSsJeD>__fm7+JfJkc>U0L7ZoGzR1J=ST9o%6{ty^ z9&P4V2gf-DRYw^N8cyk(p43!=i*I-mbi=5AuH_iCn=>|UHOz_(@`$E(QA3hv`3ELj zvndAGqRX21LnA*=P=8Rf*SydOOT&5x4S5sk3$14hqpYt5mpp%p2;W5%>z;0$z5z#K zDZ&_B@l1ZtPPmG4R=NrICZ~^%G+K_Jr{al(Zqno@Bb)2Y;28rIH|`@xAmv9#GF^aR zq?yabkj){UV02!%7g=Wpk20%l!dr!@WX;Lt;Dw_K<9cNsIo-FTzHi7vNYKMKPwnw% zFCp#7llVAgdkk_N#mv+&@@PMbdxfu$49YJ8pagHtc_Z*X8xZWP;ZWwt-J5X4JJ&kL zd}|?AJaiG%YFWBN1I07-VJk-D*F{DpZ9871mBz{z35TB+DQDG0QVp*Jt_?7VuzI@X zK%6Ttla?O<{Q%j9WX68G9MQM0Web_IKG!xtZR?*lOGWz4X;-|}I-*b0Ydl;?wgsae zL>ed*3-=iXAbP_XIzIr&`<3oR+j=qYkEM$mqSqqK->3Wc6yc4b2+^^i+ekZYIT^uh zCJP>f@3H*+T4p(z^umEkdNuvtD6E|Hb!~&_b14?82SBF28N_5@e*0aFSyl~s8yG5Z zBZE7vt65SMdH}F|LiX^V6g_&V_CPM|HDw*Q_6<_U)RQ|(20LNrBA@DKKG*MGxSIE_ zOEi!zK-u4(>8nWbd5o^@d+lXKa4Ygy#K?wN!OJ`!%_X>?$?v_DHhcgAGxyCrZK-c4 z9n@*@L%lt1(qev2Y(BM~RIOvFP2<0VV;vR)IP^Hmsut@5KoP)RT~M09vmr7?;P^U8n5KXY22boh;C+fF=8q$Z?ugkjxCj%Lpp7<_QdSNlay( z{)N(9JwjnA$-6h_k^pQ@zOa{*&2B^qziOpjplKqDp}X^nl7X@UPpxcSuLQNPz`e_l zw?@lP*khnwLt?^`;cn&)Tw@&y;gWeAaFJxtmveK90+Cxz9}Y2(RaY;5LLIR#`#V3T z%)VwH((?Zz7Wzgr0urj;J#;=#Sk2C-FaMc3O9l?^h+8)zG3|I}>tqny2Tz?&)G`Sd zbJ5dtPu0M~NG`UuUptcRw)RFnlH5W6D*TP8$nd)PqMPlHjE@PO{Rut38P0m|Kvk_# zk!Psm2^svpd-R=u)YfxNukK^&{@F#)k@u8qPz7nUGNU4Ynp|w# z-RN~ zVx^uXJ!U=RtyS8&U$0{x9W>{rNUSb&nLt*-HRjR9Rb6hpe{R*;3Yggx5yY?cc-jy` z?diD|ios~E24{F;v0T>f+iV3NAX!)CbW2-XG^js9Ni5cBuZrFj)N#BQ#K?**uJY4K zul_KC?Tz0#t%&wg!mt{GAAR(B=Ep^S&Q|48NwQ5wunsWb(%t&Du0B+!S=zaRP94k` zqrPLYiEgzx+(SzMM&?W;*$6+}*IPmuH@hlVK^sqSe`i^=23V|-)*JcHLLFRsp~|wew6_}iwld}K%tjz_Y|*7z5I*Y?d>Mnqac=kXO+DpCRyQC3V$D~MPGqH zLsX>(Ne*{NPO42mejin}uj>on9QzQ-E())zFPiN?#3>_|_|16DN?GtEd@l5JxW9PZ zCELV{=~nVrNSj_I33i)$p%!qsb8l}y@m|M_~y88)EILY%)LTigc)&TOi9Le!h;DJx}a#+K35+!0oTl&-jH z9Xbko8G9$c=VdCo|5WK;5}&$JbVoiWnz0wCkdPZE$obvH;r+ddfBbD=VeOBFHI}3y zLp*Ho#sB{fOWgcFQYX6X^=5l+-zl82BnPyjm;U!ULd7y8+7ty`L{@?Y`06&yIm9|1 z;P6r}d|rNro@F_3g7(&z!ATTr<-A|j-Du`ci?K0UzaZMw;w!e(xFCHda2`2%Y0;@HoeTlk#y8eND~NCtY&eV$E=kt>iT=bci7*15pH7>L9(mCt2QYRM4Zu zC@`T`*4llON!N(f*+rDaxbkW6g86~AV$D?H)1AqTW(yKL?ntE*J0G%LD@YTs>^3w1 z`VYEH+jKwEo(Z8{4@!IZoOVJn)ZN!vkZP9i zG`NAF_@sSWEgc1qLNXN`&Z)2$pyD4~H!c18d*G& z2Re@=xWcaJI!r|zm}vE#D&MQez2YU;c8d$ivLwNRl?+PwTz}U{24{-qZxx5NE0o(6 z9k6PrF*#F76Ors7<7M42mJGm%2mTgsW=BRmfE+4SJU!nW52Q)4Nw7E@Z??tQ5MSDh zpn71t`VL+FV-l0X?RS);Qa&bL?e0m#@D(Z?8xmDa8Atnre&=)Xcb-D~6I1y#p`m6* z)JQ&N_rm_P1>l;Ax7Wa@#cMI}HSxD+0L#1cIK-p$M*v5`Gq6>jMd9d#Dp#c9kM%Pn0WLR`f_{|DtlnIN{Hoj(ZIqsgpZ z(t(I+O7SK47(JJbI^O#U)V5&5xj?$JBg3QDbO+-~S5Zz~O(`Od4B5@cL&xI|)y-3E zHVsJmEBV2+iNO+1UD)eC2}$j(3%KjscZ}@-D)#&Ug&oj4OAJGCzgH>i6Zqo z*%LgB-oyRs7>pDfpVmhFb%??LF4r#(A>a6`Ibzo{NSMOOBQdpE2uSdy>Mu$otT$Qx zRh^hsHXVS+GjoYUtvgpGQCmOwaRwusQQ%GD&zXXRjg}F2Rq#d@RY>*XUn8wqr+g}U zsNsUtqQC9pr{zWTKguHmf&iXNp&!=%Z#m!V3K5|J)N%3V{anACgW(tBjK&vjwt}Qr zmDyO0Q}fhwv;|Iii4bz5MfRPHW<1-9tt9~@Ms*AVXrRV}pdg&WyUJ`cE#2B0W@g`% zCSlXk+F^&UM(~Cx0pA0GlAJirx28~RBvP7c(irlZ!Em$Qw* z$+rPGwx5aQ*BM8J1>AC_RXEE(#7Uy($dCC^3~txanRoBJO(pv%nd15W4W{VvgqR5Y zPkGUWJ#U@j@AqX&q_UIf?kbXG92>HyRS?w#v{7deJu=3J1j&sMrdF~yIoDH53np0^ zr>>tx9WHaY^DG( zp{4vuT&PYSAsQ({=m3(bM>;aHIf0`_SI@m(!)VKYpY`t7s;K|PR4s?cJqg2C_RaZ^ z_$Qjf>?<{tilHHH$dAq1u5#J-8{H~5=hAwkC&X`_RZ zO3?c}x6b=VqGzBZdZqbHEs{fFwZMpZkL=BY++=kA>mEF8spYN4*4X=__7rD?=w`{T zl$Q;qs#BH-I@3;fj+dCSkXObjE3_xkdmMs3iq${NKf zT7SMqMq}MB<^HH_Lwv%zfrrPmmUh7kNowLohRk$`8`54U}KwF>>22fER#1qNVfeWW=C2DfA1w_Av>H!R(!31 zX+E!+gC^lS*KuwZDNdH@#@nbmPV|oFC#M9N))~CJ7gC(_>tgob2+1tefT9`&Hki~8ggNGSR(Hd zrnkA#1MR!R@22z5oWBqr<14P6iX=ZAnz|V>#>#CpF+NFgU|wpd1)+|__NX)5ywRK9 zHysyc%>VNg_YA1<)jsS$17~0MX^8K+mY)F=@&{W)^YP1W`m^vqUstPf89%esB?m_OqW zrQ575pkMQrn2R%?niG{!nEKvP6TNNLD8H8ne$t_6h4cxoi_@>bYRp~5Gpb$GB7c&{ zdKD%!iFu8}B`W#VBB@AI<^7p4BraF~%}6?bsEVAnFL6hKX1B6mL1Z0E&DPJz&_}5N zph{69BIotzWXi1aoaW}_2Zu_HTB$gwMN0ONV<5BLSKhWbdvJfp|6Q{7gKYi(D`A~P zEeFEbUPn@@EZyq=IC%ZI410CP9aM2XFd3tc(iaUQSrNO}ngtPZW6;N3mna<@ICaya#E&i|3n^@;KI6%37&77HP|mHAa8bY4vD zyp(^toFibMgFgTD9rh&*mEeR8D|X>-E2EBNhbv-ez_0ez)-N8M)!O&91u;l8f~Q~{ z-0F1)|7q7DW)#o;ctcAm?F2OStViZGORM1VwbeajYsWabMmqy;K`~}ni1^K>8rH&b zG+<%rj864EIbP8*3{%j&;4Y_u-YO0}cnQN81cCl5h<~*)W#dMAtf&up*hAuvy>GP) z5wqKI6-IQ>)l7)(ZzqLOF(D(jjc@M1w{gbZDA0};6xCEp)1ha~Ndc4rD_6fOx_ma# z{!Vn4N@Zwj7q)nVo*lP{RR9yRw{od9r5xBUG4BblO)aKIIe}m6!^%sajoJ-$Pyu@< zgVkAJ3KRkrH#1bg8wUo&6o_cMFBvakm?X%#%tAzE-x6*hX zdZa_AncBmpj@$lTrJSIRbFxedsR!(cLoluc=eW-m(Kqf}3C=!+2Kih0<$dK13LE-b z7)DJ=pr)&unxa3GD=t^uB@qKO@a!QLYvFJ@j|hkjj@!5@WNqYGr^GA7KH$2*bDWMG z$%-jG&IFX5)FD??^(7=$USeg1ag~+<5H|-G;nA}!K~t|>Y26Xx#M((u7&;)V$-Obf zDX58O%53>28E;<97Kzl4V1VeI^r@t6*OF1Lly89$U|9$V;wdIec!XXRZHLCCe95B9 zfT7or(%0^*sD|;$%jyx6pzz}|Az}1bP8B@iLvsB-a*myAoWo0?aw7w6P+=UKs|~r=aT=xAmH%tbyw^kN8}~n9E}jZdeIUOnbg@UQ21_u zzoY+e`jP&pdlGF%Q|^D1IVAqG%whQ-T@p){v6m%dBF^VscJ$i+BTZ74`i*tT=-fx< zNgIh)gGPdbEdMj_C!S9&d-mf~ZsW{`oZ41Y9eYTj0A%I{R0$D?SwG9Mi)udRpW4PN zsY?AijRbXDZ-D6eQseXf#1$WTXI28kjfuf!MvLwJx691W6vFSEY z=$=W0|LeHd9^^!UDjW)6JjO=umAU?XyMDJ1D+AB~XbzW8(^$D6-N{+i- z_vGGh&#Pot$es^PA{qclW<--su}>V6RdaKauceNWKKu=s;h`5Eueb;861CDrUL5`@ zkaPt{TdbJlbC-RACK}i<$$isBgp}=2D&tvdY9QA(TF*jFA1CYcfFr)H7vZFfP*q=Z zYAxrYkD*GHQ~r^;+-o?t0>j|=RQJ-RiU@QUU0xXT_t21%2Spocs*R~16ERi8OX)my zaE(xFYHn^#B>lRR#9h|@IO)IDvNk^OWAUJ6N5TfEC?&ts7atzY!>Dk%gat;xsV{EJ-yX1*)E2c zLlZmL^FlgjSH@c5#L7qeTBnjIJ#L)xsZ#n^iD!ikXiW3%njMf;lX)6dh-fF7!q zc?Y^EC#u&L+|7d<^FsMFE)<8>>5RjXCD&Eob;$850dt=Zyvy>Z>>dl*hHLIgZC74m zs{V7c+kcjpheQYL>CzxE8(6y!EHU#-PaZ;8W{{{ll#?xGnB zVRbaoCzh$>EBayuckwCV5=}3}OmMK?<)h|=7WWs0Dc!K+fMP+u*v&^45%MuZ-z_=+ zZ^?8L#-)@JShG{)3JQ$zs`%)7k7r*#~(WZnI zL$u8YdsyWB5+y$*FSV?rkVlEK+PxX1&4mDd;f&FHGg?To^T~Zy>puA^)2CatIj$TG zJC<1b@EJI)GSWCSuDQyhM@j#aQBdjl=Y3wL#1_&eISnwu5WPSzj*w2=v>k1~k7)G4 zIFWU8pSxPO93!nj*{YaG#4kcB; zM;$R`mwH-r6}Lrm3BQdLHIgZ-`-b%&zkKC(ATQ>i)ZHi?_S!Fe>}s}ROvo&40iP&1 zZWl@kC)Da8i$V!u&rMv8p8oCo+c6_685#0d*CwXx83;4egg=?A7f;n+I`fK9-)-a$ zLwUJWTvExs-PBM*r=o;p5e>+KQ5Af#O1({PMWt?*))ex;Q)QOM){|INOrK0}JO;-m zmN=|h^nb`=e_4;NW;qnDj;>~7yV9~oZO7Pr&$7sPz$&eBCM8`aS9;(Va~D2icJMHY zs@ADVI%ng84yzEEcxBhsJ@NIEnFJRdL12@p|~D>{^NDnm~?~5c=R8SXQXg2K`fPdHh|}KW&~zCM|rxR9HmI zRQNq%RB7Ut+~a>P@35c#KN{#%EDH|%8U@YP+xuM#{$F|1LKnhsM($_^8ty?@KwW4* zGs7yh(+amJJ47#LI^<-}_h}CXSjY^KQe7ow;*>cc^j2Ao9hvgJvE=_mx1U`eC#PS^ zu7?dFrl2Ih%ioC-iaGEoPO@KqbuI`l^lAw>Ia!W#(EiZRmBcIsk6ppNL`p8QLT`jvWMMC%(Ujb^AltUGu?I}#ms`?LN^t0{L1%H3c>#%})-rz@4nm5h8j5*<4@0@0gbid z#b#_R#Zq{J-$y9t$tS)KkW#q-hIu5o`7%lYli*?W()eq}#7T>F{)47GwWz66Ga0 znO=?#O_zjbUci5&gB$KqCpP-Iy+VzL8?SR-UkDywsnH-|EgNb0CU+2SNexp20CK`pHzuieAI-_XI6{AplaPSa}zqZ&F@kP3_cq?_#N=)nt3>k=gcf3*EcHqA?SYD2)IaI=$eV)=LO=L+bAD_TeN zek$?8UyC5I$o5sH&g$wm9;=GG>;=8^L%G7F61l892%AD{U=*Y9c$x@%^mje039iu8 zLL27x1;yZUl!RezkflqBQv}jYiEaczj37to72#KNc;ia+#5g`Sa{#6{% zNX=671YFt^Mh7dekSq6EiQ~eNLWYq@7}-{ps3C?eg;)wO3}<$Wwy3;pP=5~!8DhON zQ=#1eA@CZjnDu|ww`;TLR2DHk5fT|49jZ=uFcjV1XIs-run-+YRS$gKt>sgDq3^)s zl3g4VMA*lyzOK&dh|{hGT@pI51-+W?o_j_1<3L07n58)FyTS`p%0HQl18@AbB1N_2 zg<#k?a$QRS50$iO2ly*Lq2NDgaffZfA@HhRzH}(U|dqxpS?hlfbY8 zczPQBO!GVgUMNl*Kc1iFh@m>*y}!BtZz?Ch#O_d z+o;SQ6PU?tE4BN5&`4;gI7KJVX~^G$*I#jhAo>Fy_A~v8 zmRKxp>4*_Q8$L8t?)-(0WW`yRgty>Akt@y?l=%d+)BM(Lk|}!zYUQO7m>J zT5-ND8`Q_d?aFfVZ|Ps{)(dPU7AT)B6rj|9|9&w&^Q67)x{-PeCbUS3fj1YCs(XF_ zlfM4-zaGtcCiuam(6au&k4@B3tN#N~%ZZ1=9rNxPPCmxUdzhsPp4iX-$AO8P*WGyO zjvTgg+%g(It}80w^`s0%pjY!L8N|7LUwzEBy6}4VCVm}I{_Sb?r*>C_#7R;oLLytn ztzuOPga(eJ<{3x;KkR`DCh1z7XmJM9Mowdxb1{Y8pRG!4oGAJo@E8$=Hv_TN{+d0Cz^Qsor3vyTL}YivXE-CTf`)U zW#+4<*tf>TB3mp&0$O0W)gOjPSp4rN2`-ZGC~;X6p4KxI8D6QM(jx^s!k7Lr4d$Ca zqOzf4;+9}aBjyi-G>FjX?zR~Y*dJrv9QV)IM%nV$hnbbHqc~FIGk-B1|Du?FZSb>w zmft7^3%^!YT?r8CAp;JG3VxsHVi)Hbs=IQ|hBHQ4HPTWI!6(4-z*JDTUO!HGfT|!2 zXT=)?nisP(kd>=8Wkr7?Y=^%JrAB1RKd2LPULPTZaAM50Kp*!*jIy@@eOxsy4TUo4 z$GXR_J3=Xp85994x}vR(*&cYgg>Adan;DOrf2U$~*Gv?u=%y!)`+o_2xIm%wf#F8( zIOezu(NmU|iJKzPi;1!FJQ04Qu#$PZZe8>YbZb~Af2PjKW&G#8q5S+I+<2-fVf+%6 z7P*9mE@rF4M5V(Se{%c3I#tq*Q6kbDO&nspYY;R1Qb4yP zB_|pr-1AL$+U4}v%3s0MN8voAmb%cuDR5ZFg;&0;=Ok{S0IjXwEsx`%{9ph%Cr@#M z;(oWrlTu55rowd2QaGQ}Vz3=ivRfgyMJXE1YAUp^Ec;VXJajHY3uR;D@@|f! zu-q>FTFv=<<$a9au4D+OeXrk1R&9j{LJCIXWhzH$hdqUIDKuCd^KuWzX4&5G6aNU& zQhE+Nm+LR%dFgB2=@(9|Kt&sL9F5Vro8waP!rH}RTyR^9!2E-nrpQ$2e0=whxohgz0`0qmKFyyYWnff~p($lW3`+ zO6luE8dr`jMB{;V$J&KJra8Fs^RPlzG}wR!*S4!4qB947d^4F52DUcxW|%TwQ7qG4 z9`OPRaR|a*NGG0xNl8!D8-`+J*V}{ygL26O4&MCSytxeuvfq*%zRLt!1-HKC*f%Xf z++S{UrH()LR2Rxm$8+<^;8w`9;jO|j<5}Y_0r}nUNoH(S_~N_3Mx5BbsY+Zv%84Mn z0gxA-E_+Xp!-hM!C4@z1pU$pfm}Z)H50^-l*&S3{D6H!`i^(MCPkB8j`Zsj#doCfi=|ntSX*$x0w|A*0!2@@4r& z3U^aZ*luHhbbFJgU2pP(VCg5A#G^LJ;JCs-2Vf;0w8=ir^9+docWk3D!LqN_=F#o{ z)4K2ex1EI^p5=I@{R&cT4f^D~+y7o%Hx|z*Y!DI0T&bMivqiqHbfF-#SQ$7Up$`{- zx{5CN%yE%?j-zRcIlMb_qg;7%H1x<@&P+&_5e13y{#O=^Vz{HB*o3|?4n zn!Q{lw)F1Fbs&nHF>c4od_1Kup?73}PXxb3MYJ3#d@fm4J#(cjAB%mRYLRHO_bUo^ z4O+J4GeG+-EAbhi8jA_A1z8R&3=c5j@>`x~2 z!egLLXE<$26%!~caoDiM*qPuAj5yVnF(Wt!n0~xGj3HAXVJ|38Y{th!sA_s~oX9s( zPeI?Ne%XoqV~-VtjmzTH=2>YDijjiTAF6XPT3p}zJV{kp3Kz8o&<7#F1l-@3jTDJq z`3yH7_e=B_CC%~V8UN&B!`)Ha=DM4;>0Vs+r0JEX{#?$Z&?MDH>hg8L3b)vG3(8tN zM4<%j?2eSkHfU;!Gs9t+X!B6@^ByY8c$|X7uT)>~c`ZHzTj^NTKjj~6kt^dTybIxf z3(VD;h;Frpo7>cpFzJGC$>REcO?iyEU9^ytT04IGi5S;;1_l&hXe&kok1-r6sj1c= zp^Xcb*_TH*^B9%T+H$;7qWhR!yyO4sf&k0y)pRH`z1yq-bG>8mQwMQuX%(+?ZRdrB z?kBKZy1wb1cN=OB14~W_Eo)bORs9x-UO_#mvlj$ERHyyn6&(Du16U&hDp&%~nkFs~ zm*sl=2Z>9<2*l?!(gXYN@WIc{{Z5e*p*r6hEs3G~_i6iJf`szYqt4DI#^QU4N58aL z?+UjP(2YDK{-&(X=3XJ{)YBO>ZPxvK0_64Bk|TnW8q% zW-f7i$sUIIT^M;K)pa8`i3A(pZOpx`?C|`m+`KYcOb30F{R~XKF~&>Doz$Q!LwEF$ z@(ZfV3?FylaL3aXy|XB6c1hBB29zhBg28x8`1xKAG^;%02TA@u$EQnt#(4w@ct2S{ z*|iioja8iNSXrNL0beh@8HQ2HFvi?3OgB>(i7z zC9Cb_ji24WOD`o3-TOQPLDicYqrLWoLG#iNd&trvzE!f~`u%7U6P32_KTjLEFrZoB zqBmGpDc1$HN)*~JH@0{JUrc~%urCK^CuATv`T043*pW62L`YJU(Szn@wW_J>Pv0FZ|WT&69 zr1W+a>L~MicrLUAB6!&m5czo?UX@l|K~uz9xfz$aKH5O$N^ap>c9Gcg1vD+b2I;Pxj;FoQ8`8!QE0Q=!rhN?fPO*D59{sCWinQLeB47eJAP5rWHHTu z6c>b*xvt*qk^XPv5YR8oxpUpK-fai5WR4AnXa$e5ga6UC?Mhn>4~6BO_O(DdUe@pc zswa7n~pXN)Vxz1FjHICl<8(i%HWik`IqI^V7Iy+I@F_Y#HV!YXD^NEIk0 zpFCk!)K;ubNfOr@;1^EbJjOpr^_fT?Z9{ywQ!lt^sLkv{mfcO#=wl1q1&fU$6x^oa3b1-lT!{NkuqF2lt(8TrAX8Dn2oi5&8s14_Ems3^; zS*8q5F{64>kP(_&(?o1Md2Eza8DsN!ir4RGZy)bMc^H>OO^Q$_@YS-Oflk}+uD{OG z?W}NXCUPIVqJftGV4;8*?97~WT3220Umeknk8ODW;T?8^!#yi0&21Mj9>YJ#+eye&TJD!ze#xYH6J%szHRWe z_O`Rbsd#dzfD*HY?t@G3*D30e8%^`QG;xr)#BQRxFF_Xb>e6}BCMSbaFASgVuqsDZatAjIP_^q; zH2H-k{P_6|Bt*iMBRBVhL^=I6CYzKHAx|m@)BqTEarSOl3V)_}BKBI;Rl|26h}eOD z$ek-;re*ITs5@SiFr8(SI=t7?Q!Z`e%Ka+mIEiwI+0rykW1Mt7F6WqFcM#q#Qp&99 z)#fgOFC&TC@H=M=bu_WfC_DEE_j2L*q;Tq^b$g=|3BfG$}j9^0t3unVG_d z(m_r>J}^YOltulR#?4?-e~3Zq18araM{;)x^SIf!xkX=AxtG-VeV$C(G-~j8k&Kiy z=M6OzFVJ0ei*m$becD6jr9stanUxm7E=O?G8bg_J-@`*Ka*x-tGds!00Kcj#1`M+b z8sD=fduC;qaExBen;Elv&adUDg{98_)Le3yM9N^( z#Y*kC1?8z4Bnwy{MGB$`{);t@n&u>#N=_wDsoF8LXhW(o9hSV{a|M10j?HLfN=rBH=NwSHMe-WGlnJ{hWV69Il zNLo3D$E7B2;VU$x{umE5ZW2`oo%UV`Ph88!lo5FicHNZSrfCk=hRD-byKpph4F@L> zr)Q4%eERbFyfb1=PN;JcOsup0DuC?&t^RHA=CrAl1In2;l~iH>YIMQ!(ry*3kscn3F|on9@e|M{#T zkEORe?zdaHU)!IiH4^cTUfdTRsydxKKKWf2Dcx>xq65&Vi#DKddi*QDol(+l^*F5| ziFr@aJ8>@K1;S|;aGVPdb9s}NGu-3oJUSqH`3xM+S`gF!)WIsfp6QM+fni=AGG@B) zKyIbUN%cy<52O2fD%tLsui0StRMS;2NBk3CZ?cY+;h+w!spZ#qp%+h0$zWo3wxZ^r zZBY3S z+G9?VY}|=H*>dWD#HE(*;MZ`Mb8%Fxr?If=8y=b5(hyujA#H_#rfsUT%`GlADf-M@ zRs!$aE{69KTom40kuh>*124l_xAB971K`z56=#0({{CdZ$gifwiuYxC(lwor%&*Xt zKA$%)9B{>T1a;Rppy*}JSs1Z0?TPK*xRonVBu0MZ<_HYsMM`xK;|$26<_e6=Y;$q6 zY-`ZuG%$!Y%vpWWNyt`oK9=jIMH1B>4Mv-lVgiZ#Yo3|V?j)%!QY%82bAPyWb^R0m z?yNyE6y`S7#3oL1i^|xel8l0PyHF-+`$LE`R${_fv_T%XQJwh*4ouPex$2Y%orDv% zGHtCi45?Ji1l29EJrHFjh5%-QXv4Ue5AWaO;j(^oBgoy{@cUv$b3~;gErqKf5mNZI zK;jQ+LJPA|o$w#&-(h>%YI&6bP(Lj9;6pGe_2Q8ql&9^gR5r&*G)Hn`i_6f**pdj_ zUmeY7dR|Cq;u$D^V6m=~8Cu{u)os3dv>g}~K5ieahokmGa+r{nFXZ(@qisDIL*>^wuf%7NXh1GvwruG3 zpAjyUc6oje+YNwb3Zk#o?0+f}Un8j4rzM3RN7w%=*^vJqq?nzWYgt~{ie{IWLhes0K~f_f z;F~8^pub^Hl2hFER8sg5s`+)3i=nPm`DP2s$9`1M`b9bAJ*j0_e(fi>c*Tji%Y_Qv z)w~eF$c$eH%R;-2M`V)4MsO5IN)`3_?98e4Q2^u`8ea+6522mrI-+KB^E4&)SuEoG zB+67NJ3t8~89JQNZ9k?VG%tx^MLL|S^Z5L*jzhnLw$BPHf9V9ga%FDG}?$2&*;@ zn84>NP#efO$Pk#+MLrRUe{=Ia1)ZLRPn9=qT`v1gOziTq(0*o1#x0eNL@dJyN%Lb%j*h?vrjPLq^lr0TtHS+f~I5EmjxaSxMUUY|e2t*r?*@t~sbQr`0Jp8rYcg)b8! zNnM7Pzwc{4rxwy0AUftt67l%v5!S8O?8qt*WtcU3d5SepTl!Wm7;FPi8ev==kM%q!Zpn`oVX`pS|QYhiVUX0*^2`WzymBx9mQveo-wX|$M}Y~plZuKUwkG%;(i6s9#9Ew z(!DAQ(YU_i41QvMS;zy{0^wH`Vn=Ln#2XjO@-6`QTF<3yEx7&H}snP`4cYsgO z)OSG9M`nWN-$(I8v7D#0w_Tv=5}ko^_-PBU;X4mVC{KdEpoyu9?ZY^ZtR7|LQmK7T z;q=6BH|WP#tlYpv#-Uqd?@*qBlHBSCv}p5ZAhB8r@uaWv-!V}#)BpHus?G!h;Yh{) zJn$0RYuVrtmo@J{YS7;-)~&;`9=kgKQH6^OHQVuJpOsJYlO%{kmBAR<{(=)FAuPb~ z$HRNcDJWdr;s%C8DQctEM7kl%68{T zkMJQ1j=-YtE^L4|6MYqPk-koIGUK8OplpziU0W*jbXMh28(^t{V&3M2^-JT4xM_Ew z?eof?eFHB>ueML}oeWnnoJJmcb2xH_p{zN#gs}Oimj1wkOwpd)c8SwWKvNBj>p#s| zoF*l|o1}nY(ycZBW7+^Zj%Xf$!7wEu@+)Krte5` z`?sPlVk|GnNzp3(dEU}`cJrUqBSCK7Z9Yybu?m zTbyp9ygHFIRM;)ry)Su!TG5OtO(aTG9nI?21%Daf3KPY>DwyaZ_qsnW(6{)fX?QPy zPRn_s2lGq#A(-IRZCzx)2s_r`I5)#Uha#t{mnCmI0`!joHD=BI%XugWlJ0}M#OlTj zHWf*jB$!a{w6ytmow0%p(?_)G zcbZ8%iIAvXV5dIXl2C;n*xjS28HfD*7OPkrGL|s_q6r>?VZ46h?NsdO&J7lyETCGafCUefch}k;XE%e8551kI$?aF z_@%|idI^u1Bu!TPHkC3XRf;Q7)eCh-K7)0z50L)|1_GZ#ax)g-4gvO<1=?Rm9pg{J zce6-d9fnSpicW$SQ#trlPy(kQ0OKW2q$wi2{46r3(#$uzNDy(hI0>pgl+QqarryoB zbGNwAJQ!wZo>{t>81w60<+z*%a=}6Pf@T$IpjL2}1(fQDf1_qq5tk>%iIRmAV4w=D zZ)?o*pz4Oc#Y;sAr#1{)B0{%$;;lD(^u(oQ44Z>Fb&o<7$TQS1{2E zNSy_dp#LWzq0Yh(Zr|{ZWZ><~TULe`l3hsv7>iEjb zAiTKxnUKzc+o8zJpUVAb@-lL>5VzVGNvzxvHUJv-D%T#1MYGeW>S_nl7L=KJBD#L! z4*p+R8an(>21WRsqF;Yq8!~~k{v(E4d%R=zqvwYh!u(06_J2jJz8CD9ejUAy`ROtE zkn7?PU?CAaBt-00dU zHfiAE=Jt@tVbNdW{6b_GE(ukkWfLYyT;@f3J<+xUO$adJ4cE`jWcst8nVh=UljBR; zCosZ*y25kVR0+K0{B@8PVr1K?E}n7@;}D=8&(x9o6Ic(-`Vxg0U`!~JRl3++``?%6 z@(x;~I1aoRGiJ%T_s>=22X!)7Hp9@+C^=P``=UDgnXEyegaC-@Qr0H~FA~{Kl%pku z);S(yX)%#T7drY>Y#A=VNCb@Ll+WC2G!xWQ1*TN#2P}+($8Fc90HPSUs5E*;=Sdg+2vGrj^RDtg5+ z?3QK4S&aOJiX4jm$5VV}@474`Y9{W%W||iXGk%mj;xsi<_43F2DNq!V+iy8O8ZRCI z$)wEtouU9mOqCcaz!dw!iRMRePJrF;Yjli4=&?q+b^OV$RG@lwmv~BU2UFqd`)`mR zPzH4jR4{XW=!CS2C86q1)5+0JiX6K*^YFz8qEjGdgX^*|aUv^xN`mjT1os&POh_B6 ze(a860f{S~Dj$Ogh#MPxLdKK4gmGb%xuV!qovrw36{L_z>K6nc_9Vvuq3w^h7i4m~ z)L%*4)RStWq~oVoQ(DcN_>k&fn z_GRhTgESsnDJNYdy6ZQekClh>>5oQ&hwjvoD{^VL8gJcxM0yfyiYLd>#2!Ri++ z<7N8<#wQ*OY`*Rjeeb&XM~)gbL$`SeEti z1qFrr1RGyAp*H|DOC_?dlKVL_G%Yc5;Vf!?z_>f)bc|at1?LwBBugC>JWN=OC1@UC z*Y{O!IA3rhQ}nnujPgvOL>s8h9OI^(b5J4e^~oGQ;J0zca%6rk_@4&Kog|PCdAlUyr6&#G4cW^j-~&nr>l=? z;*7&DAQVMcz$#F|Vj6)zT(CnZvX;sQ(lkOT9}$eMF)kL6AVsScSPnl{CSS#m45vV` zAf-5kkO>Myrfh9oQa(E>Vb~xD7GxF$R^)oO_YT+}_s89H?{mNB`8}6=?~*sSwSVzqYn03e6lM)lr8l(2?0F*&$*b0{gVT zZ|T4H_F#F=K1U0l{`CI#Ni$3EzK6~9l}0O4|0NgJrG?r4qSL7t*oq-8oLDE`Vu_>xE{-Dngyd z%5uuakO40Ob{PLI+Vfz?g;}h)dqVKk$C&U&Hl+HrAf8tjgYzh}L$LQO*e<@(Y#{Yl;BJ#idP!iep1M+}j zEzR5?xRP@{|4s~VxHt$vj}9WQFPN~Zq1D>{{2qH1?nfCRg}KWmA_{L*!&@YEv9+5g zj&T-o5KWpAYwcCP^PauMZ1|jp97(Yb*K3%^aPVSPmJ{aD#dmahQkeG|dRE2ZHc^+A z40MB#7M)+$jQb^~DeO|NG`)KwKGN6N zw-#cEGOT5psrse$AV-Zq;Cn)DA<|01LDpCGbxwpm`yeW|#fEY`?48eWpou2!a%on` zwt3Unhl|F$_(U2ZzG^2$Nu8;kJf(+~Z$&dSJso#sCgejL4;ltwrRS4irR4rid44jy zC)O@R5-;vMdA!-TUA#b0$@^uZp>O_&!CU%$NyPFrRt;&XMT?0bVtInRyVQkLTkIp@ zCo0*rI?ThKgk3f-2dD@m%}@Y$1qbc%d5E6qOCZWTQZv{+YJe@>MzqoK>v4NZWrnFW y0@2?_%2)+#Z9~|sK`&fb738)(!rAatyJc>;A^VVEMmxD-= 0: - self.proComEndDate.setCurrentIndex(self.proComEndDate.count() - stopIndex - 1) - else: - self.proComEndDate.setCurrentIndex(self.proComEndDate.count() - 1) - - @pyqtSignature("int") - def on_proComEndDate_activated(self, index): - """ - SELECCION DEL RANGO DE FECHAS-END DATE - """ - pass - - @pyqtSignature("") - def on_proOk_clicked(self): - """ - Añade al Obj XML de Projecto, name,datatype,date,time,readmode,wait,etc, crea el readUnitProcess del archivo xml. - Prepara la configuración del diágrama del Arbol del treeView numero 2 - """ - - self._disable_play_button() - self._disable_save_button() - - self.console.clear() - - if self.create: - - projectId = self.__getNewProjectId() - - if not projectId: - return 0 - - projectObjView = self.createProjectView(projectId) - - if not projectObjView: - return 0 - - self.create = False - - readUnitObj = self.createReadUnitView(projectObjView) - - if not readUnitObj: - return 0 - - else: - projectObjView = self.updateProjectView() - - if not projectObjView: - return 0 - - projectId = projectObjView.getId() - idReadUnit = projectObjView.getReadUnitId() - readUnitObj = self.updateReadUnitView(projectObjView, idReadUnit) - - if not readUnitObj: - return 0 - - self.__itemTreeDict[projectId].setText(projectObjView.name) - # Project Properties - self.refreshProjectProperties(projectObjView) - # Disable tabProject after finish the creation - - self._enable_play_button() - self._enable_save_button() - - self.console.clear() - self.console.append("The project parameters were validated") - - return 1 - - @pyqtSignature("") - def on_proClear_clicked(self): - - self.console.clear() - - @pyqtSignature("int") - def on_volOpCebChannels_stateChanged(self, p0): - """ - Check Box habilita operaciones de Selecci�n de Canales - """ - if p0 == 2: - self.volOpComChannels.setEnabled(True) - self.volOpChannel.setEnabled(True) - - if p0 == 0: - self.volOpComChannels.setEnabled(False) - self.volOpChannel.setEnabled(False) -# self.volOpChannel.clear() - - @pyqtSignature("int") - def on_volOpCebHeights_stateChanged(self, p0): - """ - Check Box habilita operaciones de Selecci�n de Alturas - """ - if p0 == 2: - self.volOpHeights.setEnabled(True) - self.volOpComHeights.setEnabled(True) - - if p0 == 0: - self.volOpHeights.setEnabled(False) -# self.volOpHeights.clear() - self.volOpComHeights.setEnabled(False) - - @pyqtSignature("int") - def on_volOpCebSplitter_stateChanged(self, p0): - """ - Name='Splitter', optype='other' - """ - if p0 == 2: - self.volOpSplitter.setEnabled(True) - - if p0 == 0: - self.volOpSplitter.setEnabled(False) - - @pyqtSignature("int") - def on_volOpCebCombiner_stateChanged(self, p0): - """ - Name='Combiner', optype='other' - """ - if p0 == 2: - self.volOpCombiner.setEnabled(True) - - if p0 == 0: - self.volOpCombiner.setEnabled(False) - - @pyqtSignature("int") - def on_volOpCebFilter_stateChanged(self, p0): - """ - Name='Decoder', optype='other' - """ - if p0 == 2: - self.volOpFilter.setEnabled(True) - - if p0 == 0: - self.volOpFilter.setEnabled(False) -# self.volOpFilter.clear() - - @pyqtSignature("int") - def on_volOpCebProfile_stateChanged(self, p0): - """ - Check Box habilita ingreso del rango de Perfiles - """ - if p0 == 2: - self.volOpComProfile.setEnabled(True) - self.volOpProfile.setEnabled(True) - - if p0 == 0: - self.volOpComProfile.setEnabled(False) - self.volOpProfile.setEnabled(False) -# self.volOpProfile.clear() - - @pyqtSignature("int") - def on_volOpComProfile_activated(self, index): - """ - Check Box habilita ingreso del rango de Perfiles - """ - #Profile List - if index == 0: - self.volOpProfile.setToolTip('List of selected profiles. Example: 0, 1, 2, 3, 4, 5, 6, 7') - - #Profile Range - if index == 1: - self.volOpProfile.setToolTip('Minimum and maximum profile index. Example: 0, 7') - - #Profile Range List - if index == 2: - self.volOpProfile.setToolTip('List of profile ranges. Example: (0, 7), (12, 19), (100, 200)') - - @pyqtSignature("int") - def on_volOpCebDecodification_stateChanged(self, p0): - """ - Check Box habilita - """ - if p0 == 2: - self.volOpComCode.setEnabled(True) - self.volOpComMode.setEnabled(True) - if p0 == 0: - self.volOpComCode.setEnabled(False) - self.volOpComMode.setEnabled(False) - - @pyqtSignature("int") - def on_volOpComCode_activated(self, index): - """ - Check Box habilita ingreso - """ - if index == 13: - self.volOpCode.setEnabled(True) - else: - self.volOpCode.setEnabled(False) - - if index == 0: -# code = '' -# self.volOpCode.setText(str(code)) - return - - if index == 1: - code = '(1,1,-1)' - nCode = '1' - nBaud = '3' - if index == 2: - code = '(1,1,-1,1)' - nCode = '1' - nBaud = '4' - if index == 3: - code = '(1,1,1,-1,1)' - nCode = '1' - nBaud = '5' - if index == 4: - code = '(1,1,1,-1,-1,1,-1)' - nCode = '1' - nBaud = '7' - if index == 5: - code = '(1,1,1,-1,-1,-1,1,-1,-1,1,-1)' - nCode = '1' - nBaud = '11' - if index == 6: - code = '(1,1,1,1,1,-1,-1,1,1,-1,1,-1,1)' - nCode = '1' - nBaud = '13' - if index == 7: - code = '(1,1,-1,-1,-1,1)' - nCode = '2' - nBaud = '3' - if index == 8: - code = '(1,1,-1,1,-1,-1,1,-1)' - nCode = '2' - nBaud = '4' - if index == 9: - code = '(1,1,1,-1,1,-1,-1,-1,1,-1)' - nCode = '2' - nBaud = '5' - if index == 10: - code = '(1,1,1,-1,-1,1,-1,-1,-1,-1,1,1,-1,1)' - nCode = '2' - nBaud = '7' - if index == 11: - code = '(1,1,1,-1,-1,-1,1,-1,-1,1,-1,-1 ,-1 ,-1 ,1 ,1,1,-1 ,1 ,1 ,-1 ,1)' - nCode = '2' - nBaud = '11' - if index == 12: - code = '(1,1,1,1,1,-1,-1,1,1,-1,1,-1,1,-1,-1,-1,-1,-1,1,1,-1,-1,1,-1,1,-1)' - nCode = '2' - nBaud = '13' - - code = ast.literal_eval(code) - nCode = int(nCode) - nBaud = int(nBaud) - - code = numpy.asarray(code).reshape((nCode, nBaud)).tolist() - - self.volOpCode.setText(str(code)) - - @pyqtSignature("int") - def on_volOpCebFlip_stateChanged(self, p0): - """ - Check Box habilita ingresode del numero de Integraciones a realizar - """ - if p0 == 2: - self.volOpFlip.setEnabled(True) - if p0 == 0: - self.volOpFlip.setEnabled(False) -# self.volOpFlip.clear() - - @pyqtSignature("int") - def on_volOpCebCohInt_stateChanged(self, p0): - """ - Check Box habilita ingresode del numero de Integraciones a realizar - """ - if p0 == 2: - self.volOpCohInt.setEnabled(True) - if p0 == 0: - self.volOpCohInt.setEnabled(False) -# self.volOpCohInt.clear() - - @pyqtSignature("int") - def on_volOpCebRadarfrequency_stateChanged(self, p0): - """ - Check Box habilita ingresode del numero de Integraciones a realizar - """ - if p0 == 2: - self.volOpRadarfrequency.setEnabled(True) - if p0 == 0: - self.volOpRadarfrequency.clear() - self.volOpRadarfrequency.setEnabled(False) - - @pyqtSignature("") - def on_volOutputToolPath_clicked(self): - dirOutPath = str(QtGui.QFileDialog.getExistingDirectory(self, 'Open Directory', './', QtGui.QFileDialog.ShowDirsOnly)) - self.volOutputPath.setText(dirOutPath) - - @pyqtSignature("") - def on_specOutputToolPath_clicked(self): - dirOutPath = str(QtGui.QFileDialog.getExistingDirectory(self, 'Open Directory', './', QtGui.QFileDialog.ShowDirsOnly)) - self.specOutputPath.setText(dirOutPath) - - @pyqtSignature("") - def on_specHeisOutputToolPath_clicked(self): - dirOutPath = str(QtGui.QFileDialog.getExistingDirectory(self, 'Open Directory', './', QtGui.QFileDialog.ShowDirsOnly)) - self.specHeisOutputPath.setText(dirOutPath) - - @pyqtSignature("") - def on_specHeisOutputMetadaToolPath_clicked(self): - - filename = str(QtGui.QFileDialog.getOpenFileName(self, "Open text file", self.pathWorkSpace, self.tr("Text Files (*.xml)"))) - self.specHeisOutputMetada.setText(filename) - - @pyqtSignature("") - def on_volOpOk_clicked(self): - """ - BUSCA EN LA LISTA DE OPERACIONES DEL TIPO VOLTAJE Y LES A�ADE EL PARAMETRO ADECUADO ESPERANDO LA ACEPTACION DEL USUARIO - PARA AGREGARLO AL ARCHIVO DE CONFIGURACION XML - """ - - checkPath = False - - self._disable_play_button() - self._disable_save_button() - - self.console.clear() - self.console.append("Checking input parameters:\n") - - puObj = self.getSelectedItemObj() - puObj.removeOperations() - - if self.volOpCebRadarfrequency.isChecked(): - value = str(self.volOpRadarfrequency.text()) - format = 'float' - name_operation = 'setRadarFrequency' - name_parameter = 'frequency' - - if not isFloat(value): - self.console.append("Invalid value '%s' for Radar Frequency" %value) - return 0 - - opObj = puObj.addOperation(name=name_operation) - opObj.addParameter(name=name_parameter, value=radarfreq, format=format) - - if self.volOpCebChannels.isChecked(): - - format = 'intlist' - if self.volOpComChannels.currentIndex() == 0: - name_operation = "selectChannels" - name_parameter = 'channelList' - else: - name_operation = "selectChannelsByIndex" - name_parameter = 'channelIndexList' - - value = str(self.volOpChannel.text()) - - if not isIntList(value): - self.console.append("Invalid value '%s' for %s" %(value,name_parameter)) - return 0 - - opObj = puObj.addOperation(name=name_operation) - opObj.addParameter(name=name_parameter, value=value, format=format) - - if self.volOpCebHeights.isChecked(): - value = str(self.volOpHeights.text()) - - if not isFloatRange(value): - self.console.append("Invalid value '%s' for Height range" %value) - return 0 - - valueList = value.split(',') - - if self.volOpComHeights.currentIndex() == 0: - format = 'float' - name_operation = 'selectHeights' - name_parameter1 = 'minHei' - name_parameter2 = 'maxHei' - else: - format = 'int' - name_operation = 'selectHeightsByIndex' - name_parameter1 = 'minIndex' - name_parameter2 = 'maxIndex' - - opObj = puObj.addOperation(name=name_operation) - opObj.addParameter(name=name_parameter1, value=valueList[0], format=format) - opObj.addParameter(name=name_parameter2, value=valueList[1], format=format) - - if self.volOpCebSplitter.isChecked(): - value = str(self.volOpSplitter.text()) - - if not isInt(value): - self.console.append("Invalid value '%s' for Profile Splitter" %value) - return 0 - - opObj = puObj.addOperation(name="SplitProfiles", optype='external') - opObj.addParameter(name='n', value=value, format='int') - - if self.volOpCebProfile.isChecked(): - value = str(self.volOpProfile.text()) - - format = 'intlist' - optype = 'other' - name_operation = 'ProfileSelector' - - if self.volOpComProfile.currentIndex() == 0: - name_parameter = 'profileList' - if self.volOpComProfile.currentIndex() == 1: - name_parameter = 'profileRangeList' - if self.volOpComProfile.currentIndex() == 2: - name_parameter = 'rangeList' - - if not isIntList(value): - self.console.append("Invalid value '%s' for %s" %(value, name_parameter) ) - return 0 - - opObj = puObj.addOperation(name='ProfileSelector', optype='other') - opObj.addParameter(name=name_parameter, value=value, format=format) - - if self.volOpCebCombiner.isChecked(): - value = str(self.volOpCombiner.text()) - - if not isInt(value): - self.console.append("Invalid value '%s' for Profile Combiner" %value) - return 0 - - opObj = puObj.addOperation(name="CombineProfiles", optype='external') - opObj.addParameter(name='n', value=value, format='int') - - if self.volOpCebFilter.isChecked(): - value = str(self.volOpFilter.text()) - - if not isInt(value): - self.console.append("Invalid value '%s' for Filter" %value) - return 0 - - format = 'int' - name_operation = 'filterByHeights' - name_parameter = 'window' - opObj = puObj.addOperation(name=name_operation) - opObj.addParameter(name=name_parameter, value=value, format=format) - - if self.volOpCebDecodification.isChecked(): - name_operation = 'Decoder' - opObj = puObj.addOperation(name=name_operation, optype='other') - - if self.volOpComCode.currentIndex() != 0: - - code = str(self.volOpCode.text()) - - if not isMultiList(code): - self.console.append("Please write a valid Code (Example: [1,1,-1], [1,-1,1])") - return 0 - - real_code = getCode(code) - nCode = len(real_code) - nBaud = len(real_code[0]) - - opObj.addParameter(name='code', value=code, format='intlist') - opObj.addParameter(name='nCode', value=nCode, format='int') - opObj.addParameter(name='nBaud', value=nBaud, format='int') - - name_parameter = 'mode' - format = 'int' - value = str(self.volOpComMode.currentIndex()) - - opObj.addParameter(name=name_parameter, value=value, format=format) - - - if self.volOpCebFlip.isChecked(): - name_operation = 'deFlip' - optype = 'self' - - opObj = puObj.addOperation(name=name_operation, optype=optype) - - value = str(self.volOpFlip.text()) - - if value: - - name_parameter = 'channelList' - format = 'intlist' - - if not isIntList(value): - self.console.append("Invalid value '%s' for '%s'" %(value,name_parameter)) - return 0 - - opObj.addParameter(name=name_parameter, value=value, format=format) - - if self.volOpCebCohInt.isChecked(): - name_operation = 'CohInt' - optype = 'other' - value = str(self.volOpCohInt.text()) - - if not isInt(value): - self.console.append("Invalid value '%s' for '%s'" %(value,name_parameter)) - return 0 - - name_parameter = 'n' - format = 'int' - - opObj = puObj.addOperation(name=name_operation, optype=optype) - opObj.addParameter(name=name_parameter, value=value, format=format) - - if self.volGraphCebshow.isChecked(): - name_operation = 'Scope' - optype = 'other' - - name_parameter1 = 'id' - format1 = 'int' - - opObj = puObj.addOperation(name=name_operation, optype=optype) - opObj.addParameter(name=name_parameter1, value=opObj.id, format=format1) - - channelList = str(self.volGraphChannelList.text()).strip() - xvalue = str(self.volGraphHeightrange.text()).strip() - yvalue = str(self.volGraphIntensityRange.text()).strip() - figpath = str(self.volGraphPath.text()).strip() - figfile = str(self.volGraphPrefix.text()).strip() - - if channelList != "": - if not isIntList(channelList): - self.console.append("Invalid value '%s' for 'Graphics:ChannelList'" %(channelList)) - return 0 - - if xvalue != "": - if not isFloatRange(xvalue): - self.console.append("Invalid value '%s' for 'Graphics:Height-Range'" %(xvalue)) - return 0 - - if yvalue != "": - if not isFloatRange(yvalue): - self.console.append("Invalid value '%s' for 'Graphics:Amplitude-Range'" %(yvalue)) - return 0 - - - if channelList: - opObj.addParameter(name='channelList', value=channelList, format='intlist') - - if xvalue: - xvalueList = xvalue.split(',') - - opObj.addParameter(name='xmin', value=xvalueList[0], format='float') - opObj.addParameter(name='xmax', value=xvalueList[1], format='float') - - if yvalue: - yvalueList = yvalue.split(",") - - opObj.addParameter(name='ymin', value=yvalueList[0], format='int') - opObj.addParameter(name='ymax', value=yvalueList[1], format='int') - - plot_type = self.volComScopeType.currentIndex() - - if plot_type == 0: - opObj.addParameter(name='type', value="iq") - if plot_type == 1: - opObj.addParameter(name='type', value="power") - - if self.volGraphCebSave.isChecked(): - checkPath = True - - opObj.addParameter(name='save', value='1', format='int') - opObj.addParameter(name='figpath', value=figpath, format='str') - - if figfile: - opObj.addParameter(name='figfile', value=value, format='str') - - if checkPath: - - if not figpath: - self.console.clear() - self.console.append("Graphic path should be defined") - return 0 - -# if os.path.isdir(figpath): -# self.console.clear() -# self.console.append("Graphic path does not exist, it has to be created") -# return 0 - - self.console.clear() - - # if something happend - parms_ok, output_path, blocksperfile, profilesperblock = self.checkInputsPUSave(datatype='Voltage') - if parms_ok: - name_operation = 'VoltageWriter' - optype = 'other' - name_parameter1 = 'path' - name_parameter2 = 'blocksPerFile' - name_parameter3 = 'profilesPerBlock' - value1 = output_path - value2 = blocksperfile - value3 = profilesperblock - format = "int" - opObj = puObj.addOperation(name=name_operation, optype=optype) - opObj.addParameter(name=name_parameter1, value=value1) - opObj.addParameter(name=name_parameter2, value=value2, format=format) - opObj.addParameter(name=name_parameter3, value=value3, format=format) - - try: - self.refreshPUProperties(puObj) - except: - self.console.append("An error reading input parameters was found ...Check them!") - return 0 - - self.console.append("Save your project and press Play button to start signal processing") - - self._enable_play_button() - self._enable_save_button() - - return 1 - - @pyqtSignature("") - def on_volGraphClear_clicked(self): - - self.console.clear() - - """ - Voltage Graph - """ - @pyqtSignature("int") - def on_volGraphCebSave_stateChanged(self, p0): - """ - Check Box habilita ingresode del numero de Integraciones a realizar - """ - if p0 == 2: - self.volGraphPath.setEnabled(True) - self.volGraphPrefix.setEnabled(True) - self.volGraphToolPath.setEnabled(True) - - if p0 == 0: - self.volGraphPath.setEnabled(False) - self.volGraphPrefix.setEnabled(False) - self.volGraphToolPath.setEnabled(False) - - @pyqtSignature("") - def on_volGraphToolPath_clicked(self): - """ - Donde se guardan los DATOS - """ - save_path = str(QtGui.QFileDialog.getExistingDirectory(self, 'Open Directory', './', QtGui.QFileDialog.ShowDirsOnly)) - self.volGraphPath.setText(save_path) - - if not os.path.exists(save_path): - self.console.clear() - self.console.append("Set a valid path") - self.volGraphOk.setEnabled(False) - return - - @pyqtSignature("int") - def on_volGraphCebshow_stateChanged(self, p0): - """ - Check Box habilita ingresode del numero de Integraciones a realizar - """ - if p0 == 0: - - self.volGraphChannelList.setEnabled(False) - self.volGraphIntensityRange.setEnabled(False) - self.volGraphHeightrange.setEnabled(False) - if p0 == 2: - - self.volGraphChannelList.setEnabled(True) - self.volGraphIntensityRange.setEnabled(True) - self.volGraphHeightrange.setEnabled(True) - - """ - Spectra operation - """ - @pyqtSignature("int") - def on_specOpCebRadarfrequency_stateChanged(self, p0): - """ - Check Box habilita ingresode del numero de Integraciones a realizar - """ - if p0 == 2: - self.specOpRadarfrequency.setEnabled(True) - if p0 == 0: -# self.specOpRadarfrequency.clear() - self.specOpRadarfrequency.setEnabled(False) - - - @pyqtSignature("int") - def on_specOpCebCrossSpectra_stateChanged(self, p0): - """ - Habilita la opcion de a�adir el par�metro CrossSpectra a la Unidad de Procesamiento . - """ - if p0 == 2: - # self.specOpnFFTpoints.setEnabled(True) - self.specOpComCrossSpectra.setEnabled(True) - self.specOppairsList.setEnabled(True) - - if p0 == 0: - # self.specOpnFFTpoints.setEnabled(False) - self.specOpComCrossSpectra.setEnabled(False) - self.specOppairsList.setEnabled(False) - - @pyqtSignature("int") - def on_specOpCebChannel_stateChanged(self, p0): - """ - Habilita la opcion de a�adir el par�metro numero de Canales a la Unidad de Procesamiento . - """ - if p0 == 2: - self.specOpChannel.setEnabled(True) - self.specOpComChannel.setEnabled(True) - if p0 == 0: - self.specOpChannel.setEnabled(False) - self.specOpComChannel.setEnabled(False) - - @pyqtSignature("int") - def on_specOpCebHeights_stateChanged(self, p0): - """ - Habilita la opcion de a�adir el par�metro de alturas a la Unidad de Procesamiento . - """ - if p0 == 2: - self.specOpComHeights.setEnabled(True) - self.specOpHeights.setEnabled(True) - if p0 == 0: - self.specOpComHeights.setEnabled(False) - self.specOpHeights.setEnabled(False) - - - @pyqtSignature("int") - def on_specOpCebIncoherent_stateChanged(self, p0): - """ - Habilita la opcion de a�adir el par�metro integraciones incoherentes a la Unidad de Procesamiento . - """ - if p0 == 2: - self.specOpIncoherent.setEnabled(True) - if p0 == 0: - self.specOpIncoherent.setEnabled(False) - - @pyqtSignature("int") - def on_specOpCebRemoveDC_stateChanged(self, p0): - """ - Habilita la opcion de a�adir el par�metro remover DC a la Unidad de Procesamiento . - """ - if p0 == 2: - self.specOpComRemoveDC.setEnabled(True) - if p0 == 0: - self.specOpComRemoveDC.setEnabled(False) - - @pyqtSignature("int") - def on_specOpCebgetNoise_stateChanged(self, p0): - """ - Habilita la opcion de a�adir la estimacion de ruido a la Unidad de Procesamiento . - """ - if p0 == 2: - self.specOpgetNoise.setEnabled(True) - - if p0 == 0: - self.specOpgetNoise.setEnabled(False) - - @pyqtSignature("") - def on_specOpOk_clicked(self): - """ - AÑADE OPERACION SPECTRA - """ - - addFTP = False - checkPath = False - - self._disable_play_button() - self._disable_save_button() - - self.console.clear() - self.console.append("Checking input parameters:\n") - - projectObj = self.getSelectedProjectObj() - - if not projectObj: - self.console.append("Please select a project before update it") - return - - puObj = self.getSelectedItemObj() - - puObj.removeOperations() - - if self.specOpCebRadarfrequency.isChecked(): - value = str(self.specOpRadarfrequency.text()) - format = 'float' - name_operation = 'setRadarFrequency' - name_parameter = 'frequency' - - if not isFloat(value): - self.console.clear() - self.console.append("Invalid value [%s] for '%s'" %(value, name_parameter)) - return 0 - - radarfreq = float(value)*1e6 - opObj = puObj.addOperation(name=name_operation) - opObj.addParameter(name=name_parameter, value=radarfreq, format=format) - - inputId = puObj.getInputId() - inputPuObj = projectObj.getProcUnitObj(inputId) - - if inputPuObj.datatype == 'Voltage' or inputPuObj.datatype == 'USRP': - - value = str(self.specOpnFFTpoints.text()) - - if not isInt(value): - self.console.append("Invalid value [%s] for '%s'" %(value, 'nFFTPoints')) - return 0 - - puObj.addParameter(name='nFFTPoints', value=value, format='int') - - value = str(self.specOpProfiles.text()) - if not isInt(value): - self.console.append("Please write a value on Profiles field") - else: - puObj.addParameter(name='nProfiles', value=value, format='int') - - value = str(self.specOpippFactor.text()) - if not isInt(value): - self.console.append("Please write a value on IppFactor field") - else: - puObj.addParameter(name='ippFactor' , value=value , format='int') - - if self.specOpCebCrossSpectra.isChecked(): - name_parameter = 'pairsList' - format = 'pairslist' - value = str(self.specOppairsList.text()) - - if not isPairList(value): - self.console.append("Invalid value [%s] for '%s'" %(value, name_parameter)) - return 0 - - puObj.addParameter(name=name_parameter, value=value, format=format) - - if self.specOpCebHeights.isChecked(): - value = str(self.specOpHeights.text()) - - if not isFloatRange(value): - self.console.append("Invalid value [%s] for Height range" %value) - return 0 - - valueList = value.split(',') - value0 = valueList[0] - value1 = valueList[1] - - if self.specOpComHeights.currentIndex() == 0: - name_operation = 'selectHeights' - name_parameter1 = 'minHei' - name_parameter2 = 'maxHei' - else: - name_operation = 'selectHeightsByIndex' - name_parameter1 = 'minIndex' - name_parameter2 = 'maxIndex' - - format = 'float' - - opObj = puObj.addOperation(name=name_operation) - opObj.addParameter(name=name_parameter1, value=value0, format=format) - opObj.addParameter(name=name_parameter2, value=value1, format=format) - - if self.specOpCebChannel.isChecked(): - - if self.specOpComChannel.currentIndex() == 0: - name_operation = "selectChannels" - name_parameter = 'channelList' - else: - name_operation = "selectChannelsByIndex" - name_parameter = 'channelIndexList' - - format = 'intlist' - value = str(self.specOpChannel.text()) - - if not isIntList(value): - self.console.append("Invalid value [%s] for '%s'" %(value, name_parameter)) - return 0 - - opObj = puObj.addOperation(name=name_operation) - opObj.addParameter(name=name_parameter, value=value, format=format) - - if self.specOpCebIncoherent.isChecked(): - - name_operation = 'IncohInt' - optype = 'other' - - if self.specOpCobIncInt.currentIndex() == 0: - name_parameter = 'timeInterval' - format = 'float' - else: - name_parameter = 'n' - format = 'int' - - value = str(self.specOpIncoherent.text()) - - if not isFloat(value): - self.console.append("Invalid value [%s] for '%s'" %(value, name_parameter)) - return 0 - - opObj = puObj.addOperation(name=name_operation, optype=optype) - opObj.addParameter(name=name_parameter, value=value, format=format) - - if self.specOpCebRemoveDC.isChecked(): - name_operation = 'removeDC' - name_parameter = 'mode' - format = 'int' - - if self.specOpComRemoveDC.currentIndex() == 0: - value = 1 - else: - value = 2 - - opObj = puObj.addOperation(name=name_operation) - opObj.addParameter(name=name_parameter, value=value, format=format) - - if self.specOpCebRemoveInt.isChecked(): - name_operation = 'removeInterference' - opObj = puObj.addOperation(name=name_operation) - - - if self.specOpCebgetNoise.isChecked(): - value = str(self.specOpgetNoise.text()) - valueList = value.split(',') - format = 'float' - name_operation = "getNoise" - opObj = puObj.addOperation(name=name_operation) - - if not value == '': - valueList = value.split(',') - length = len(valueList) - if length == 1: - try: - value1 = float(valueList[0]) - except: - self.console.clear() - self.console.append("Please Write correct parameter Get Noise") - return 0 - name1 = 'minHei' - opObj.addParameter(name=name1, value=value1, format=format) - elif length == 2: - try: - value1 = float(valueList[0]) - value2 = float(valueList[1]) - except: - self.console.clear() - self.console.append("Please Write corrects parameter Get Noise") - return 0 - name1 = 'minHei' - name2 = 'maxHei' - opObj.addParameter(name=name1, value=value1, format=format) - opObj.addParameter(name=name2, value=value2, format=format) - - elif length == 3: - try: - value1 = float(valueList[0]) - value2 = float(valueList[1]) - value3 = float(valueList[2]) - except: - self.console.clear() - self.console.append("Please Write corrects parameter Get Noise") - return 0 - name1 = 'minHei' - name2 = 'maxHei' - name3 = 'minVel' - opObj.addParameter(name=name1, value=value1, format=format) - opObj.addParameter(name=name2, value=value2, format=format) - opObj.addParameter(name=name3, value=value3, format=format) - - elif length == 4: - try: - value1 = float(valueList[0]) - value2 = float(valueList[1]) - value3 = float(valueList[2]) - value4 = float(valueList[3]) - except: - self.console.clear() - self.console.append("Please Write corrects parameter Get Noise") - return 0 - name1 = 'minHei' - name2 = 'maxHei' - name3 = 'minVel' - name4 = 'maxVel' - opObj.addParameter(name=name1, value=value1, format=format) - opObj.addParameter(name=name2, value=value2, format=format) - opObj.addParameter(name=name3, value=value3, format=format) - opObj.addParameter(name=name4, value=value4, format=format) - - elif length > 4: - self.console.clear() - self.console.append("Get Noise Operation only accepts 4 parameters") - return 0 - - channelList = str(self.specGgraphChannelList.text()).strip() - vel_range = str(self.specGgraphFreq.text()).strip() - hei_range = str(self.specGgraphHeight.text()).strip() - db_range = str(self.specGgraphDbsrange.text()).strip() - - trange = str(self.specGgraphTminTmax.text()).strip() - magrange = str(self.specGgraphmagnitud.text()).strip() - phaserange = str(self.specGgraphPhase.text()).strip() - timerange = str(self.specGgraphTimeRange.text()).strip() - - figpath = str(self.specGraphPath.text()).strip() - figfile = str(self.specGraphPrefix.text()).strip() - - try: - wrperiod = int(str(self.specGgraphftpratio.text()).strip()) - except: - wrperiod = None - - #-----Spectra Plot----- - if self.specGraphCebSpectraplot.isChecked() or self.specGraphSaveSpectra.isChecked(): - - opObj = puObj.addOperation(name='SpectraPlot', optype='other') - opObj.addParameter(name='id', value=opObj.id, format='int') - - if channelList: - - if not isList(channelList): - self.console.append("Invalid value [%s] for 'Graphic:ChannelList" %(channelList)) - return 0 - - opObj.addParameter(name='channelList', value=channelList, format='intlist') - - if vel_range: - - if not isFloatRange(vel_range): - self.console.append("Invalid value [%s] for 'Graphic:Velocity-Range" %(vel_range)) - return 0 - - xvalueList = vel_range.split(',') - value1 = float(xvalueList[0]) - value2 = float(xvalueList[1]) - - opObj.addParameter(name='xmin', value=value1, format='float') - opObj.addParameter(name='xmax', value=value2, format='float') - - if hei_range: - - if not isFloatRange(hei_range): - self.console.append("Invalid value [%s] for 'Graphic:Height-Range" %(hei_range)) - return 0 - - yvalueList = hei_range.split(",") - value1 = float(yvalueList[0]) - value2 = float(yvalueList[1]) - - opObj.addParameter(name='ymin', value=value1, format='float') - opObj.addParameter(name='ymax', value=value2, format='float') - - if db_range: - - if not isFloatRange(db_range): - self.console.append("Invalid value [%s] for 'Graphic:dB-Range" %(db_range)) - return 0 - - zvalueList = db_range.split(",") - value1 = float(zvalueList[0]) - value2 = float(zvalueList[1]) - - opObj.addParameter(name='zmin', value=value1, format='float') - opObj.addParameter(name='zmax', value=value2, format='float') - - if not self.specGraphCebSpectraplot.isChecked(): - - opObj.addParameter(name='show', value=0 , format='bool') - - if self.specGraphSaveSpectra.isChecked(): - - checkPath = True - opObj.addParameter(name='save', value=1 , format='bool') - opObj.addParameter(name='figpath', value=figpath, format='str') - if figfile: - opObj.addParameter(name='figfile', value=figfile, format='str') - if wrperiod: - opObj.addParameter(name='wr_period', value=wrperiod,format='int') - - if self.specGraphftpSpectra.isChecked(): - - opObj.addParameter(name='ftp', value='1', format='int') - self.addFTPConf2Operation(puObj, opObj) - addFTP = True - - if self.specGraphCebCrossSpectraplot.isChecked() or self.specGraphSaveCross.isChecked(): - - opObj = puObj.addOperation(name='CrossSpectraPlot', optype='other') - opObj.addParameter(name='id', value=opObj.id, format='int') - - if vel_range: - - if not isFloatRange(vel_range): - self.console.append("Invalid value [%s] for 'Graphic:Velocity-Range" %(vel_range)) - return 0 - - xvalueList = vel_range.split(',') - value1 = float(xvalueList[0]) - value2 = float(xvalueList[1]) - - opObj.addParameter(name='xmin', value=value1, format='float') - opObj.addParameter(name='xmax', value=value2, format='float') - - if hei_range: - - if not isFloatRange(hei_range): - self.console.append("Invalid value [%s] for 'Graphic:Height-Range" %(hei_range)) - return 0 - - yvalueList = hei_range.split(",") - value1 = float(yvalueList[0]) - value2 = float(yvalueList[1]) - - opObj.addParameter(name='ymin', value=value1, format='float') - opObj.addParameter(name='ymax', value=value2, format='float') - - if db_range: - - if not isFloatRange(db_range): - self.console.append("Invalid value [%s] for 'Graphic:dB-Range" %(db_range)) - return 0 - - zvalueList = db_range.split(",") - value1 = float(zvalueList[0]) - value2 = float(zvalueList[1]) - - opObj.addParameter(name='zmin', value=value1, format='float') - opObj.addParameter(name='zmax', value=value2, format='float') - - if magrange: - - if not isFloatRange(magrange): - self.console.append("Invalid value [%s] for 'Graphic:Magnitud-Range" %(magrange)) - return 0 - - zvalueList = magrange.split(",") - value1 = float(zvalueList[0]) - value2 = float(zvalueList[1]) - - opObj.addParameter(name='coh_min', value=value1, format='float') - opObj.addParameter(name='coh_max', value=value2, format='float') - - if phaserange: - - if not isFloatRange(phaserange): - self.console.append("Invalid value [%s] for 'Graphic:Phase-Range" %(phaserange)) - return 0 - - zvalueList = phaserange.split(",") - value1 = float(zvalueList[0]) - value2 = float(zvalueList[1]) - - opObj.addParameter(name='phase_min', value=value1, format='float') - opObj.addParameter(name='phase_max', value=value2, format='float') - - if not self.specGraphCebCrossSpectraplot.isChecked(): - - opObj.addParameter(name='show', value=0 , format='bool') - - if self.specGraphSaveCross.isChecked(): - - checkPath = True - opObj.addParameter(name='save', value='1', format='bool') - opObj.addParameter(name='figpath', value=figpath, format='str') - if figfile: - opObj.addParameter(name='figfile', value=figfile, format='str') - if wrperiod: - opObj.addParameter(name='wr_period', value=wrperiod,format='int') - - if self.specGraphftpCross.isChecked(): - - opObj.addParameter(name='ftp', value='1', format='int') - self.addFTPConf2Operation(puObj, opObj) - addFTP = True - - if self.specGraphCebRTIplot.isChecked() or self.specGraphSaveRTIplot.isChecked(): - - opObj = puObj.addOperation(name='RTIPlot', optype='other') - opObj.addParameter(name='id', value=opObj.id, format='int') - - if channelList: - - if not isIntList(channelList): - self.console.append("Invalid value [%s] for 'Graphic:ChannelList" %(channelList)) - return 0 - - opObj.addParameter(name='channelList', value=channelList, format='intlist') - - if trange: - - if not isFloatRange(trange): - self.console.append("Invalid value [%s] for 'Graphic:Time-Range" %(trange)) - return 0 - - zvalueList = trange.split(",") - value1 = float(zvalueList[0]) - value2 = float(zvalueList[1]) - - opObj.addParameter(name='xmin', value=value1, format='float') - opObj.addParameter(name='xmax', value=value2, format='float') - - if timerange: - try: - timerange = int(timerange) - except: - return 0 - opObj.addParameter(name='timerange', value=timerange, format='int') - - if hei_range: - - if not isFloatRange(hei_range): - self.console.append("Invalid value [%s] for 'Graphic:Height-Range" %(hei_range)) - return 0 - - yvalueList = hei_range.split(",") - value1 = float(yvalueList[0]) - value2 = float(yvalueList[1]) - - opObj.addParameter(name='ymin', value=value1, format='float') - opObj.addParameter(name='ymax', value=value2, format='float') - - if db_range: - - if not isFloatRange(db_range): - self.console.append("Invalid value [%s] for 'Graphic:dB-Range" %(db_range)) - return 0 - - zvalueList = db_range.split(",") - value1 = float(zvalueList[0]) - value2 = float(zvalueList[1]) - - opObj.addParameter(name='zmin', value=value1, format='float') - opObj.addParameter(name='zmax', value=value2, format='float') - - if not self.specGraphCebRTIplot.isChecked(): - - opObj.addParameter(name='show', value=0 , format='bool') - - if self.specGraphSaveRTIplot.isChecked(): - - checkPath = True - opObj.addParameter(name='save', value='1', format='bool') - opObj.addParameter(name='figpath', value=figpath, format='str') - if figfile: - opObj.addParameter(name='figfile', value=value, format='str') - if wrperiod: - opObj.addParameter(name='wr_period', value=wrperiod,format='int') - - if self.specGraphftpRTIplot.isChecked(): - - opObj.addParameter(name='ftp', value='1', format='int') - self.addFTPConf2Operation(puObj, opObj) - addFTP = True - - if self.specGraphCebCoherencmap.isChecked() or self.specGraphSaveCoherencemap.isChecked(): - - opObj = puObj.addOperation(name='CoherenceMap', optype='other') - opObj.addParameter(name='id', value=opObj.id, format='int') - - if trange: - - if not isFloatRange(trange): - self.console.append("Invalid value [%s] for 'Graphic:Time-Range" %(trange)) - return 0 - - zvalueList = trange.split(",") - value1 = float(zvalueList[0]) - value2 = float(zvalueList[1]) - - opObj.addParameter(name='xmin', value=value1, format='float') - opObj.addParameter(name='xmax', value=value2, format='float') - - if hei_range: - - if not isFloatRange(hei_range): - self.console.append("Invalid value [%s] for 'Graphic:Height-Range" %(hei_range)) - return 0 - - yvalueList = hei_range.split(",") - value1 = float(yvalueList[0]) - value2 = float(yvalueList[1]) - - opObj.addParameter(name='ymin', value=value1, format='float') - opObj.addParameter(name='ymax', value=value2, format='float') - - if magrange: - - if not isFloatRange(magrange): - self.console.append("Invalid value [%s] for 'Graphic:Magnitud-Range" %(magrange)) - return 0 - - zvalueList = magrange.split(",") - value1 = float(zvalueList[0]) - value2 = float(zvalueList[1]) - - opObj.addParameter(name='zmin', value=value1, format='float') - opObj.addParameter(name='zmax', value=value2, format='float') - - if phaserange: - - if not isFloatRange(phaserange): - self.console.append("Invalid value [%s] for 'Graphic:Phase-Range" %(phaserange)) - return 0 - - zvalueList = phaserange.split(",") - value1 = float(zvalueList[0]) - value2 = float(zvalueList[1]) - - opObj.addParameter(name='phase_min', value=value1, format='float') - opObj.addParameter(name='phase_max', value=value2, format='float') - - if not self.specGraphCebCoherencmap.isChecked(): - opObj.addParameter(name='show', value=0 , format='bool') - - if self.specGraphSaveCoherencemap.isChecked(): - checkPath = True - opObj.addParameter(name='save', value='1', format='bool') - opObj.addParameter(name='figpath', value=figpath, format='str') - if figfile: - opObj.addParameter(name='figfile', value=value, format='str') - if wrperiod: - opObj.addParameter(name='wr_period', value=wrperiod,format='int') - - if self.specGraphftpCoherencemap.isChecked(): - opObj.addParameter(name='ftp', value='1', format='int') - self.addFTPConf2Operation(puObj, opObj) - addFTP = True - - if self.specGraphPowerprofile.isChecked() or self.specGraphSavePowerprofile.isChecked(): - - opObj = puObj.addOperation(name='PowerProfilePlot', optype='other') - opObj.addParameter(name='id', value=opObj.id, format='int') - - if channelList: - - if not isList(channelList): - self.console.append("Invalid value [%s] for 'Graphic:ChannelList" %(channelList)) - return 0 - - opObj.addParameter(name='channelList', value=channelList, format='intlist') - - if hei_range: - - if not isFloatRange(hei_range): - self.console.append("Invalid value [%s] for 'Graphic:Height-Range" %(hei_range)) - return 0 - - yvalueList = hei_range.split(",") - value1 = float(yvalueList[0]) - value2 = float(yvalueList[1]) - - opObj.addParameter(name='ymin', value=value1, format='float') - opObj.addParameter(name='ymax', value=value2, format='float') - - if db_range: - - if not isFloatRange(db_range): - self.console.append("Invalid value [%s] for 'Graphic:dB-Range" %(db_range)) - return 0 - - zvalueList = db_range.split(",") - value1 = float(zvalueList[0]) - value2 = float(zvalueList[1]) - - opObj.addParameter(name='xmin', value=value1, format='float') - opObj.addParameter(name='xmax', value=value2, format='float') - - if not self.specGraphPowerprofile.isChecked(): - opObj.addParameter(name='show', value=0 , format='bool') - - if self.specGraphSavePowerprofile.isChecked(): - checkPath = True - opObj.addParameter(name='save', value='1', format='bool') - opObj.addParameter(name='figpath', value=figpath, format='str') - if figfile: - opObj.addParameter(name='figfile', value=value, format='str') - if wrperiod: - opObj.addParameter(name='wr_period', value=wrperiod,format='int') - - if self.specGraphftpPowerprofile.isChecked(): - opObj.addParameter(name='ftp', value='1', format='int') - self.addFTPConf2Operation(puObj, opObj) - addFTP = True - # rti noise - - if self.specGraphCebRTInoise.isChecked() or self.specGraphSaveRTInoise.isChecked(): - - opObj = puObj.addOperation(name='Noise', optype='other') - opObj.addParameter(name='id', value=opObj.id, format='int') - - if channelList: - - if not isList(channelList): - self.console.append("Invalid value [%s] for 'Graphic:ChannelList" %(channelList)) - return 0 - - opObj.addParameter(name='channelList', value=channelList, format='intlist') - - if trange: - - if not isFloatRange(trange): - self.console.append("Invalid value [%s] for 'Graphic:Time-Range" %(trange)) - return 0 - - zvalueList = trange.split(",") - value1 = float(zvalueList[0]) - value2 = float(zvalueList[1]) - - opObj.addParameter(name='xmin', value=value1, format='float') - opObj.addParameter(name='xmax', value=value2, format='float') - - if db_range: - - if not isFloatRange(db_range): - self.console.append("Invalid value [%s] for 'Graphic:dB-Range" %(db_range)) - return 0 - - zvalueList = db_range.split(",") - value1 = float(zvalueList[0]) - value2 = float(zvalueList[1]) - - opObj.addParameter(name='ymin', value=value1, format='float') - opObj.addParameter(name='ymax', value=value2, format='float') - - if not self.specGraphCebRTInoise.isChecked(): - opObj.addParameter(name='show', value=0 , format='bool') - - if self.specGraphSaveRTInoise.isChecked(): - checkPath = True - opObj.addParameter(name='save', value='1', format='bool') - opObj.addParameter(name='figpath', value=figpath, format='str') - if figfile: - opObj.addParameter(name='figfile', value=value, format='str') - if wrperiod: - opObj.addParameter(name='wr_period', value=wrperiod,format='int') - - # test_ftp - if self.specGraphftpRTInoise.isChecked(): - opObj.addParameter(name='ftp', value='1', format='int') - self.addFTPConf2Operation(puObj, opObj) - addFTP = True - - if checkPath: - if not figpath: - self.console.clear() - self.console.append("Graphic path should be defined") - return 0 - - if addFTP and not figpath: - self.console.clear() - self.console.append("You have to save the plots before sending them to FTP Server") - return 0 - - self.console.clear() - -# if something happend - parms_ok, output_path, blocksperfile, profilesperblock = self.checkInputsPUSave(datatype='Spectra') - if parms_ok: - opObj = puObj.addOperation(name='SpectraWriter', optype='other') - opObj.addParameter(name='path', value=output_path) - opObj.addParameter(name='blocksPerFile', value=blocksperfile, format='int') - - try: - self.refreshPUProperties(puObj) - except: - self.console.append("An error reading input parameters was found ... Check them!") - return 0 - - self.console.append("Save your project and press Play button to start signal processing") - - self._enable_play_button() - self._enable_save_button() - - return 1 - - - @pyqtSignature("") - def on_specGraphClear_clicked(self): - - self.console.clear() - - """ - Spectra Graph - """ - @pyqtSignature("int") - def on_specGraphCebSpectraplot_stateChanged(self, p0): - - self.__checkSpecGraphFilters() - - - @pyqtSignature("int") - def on_specGraphCebCrossSpectraplot_stateChanged(self, p0): - - self.__checkSpecGraphFilters() - - @pyqtSignature("int") - def on_specGraphCebRTIplot_stateChanged(self, p0): - - self.__checkSpecGraphFilters() - - - @pyqtSignature("int") - def on_specGraphCebRTInoise_stateChanged(self, p0): - - self.__checkSpecGraphFilters() - - - @pyqtSignature("int") - def on_specGraphCebCoherencmap_stateChanged(self, p0): - - self.__checkSpecGraphFilters() - - @pyqtSignature("int") - def on_specGraphPowerprofile_stateChanged(self, p0): - - self.__checkSpecGraphFilters() - - @pyqtSignature("int") - def on_specGraphPhase_stateChanged(self, p0): - - self.__checkSpecGraphFilters() - - @pyqtSignature("int") - def on_specGraphSaveSpectra_stateChanged(self, p0): - """ - """ - self.__checkSpecGraphFilters() - self.__checkSpecGraphSaving() - - @pyqtSignature("int") - def on_specGraphSaveCross_stateChanged(self, p0): - - self.__checkSpecGraphFilters() - self.__checkSpecGraphSaving() - - @pyqtSignature("int") - def on_specGraphSaveRTIplot_stateChanged(self, p0): - - self.__checkSpecGraphFilters() - self.__checkSpecGraphSaving() - - @pyqtSignature("int") - def on_specGraphSaveRTInoise_stateChanged(self, p0): - - self.__checkSpecGraphFilters() - self.__checkSpecGraphSaving() - - @pyqtSignature("int") - def on_specGraphSaveCoherencemap_stateChanged(self, p0): - - self.__checkSpecGraphFilters() - self.__checkSpecGraphSaving() - - @pyqtSignature("int") - def on_specGraphSavePowerprofile_stateChanged(self, p0): - - self.__checkSpecGraphFilters() - self.__checkSpecGraphSaving() - - @pyqtSignature("int") - def on_specGraphftpSpectra_stateChanged(self, p0): - """ - """ - self.__checkSpecGraphFTP() - - - @pyqtSignature("int") - def on_specGraphftpCross_stateChanged(self, p0): - - self.__checkSpecGraphFTP() - - @pyqtSignature("int") - def on_specGraphftpRTIplot_stateChanged(self, p0): - - self.__checkSpecGraphFTP() - - @pyqtSignature("int") - def on_specGraphftpRTInoise_stateChanged(self, p0): - - self.__checkSpecGraphFTP() - - @pyqtSignature("int") - def on_specGraphftpCoherencemap_stateChanged(self, p0): - - self.__checkSpecGraphFTP() - - @pyqtSignature("int") - def on_specGraphftpPowerprofile_stateChanged(self, p0): - - self.__checkSpecGraphFTP() - - @pyqtSignature("") - def on_specGraphToolPath_clicked(self): - """ - """ - save_path = str(QtGui.QFileDialog.getExistingDirectory(self, 'Open Directory', './', QtGui.QFileDialog.ShowDirsOnly)) - self.specGraphPath.setText(save_path) - if not os.path.exists(save_path): - self.console.clear() - self.console.append("Write a valid path") - return - - @pyqtSignature("") - def on_specHeisGraphToolPath_clicked(self): - """ - """ - save_path = str(QtGui.QFileDialog.getExistingDirectory(self, 'Open Directory', './', QtGui.QFileDialog.ShowDirsOnly)) - self.specHeisGraphPath.setText(save_path) - if not os.path.exists(save_path): - self.console.clear() - self.console.append("Write a valid path") - return - - @pyqtSignature("int") - def on_specHeisOpCebIncoherent_stateChanged(self, p0): - """ - Habilita la opcion de a�adir el par�metro integraciones incoherentes a la Unidad de Procesamiento . - """ - if p0 == 2: - self.specHeisOpIncoherent.setEnabled(True) - self.specHeisOpCobIncInt.setEnabled(True) - if p0 == 0: - self.specHeisOpIncoherent.setEnabled(False) - self.specHeisOpCobIncInt.setEnabled(False) - - @pyqtSignature("") - def on_specHeisOpOk_clicked(self): - """ - AÑADE OPERACION SPECTRAHEIS - """ - addFTP = False - checkPath = False - - self._disable_play_button() - self._disable_save_button() - - self.console.clear() - self.console.append("Checking input parameters ...") - - puObj = self.getSelectedItemObj() - puObj.removeOperations() - - if self.specHeisOpCebIncoherent.isChecked(): - value = str(self.specHeisOpIncoherent.text()) - name_operation = 'IncohInt4SpectraHeis' - optype = 'other' - - name_parameter = 'timeInterval' - format = 'float' - - if self.specOpCobIncInt.currentIndex() == 0: - name_parameter = 'timeInterval' - format = 'float' - - if not isFloat(value): - self.console.append("Invalid value '%s' for '%s'" %(value, name_parameter)) - return 0 - - opObj = puObj.addOperation(name=name_operation, optype=optype) - - if not opObj.addParameter(name=name_parameter, value=value, format=format): - self.console.append("Invalid value '%s' for '%s'" %(value, name_parameter)) - return 0 - - channelList = str(self.specHeisGgraphChannelList.text()) - freq_range = str(self.specHeisGgraphXminXmax.text()) - power_range = str(self.specHeisGgraphYminYmax.text()) - time_range = str(self.specHeisGgraphTminTmax.text()) - timerange = str(self.specHeisGgraphTimeRange.text()) - - if self.specHeisGraphCebSpectraplot.isChecked(): - - name_operation = 'SpectraHeisScope' - optype = 'other' - opObj = puObj.addOperation(name=name_operation, optype=optype) - - name_parameter = 'id' - format = 'int' - value = opObj.id - - if not opObj.addParameter(name=name_parameter, value=value, format=format): - self.console.append("Invalid value '%s' for '%s'" %(value, name_parameter)) - return 0 - - if not (channelList == ''): - name_parameter = 'channelList' - format = 'intlist' - - if not isList(channelList): - self.console.append("Invalid value '%s' for '%s'" %(channelList, name_parameter)) - return 0 - - opObj.addParameter(name=name_parameter, value=channelList, format=format) - - if not freq_range == '': - xvalueList = freq_range.split(',') - - if len(xvalueList) != 2: - self.console.append("Invalid value '%s' for '%s'" %(freq_range, "xrange")) - return 0 - - value1 = xvalueList[0] - value2 = xvalueList[1] - - if not isFloat(value1) or not isFloat(value2): - self.console.append("Invalid value '%s' for '%s'" %(freq_range, "xrange")) - return 0 - - name1 = 'xmin' - name2 = 'xmax' - format = 'float' - - opObj.addParameter(name=name1, value=value1, format=format) - opObj.addParameter(name=name2, value=value2, format=format) - - if not power_range == '': - yvalueList = power_range.split(",") - - if len(yvalueList) != 2: - self.console.append("Invalid value '%s' for '%s'" %(power_range, "xrange")) - return 0 - - value1 = yvalueList[0] - value2 = yvalueList[1] - - if not isFloat(value1) or not isFloat(value2): - self.console.append("Invalid value '%s' for '%s'" %(power_range, "yrange")) - return 0 - - name1 = 'ymin' - name2 = 'ymax' - format = 'float' - opObj.addParameter(name=name1, value=value1, format=format) - opObj.addParameter(name=name2, value=value2, format=format) - - if self.specHeisGraphSaveSpectra.isChecked(): - checkPath = True - name_parameter1 = 'save' - name_parameter2 = 'figpath' - name_parameter3 = 'figfile' - value1 = '1' - value2 = str(self.specHeisGraphPath.text()) - value3 = str(self.specHeisGraphPrefix.text()) - format1 = 'bool' - format2 = 'str' - opObj.addParameter(name=name_parameter1, value=value1 , format=format1) - opObj.addParameter(name=name_parameter2, value=value2, format=format2) - if not value3 == "": - try: - value3 = str(self.specHeisGraphPrefix.text()) - except: - self.console.clear() - self.console.append("Please Write prefix") - return 0 - opObj.addParameter(name='figfile', value=str(self.specHeisGraphPrefix.text()), format='str') - - # opObj.addParameter(name=name_parameter3, value=value3, format=format2) - # opObj.addParameter(name='wr_period', value='5',format='int') - - if self.specHeisGraphftpSpectra.isChecked(): - opObj.addParameter(name='ftp', value='1', format='int') - self.addFTPConf2Operation(puObj, opObj) - addFTP = True - - if self.specHeisGraphCebRTIplot.isChecked(): - name_operation = 'RTIfromSpectraHeis' - optype = 'other' - - name_parameter = 'id' - format = 'int' - - opObj = puObj.addOperation(name=name_operation, optype=optype) - value = opObj.id - opObj.addParameter(name=name_parameter, value=value, format=format) - - if not channelList == '': - opObj.addParameter(name='channelList', value=channelList, format='intlist') - - if not time_range == '': - xvalueList = time_range.split(',') - try: - value = float(xvalueList[0]) - value = float(xvalueList[1]) - except: - return 0 - format = 'float' - opObj.addParameter(name='xmin', value=xvalueList[0], format=format) - opObj.addParameter(name='xmax', value=xvalueList[1], format=format) - - if not timerange == '': - format = 'int' - try: - timerange = int(timerange) - except: - return 0 - opObj.addParameter(name='timerange', value=timerange, format=format) - - - if not power_range == '': - yvalueList = power_range.split(",") - try: - value = float(yvalueList[0]) - value = float(yvalueList[1]) - except: - return 0 - - format = 'float' - opObj.addParameter(name='ymin', value=yvalueList[0], format=format) - opObj.addParameter(name='ymax', value=yvalueList[1], format=format) - - if self.specHeisGraphSaveRTIplot.isChecked(): - checkPath = True - opObj.addParameter(name='save', value='1', format='bool') - opObj.addParameter(name='figpath', value=str(self.specHeisGraphPath.text()), format='str') - value = str(self.specHeisGraphPrefix.text()) - if not value == "": - try: - value = str(self.specHeisGraphPrefix.text()) - except: - self.console.clear() - self.console.append("Please Write prefix") - return 0 - opObj.addParameter(name='figfile', value=value, format='str') - - # test_ftp - if self.specHeisGraphftpRTIplot.isChecked(): - opObj.addParameter(name='ftp', value='1', format='int') - self.addFTPConf2Operation(puObj, opObj) - addFTP = True - - localfolder = None - if checkPath: - localfolder = str(self.specHeisGraphPath.text()) - if localfolder == '': - self.console.clear() - self.console.append("Graphic path should be defined") - return 0 - - if addFTP and not localfolder: - self.console.clear() - self.console.append("You should save plots before send them to FTP Server") - return 0 - - # if something happened - parms_ok, output_path, blocksperfile, metadata_file = self.checkInputsPUSave(datatype='SpectraHeis') - if parms_ok: - name_operation = 'FitsWriter' - optype = 'other' - name_parameter1 = 'path' - name_parameter2 = 'dataBlocksPerFile' - name_parameter3 = 'metadatafile' - value1 = output_path - value2 = blocksperfile - value3 = metadata_file - format2 = "int" - format3 = "str" - opObj = puObj.addOperation(name=name_operation, optype=optype) - - opObj.addParameter(name=name_parameter1, value=value1) - - if blocksperfile: - opObj.addParameter(name=name_parameter2, value=value2, format=format2) - - if metadata_file: - opObj.addParameter(name=name_parameter3, value=value3, format=format3) - - self.console.clear() - try: - self.refreshPUProperties(puObj) - except: - self.console.append("An error reading input parameters was found ... Check them!") - return 0 - - self.console.append("Save your project and press Play button to start signal processing") - - self._enable_save_button() - self._enable_play_button() - - return 1 - - @pyqtSignature("") - def on_specHeisGraphClear_clicked(self): - - self.console.clear() - - @pyqtSignature("int") - def on_specHeisGraphCebSpectraplot_stateChanged(self, p0): - - if p0 == 2: - self.specHeisGgraphChannelList.setEnabled(True) - self.specHeisGgraphXminXmax.setEnabled(True) - self.specHeisGgraphYminYmax.setEnabled(True) - if p0 == 0: - self.specHeisGgraphXminXmax.setEnabled(False) - self.specHeisGgraphYminYmax.setEnabled(False) - - @pyqtSignature("int") - def on_specHeisGraphCebRTIplot_stateChanged(self, p0): - - if p0 == 2: - self.specHeisGgraphChannelList.setEnabled(True) - self.specHeisGgraphTminTmax.setEnabled(True) - self.specHeisGgraphYminYmax.setEnabled(True) - self.specHeisGgraphTimeRange.setEnabled(True) - - if p0 == 0: - self.specHeisGgraphTminTmax.setEnabled(False) - self.specHeisGgraphYminYmax.setEnabled(False) - self.specHeisGgraphTimeRange.setEnabled(False) - - @pyqtSignature("int") - def on_specHeisGraphSaveSpectra_stateChanged(self, p0): - """ - """ - if p0 == 2: - self.specHeisGraphPath.setEnabled(True) - self.specHeisGraphPrefix.setEnabled(True) - self.specHeisGraphToolPath.setEnabled(True) - if p0 == 0: - self.specHeisGraphPath.setEnabled(False) - self.specHeisGraphPrefix.setEnabled(False) - self.specHeisGraphToolPath.setEnabled(False) - - @pyqtSignature("int") - def on_specHeisGraphSaveRTIplot_stateChanged(self, p0): - if p0 == 2: - self.specHeisGraphPath.setEnabled(True) - self.specHeisGraphPrefix.setEnabled(True) - self.specHeisGraphToolPath.setEnabled(True) - - @pyqtSignature("int") - def on_specHeisGraphftpSpectra_stateChanged(self, p0): - """ - """ - if p0 == 2: - self.specHeisGgraphftpratio.setEnabled(True) - - if p0 == 0: - self.specHeisGgraphftpratio.setEnabled(False) - - @pyqtSignature("int") - def on_specHeisGraphftpRTIplot_stateChanged(self, p0): - if p0 == 2: - self.specHeisGgraphftpratio.setEnabled(True) - - def __checkSpecGraphSaving(self): - - enable = False - - if self.specGraphSaveSpectra.checkState(): - enable = True - - if self.specGraphSaveCross.checkState(): - enable = True - - if self.specGraphSaveRTIplot.checkState(): - enable = True - - if self.specGraphSaveCoherencemap.checkState(): - enable = True - - if self.specGraphSavePowerprofile.checkState(): - enable = True - - if self.specGraphSaveRTInoise.checkState(): - enable = True - - self.specGraphPath.setEnabled(enable) - self.specGraphPrefix.setEnabled(enable) - self.specGraphToolPath.setEnabled(enable) - - self.specGgraphftpratio.setEnabled(enable) - - def __checkSpecGraphFTP(self): - - enable = False - - if self.specGraphftpSpectra.checkState(): - enable = True - - if self.specGraphftpCross.checkState(): - enable = True - - if self.specGraphftpRTIplot.checkState(): - enable = True - - if self.specGraphftpCoherencemap.checkState(): - enable = True - - if self.specGraphftpPowerprofile.checkState(): - enable = True - - if self.specGraphftpRTInoise.checkState(): - enable = True - -# self.specGgraphftpratio.setEnabled(enable) - - def __checkSpecGraphFilters(self): - - freq = False - height = False - db = False - timerange = False - magnitud = False - phase = False - channelList = False - - if self.specGraphCebSpectraplot.checkState() or self.specGraphSaveSpectra.checkState(): - freq = True - height = True - db = True - channelList = True - - if self.specGraphCebCrossSpectraplot.checkState() or self.specGraphSaveCross.checkState(): - freq = True - height = True - db = True - magnitud = True - phase = True - - if self.specGraphCebRTIplot.checkState() or self.specGraphSaveRTIplot.checkState(): - height = True - db = True - timerange = True - channelList = True - - if self.specGraphCebCoherencmap.checkState() or self.specGraphSaveCoherencemap.checkState(): - height = True - timerange = True - magnitud = True - phase = True - - if self.specGraphPowerprofile.checkState() or self.specGraphSavePowerprofile.checkState(): - height = True - db = True - channelList = True - - if self.specGraphCebRTInoise.checkState() or self.specGraphSaveRTInoise.checkState(): - db = True - timerange = True - channelList = True - - - self.specGgraphFreq.setEnabled(freq) - self.specGgraphHeight.setEnabled(height) - self.specGgraphDbsrange.setEnabled(db) - self.specGgraphTminTmax.setEnabled(timerange) - self.specGgraphTimeRange.setEnabled(timerange) - - self.specGgraphmagnitud.setEnabled(magnitud) - self.specGgraphPhase.setEnabled(phase) - self.specGgraphChannelList.setEnabled(channelList) - - def __getParmsFromProjectWindow(self): - """ - Check Inputs Project: - - project_name - - datatype - - ext - - data_path - - readmode - - delay - - set - - walk - """ - parms_ok = True - - project_name = str(self.proName.text()) - - if project_name == '' or project_name == None: - outputstr = "Enter a project Name" - self.console.append(outputstr) - parms_ok = False - project_name = None - - description = str(self.proDescription.toPlainText()) - - datatype = str(self.proComDataType.currentText()) - - ext = str(self.proDataType.text()) - - dpath = str(self.proDataPath.text()) - - if dpath == '': - outputstr = 'Datapath is empty' - self.console.append(outputstr) - parms_ok = False - dpath = None - - if dpath != None: - if not os.path.isdir(dpath): - outputstr = 'Datapath (%s) does not exist' % dpath - self.console.append(outputstr) - parms_ok = False - dpath = None - - online = int(self.proComReadMode.currentIndex()) - - delay = None - if online==1: - try: - delay = int(str(self.proDelay.text())) - except: - outputstr = 'Delay value (%s) must be a integer number' %str(self.proDelay.text()) - self.console.append(outputstr) - parms_ok = False - - - set = None - value = str(self.proSet.text()) - try: - set = int(value) - except: - pass - - ippKm = None - - value = str(self.proIPPKm.text()) - - try: - ippKm = float(value) - except: - if datatype=="USRP": - outputstr = 'IPP value "%s" must be a float number' % str(self.proIPPKm.text()) - self.console.append(outputstr) - parms_ok = False - - walk = int(self.proComWalk.currentIndex()) - expLabel = str(self.proExpLabel.text()) - - startDate = str(self.proComStartDate.currentText()).strip() - endDate = str(self.proComEndDate.currentText()).strip() - - if not startDate: - parms_ok = False - - if not endDate: - parms_ok = False - -# startDateList = startDate.split("/") -# endDateList = endDate.split("/") -# -# startDate = datetime.date(int(startDateList[0]), int(startDateList[1]), int(startDateList[2])) -# endDate = datetime.date(int(endDateList[0]), int(endDateList[1]), int(endDateList[2])) - - startTime = self.proStartTime.time() - endTime = self.proEndTime.time() - - startTime = str(startTime.toString("H:m:s")) - endTime = str(endTime.toString("H:m:s")) - - projectParms = ProjectParms() - - projectParms.name = project_name - projectParms.description = description - projectParms.datatype = datatype - projectParms.ext = ext - projectParms.dpath = dpath - projectParms.online = online - projectParms.startDate = startDate - projectParms.endDate = endDate - projectParms.startTime = startTime - projectParms.endTime = endTime - projectParms.delay = delay - projectParms.walk = walk - projectParms.expLabel = expLabel - projectParms.set = set - projectParms.ippKm = ippKm - projectParms.parmsOk = parms_ok - - return projectParms - - - def __getParmsFromProjectObj(self, projectObjView): - - parms_ok = True - - project_name, description = projectObjView.name, projectObjView.description - - readUnitObj = projectObjView.getReadUnitObj() - datatype = readUnitObj.datatype - - operationObj = readUnitObj.getOperationObj(name='run') - - dpath = operationObj.getParameterValue(parameterName='path') - startDate = operationObj.getParameterValue(parameterName='startDate') - endDate = operationObj.getParameterValue(parameterName='endDate') - - startDate = startDate.strftime("%Y/%m/%d") - endDate = endDate.strftime("%Y/%m/%d") - - startTime = operationObj.getParameterValue(parameterName='startTime') - endTime = operationObj.getParameterValue(parameterName='endTime') - - startTime = startTime.strftime("%H:%M:%S") - endTime = endTime.strftime("%H:%M:%S") - - online = 0 - try: - online = operationObj.getParameterValue(parameterName='online') - except: - pass - - delay = '' - try: - delay = operationObj.getParameterValue(parameterName='delay') - except: - pass - - walk = 0 - try: - walk = operationObj.getParameterValue(parameterName='walk') - except: - pass - - set = '' - try: - set = operationObj.getParameterValue(parameterName='set') - except: - pass - - expLabel = '' - try: - expLabel = operationObj.getParameterValue(parameterName='expLabel') - except: - pass - - ippKm = '' - if datatype.lower() == 'usrp': - try: - ippKm = operationObj.getParameterValue(parameterName='ippKm') - except: - pass - - projectParms = ProjectParms() - - projectParms.name = project_name - projectParms.description = description - projectParms.datatype = datatype - projectParms.ext = None - projectParms.dpath = dpath - projectParms.online = online - projectParms.startDate = startDate - projectParms.endDate = endDate - projectParms.startTime = startTime - projectParms.endTime = endTime - projectParms.delay=delay - projectParms.walk=walk - projectParms.set=set - projectParms.ippKm=ippKm - projectParms.expLabel = expLabel - projectParms.parmsOk=parms_ok - - return projectParms - - def refreshProjectWindow(self, projectObjView): - - self.proOk.setEnabled(False) - - projectParms = self.__getParmsFromProjectObj(projectObjView) - - index = projectParms.getDatatypeIndex() - - self.proName.setText(projectParms.name) - self.proDescription.clear() - self.proDescription.append(projectParms.description) - - self.on_proComDataType_activated(index=index) - self.proDataPath.setText(projectParms.dpath) - self.proComDataType.setCurrentIndex(index) - self.proComReadMode.setCurrentIndex(projectParms.online) - self.proDelay.setText(str(projectParms.delay)) - self.proSet.setText(str(projectParms.set)) - self.proIPPKm.setText(str(projectParms.ippKm)) - self.proComWalk.setCurrentIndex(projectParms.walk) - self.proExpLabel.setText(str(projectParms.expLabel).strip()) - - self.on_proComReadMode_activated(projectParms.online) - self.on_proComWalk_activated(projectParms.walk) - - dateList = self.loadDays(data_path = projectParms.dpath, - ext = projectParms.getExt(), - walk = projectParms.walk, - expLabel = projectParms.expLabel) - - if not dateList: - return 0 - - try: - startDateIndex = dateList.index(projectParms.startDate) - except: - startDateIndex = 0 - - try: - endDateIndex = dateList.index(projectParms.endDate) - except: - endDateIndex = int(self.proComEndDate.count()-1) - - self.proComStartDate.setCurrentIndex(startDateIndex) - self.proComEndDate.setCurrentIndex(endDateIndex) - - startlist = projectParms.startTime.split(":") - endlist = projectParms.endTime.split(":") - - self.time.setHMS(int(startlist[0]), int(startlist[1]), int(startlist[2])) - self.proStartTime.setTime(self.time) - - self.time.setHMS(int(endlist[0]), int(endlist[1]), int(endlist[2])) - self.proEndTime.setTime(self.time) - - self.proOk.setEnabled(True) - - return 1 - - def __refreshVoltageWindow(self, puObj): - - opObj = puObj.getOperationObj(name='setRadarFrequency') - if opObj == None: - self.volOpRadarfrequency.clear() - self.volOpCebRadarfrequency.setCheckState(0) - else: - value = opObj.getParameterValue(parameterName='frequency') - value = str(float(value)/1e6) - self.volOpRadarfrequency.setText(value) - self.volOpRadarfrequency.setEnabled(True) - self.volOpCebRadarfrequency.setCheckState(QtCore.Qt.Checked) - - opObj = puObj.getOperationObj(name="selectChannels") - - if opObj == None: - opObj = puObj.getOperationObj(name="selectChannelsByIndex") - - if opObj == None: - self.volOpChannel.clear() - self.volOpCebChannels.setCheckState(0) - else: - channelEnabled = False - try: - value = opObj.getParameterValue(parameterName='channelList') - value = str(value)[1:-1] - channelEnabled = True - channelMode = 0 - except: - pass - try: - value = opObj.getParameterValue(parameterName='channelIndexList') - value = str(value)[1:-1] - channelEnabled = True - channelMode = 1 - except: - pass - - if channelEnabled: - self.volOpChannel.setText(value) - self.volOpChannel.setEnabled(True) - self.volOpCebChannels.setCheckState(QtCore.Qt.Checked) - self.volOpComChannels.setCurrentIndex(channelMode) - - opObj = puObj.getOperationObj(name="selectHeights") - if opObj == None: - self.volOpHeights.clear() - self.volOpCebHeights.setCheckState(0) - else: - value1 = str(opObj.getParameterValue(parameterName='minHei')) - value2 = str(opObj.getParameterValue(parameterName='maxHei')) - value = value1 + "," + value2 - self.volOpHeights.setText(value) - self.volOpHeights.setEnabled(True) - self.volOpCebHeights.setCheckState(QtCore.Qt.Checked) - - opObj = puObj.getOperationObj(name="SplitProfiles") - if opObj == None: - self.volOpSplitter.clear() - self.volOpCebSplitter.setCheckState(0) - else: - value = opObj.getParameterValue(parameterName='n') - value = str(value) - self.volOpSplitter.setText(value) - self.volOpSplitter.setEnabled(True) - self.volOpCebSplitter.setCheckState(QtCore.Qt.Checked) - - opObj = puObj.getOperationObj(name="CombineProfiles") - if opObj == None: - self.volOpCombiner.clear() - self.volOpCebCombiner.setCheckState(0) - else: - value = opObj.getParameterValue(parameterName='n') - value = str(value) - self.volOpCombiner.setText(value) - self.volOpCombiner.setEnabled(True) - self.volOpCebCombiner.setCheckState(QtCore.Qt.Checked) - - opObj = puObj.getOperationObj(name="filterByHeights") - if opObj == None: - self.volOpFilter.clear() - self.volOpCebFilter.setCheckState(0) - else: - value = opObj.getParameterValue(parameterName='window') - value = str(value) - self.volOpFilter.setText(value) - self.volOpFilter.setEnabled(True) - self.volOpCebFilter.setCheckState(QtCore.Qt.Checked) - - opObj = puObj.getOperationObj(name="ProfileSelector") - if opObj == None: - self.volOpProfile.clear() - self.volOpCebProfile.setCheckState(0) - else: - for parmObj in opObj.getParameterObjList(): - - if parmObj.name == "profileList": - value = parmObj.getValue() - value = str(value)[1:-1] - self.volOpProfile.setText(value) - self.volOpProfile.setEnabled(True) - self.volOpCebProfile.setCheckState(QtCore.Qt.Checked) - self.volOpComProfile.setCurrentIndex(0) - - if parmObj.name == "profileRangeList": - value = parmObj.getValue() - value = str(value)[1:-1] - self.volOpProfile.setText(value) - self.volOpProfile.setEnabled(True) - self.volOpCebProfile.setCheckState(QtCore.Qt.Checked) - self.volOpComProfile.setCurrentIndex(1) - - if parmObj.name == "rangeList": - value = parmObj.getValue() - value = str(value)[1:-1] - self.volOpProfile.setText(value) - self.volOpProfile.setEnabled(True) - self.volOpCebProfile.setCheckState(QtCore.Qt.Checked) - self.volOpComProfile.setCurrentIndex(2) - - opObj = puObj.getOperationObj(name="Decoder") - self.volOpCode.setText("") - if opObj == None: - self.volOpCebDecodification.setCheckState(0) - else: - self.volOpCebDecodification.setCheckState(QtCore.Qt.Checked) - - parmObj = opObj.getParameterObj('code') - - if parmObj == None: - self.volOpComCode.setCurrentIndex(0) - else: - - parmObj1 = opObj.getParameterObj('nCode') - parmObj2 = opObj.getParameterObj('nBaud') - - if parmObj1 == None or parmObj2 == None: - self.volOpComCode.setCurrentIndex(0) - else: - code = ast.literal_eval(str(parmObj.getValue())) - nCode = parmObj1.getValue() - nBaud = parmObj2.getValue() - - code = numpy.asarray(code).reshape((nCode, nBaud)).tolist() - - #User defined by default - self.volOpComCode.setCurrentIndex(13) - self.volOpCode.setText(str(code)) - - if nCode == 1: - if nBaud == 3: - self.volOpComCode.setCurrentIndex(1) - if nBaud == 4: - self.volOpComCode.setCurrentIndex(2) - if nBaud == 5: - self.volOpComCode.setCurrentIndex(3) - if nBaud == 7: - self.volOpComCode.setCurrentIndex(4) - if nBaud == 11: - self.volOpComCode.setCurrentIndex(5) - if nBaud == 13: - self.volOpComCode.setCurrentIndex(6) - - if nCode == 2: - if nBaud == 3: - self.volOpComCode.setCurrentIndex(7) - if nBaud == 4: - self.volOpComCode.setCurrentIndex(8) - if nBaud == 5: - self.volOpComCode.setCurrentIndex(9) - if nBaud == 7: - self.volOpComCode.setCurrentIndex(10) - if nBaud == 11: - self.volOpComCode.setCurrentIndex(11) - if nBaud == 13: - self.volOpComCode.setCurrentIndex(12) - - - opObj = puObj.getOperationObj(name="deFlip") - if opObj == None: - self.volOpFlip.clear() - self.volOpFlip.setEnabled(False) - self.volOpCebFlip.setCheckState(0) - else: - try: - value = opObj.getParameterValue(parameterName='channelList') - value = str(value)[1:-1] - except: - value = "" - - self.volOpFlip.setText(value) - self.volOpFlip.setEnabled(True) - self.volOpCebFlip.setCheckState(QtCore.Qt.Checked) - - opObj = puObj.getOperationObj(name="CohInt") - if opObj == None: - self.volOpCohInt.clear() - self.volOpCebCohInt.setCheckState(0) - else: - value = opObj.getParameterValue(parameterName='n') - self.volOpCohInt.setText(str(value)) - self.volOpCohInt.setEnabled(True) - self.volOpCebCohInt.setCheckState(QtCore.Qt.Checked) - - opObj = puObj.getOperationObj(name='Scope') - if opObj == None: - self.volGraphCebshow.setCheckState(0) - else: - self.volGraphCebshow.setCheckState(QtCore.Qt.Checked) - - parmObj = opObj.getParameterObj(parameterName='channelList') - - if parmObj == None: - self.volGraphChannelList.clear() - else: - value = parmObj.getValue() - value = str(value) - self.volGraphChannelList.setText(value) -# self.volOpChannel.setEnabled(True) - - parmObj1 = opObj.getParameterObj(parameterName='ymin') - parmObj2 = opObj.getParameterObj(parameterName='ymax') - - if parmObj1 == None or parmObj2 ==None: - self.volGraphIntensityRange.clear() - else: - value1 = parmObj1.getValue() - value1 = str(value1) - value2 = parmObj2.getValue() - value2 = str(value2) - value = value1 + "," + value2 - self.volGraphIntensityRange.setText(value) - - parmObj1 = opObj.getParameterObj(parameterName='xmin') - parmObj2 = opObj.getParameterObj(parameterName='xmax') - - if parmObj1 == None or parmObj2 ==None: - self.volGraphHeightrange.clear() - else: - value1 = parmObj1.getValue() - value1 = str(value1) - value2 = parmObj2.getValue() - value2 = str(value2) - value = value1 + "," + value2 - value2 = str(value2) - self.volGraphHeightrange.setText(value) - - parmObj = opObj.getParameterObj(parameterName='type') - - if parmObj == None: - self.volComScopeType.setCurrentIndex(0) - else: - value = parmObj.getValue() - if value == "iq": - self.volComScopeType.setCurrentIndex(0) - if value == "power": - self.volComScopeType.setCurrentIndex(1) - - parmObj = opObj.getParameterObj(parameterName='save') - - if parmObj == None: - self.volGraphCebSave.setCheckState(QtCore.Qt.Unchecked) - else: - value = parmObj.getValue() - if value: - self.volGraphCebSave.setCheckState(QtCore.Qt.Checked) - else: - self.volGraphCebSave.setCheckState(QtCore.Qt.Unchecked) - - parmObj = opObj.getParameterObj(parameterName='figpath') - if parmObj == None: - self.volGraphPath.clear() - else: - value = parmObj.getValue() - path = str(value) - self.volGraphPath.setText(path) - - parmObj = opObj.getParameterObj(parameterName='figfile') - if parmObj == None: - self.volGraphPrefix.clear() - else: - value = parmObj.getValue() - figfile = str(value) - self.volGraphPrefix.setText(figfile) - - # outputVoltageWrite - opObj = puObj.getOperationObj(name='VoltageWriter') - - if opObj == None: - self.volOutputPath.clear() - self.volOutputblocksperfile.clear() - self.volOutputprofilesperblock.clear() - else: - parmObj = opObj.getParameterObj(parameterName='path') - if parmObj == None: - self.volOutputPath.clear() - else: - value = parmObj.getValue() - path = str(value) - self.volOutputPath.setText(path) - - parmObj = opObj.getParameterObj(parameterName='blocksPerFile') - if parmObj == None: - self.volOutputblocksperfile.clear() - else: - value = parmObj.getValue() - blocksperfile = str(value) - self.volOutputblocksperfile.setText(blocksperfile) - - parmObj = opObj.getParameterObj(parameterName='profilesPerBlock') - if parmObj == None: - self.volOutputprofilesperblock.clear() - else: - value = parmObj.getValue() - profilesPerBlock = str(value) - self.volOutputprofilesperblock.setText(profilesPerBlock) - - return - - def __refreshSpectraWindow(self, puObj): - - inputId = puObj.getInputId() - inputPUObj = self.__puObjDict[inputId] - - if inputPUObj.datatype == 'Voltage': - self.specOpnFFTpoints.setEnabled(True) - self.specOpProfiles.setEnabled(True) - self.specOpippFactor.setEnabled(True) - else: - self.specOpnFFTpoints.setEnabled(False) - self.specOpProfiles.setEnabled(False) - self.specOpippFactor.setEnabled(False) - - opObj = puObj.getOperationObj(name='setRadarFrequency') - if opObj == None: - self.specOpRadarfrequency.clear() - self.specOpCebRadarfrequency.setCheckState(0) - else: - value = opObj.getParameterValue(parameterName='frequency') - value = str(float(value)/1e6) - self.specOpRadarfrequency.setText(value) - self.specOpRadarfrequency.setEnabled(True) - self.specOpCebRadarfrequency.setCheckState(QtCore.Qt.Checked) - - opObj = puObj.getOperationObj(name="run") - if opObj == None: - self.specOpnFFTpoints.clear() - self.specOpProfiles.clear() - self.specOpippFactor.clear() - else: - parmObj = opObj.getParameterObj(parameterName='nFFTPoints') - if parmObj == None: - self.specOpnFFTpoints.clear() - else: - self.specOpnFFTpoints.setEnabled(True) - value = opObj.getParameterValue(parameterName='nFFTPoints') - self.specOpnFFTpoints.setText(str(value)) - - parmObj = opObj.getParameterObj(parameterName='nProfiles') - if parmObj == None: - self.specOpProfiles.clear() - else: - self.specOpProfiles.setEnabled(True) - value = opObj.getParameterValue(parameterName='nProfiles') - self.specOpProfiles.setText(str(value)) - - parmObj = opObj.getParameterObj(parameterName='ippFactor') - if parmObj == None: - self.specOpippFactor.clear() - else: - self.specOpippFactor.setEnabled(True) - value = opObj.getParameterValue(parameterName='ippFactor') - self.specOpippFactor.setText(str(value)) - - opObj = puObj.getOperationObj(name="run") - if opObj == None: - self.specOppairsList.clear() - self.specOpCebCrossSpectra.setCheckState(0) - else: - parmObj = opObj.getParameterObj(parameterName='pairsList') - if parmObj == None: - self.specOppairsList.clear() - self.specOpCebCrossSpectra.setCheckState(0) - else: - value = opObj.getParameterValue(parameterName='pairsList') - value = str(value)[1:-1] - self.specOppairsList.setText(str(value)) - self.specOppairsList.setEnabled(True) - self.specOpCebCrossSpectra.setCheckState(QtCore.Qt.Checked) - - opObj = puObj.getOperationObj(name="selectChannels") - - if opObj == None: - opObj = puObj.getOperationObj(name="selectChannelsByIndex") - - if opObj == None: - self.specOpChannel.clear() - self.specOpCebChannel.setCheckState(0) - else: - channelEnabled = False - try: - value = opObj.getParameterValue(parameterName='channelList') - value = str(value)[1:-1] - channelEnabled = True - channelMode = 0 - except: - pass - try: - value = opObj.getParameterValue(parameterName='channelIndexList') - value = str(value)[1:-1] - channelEnabled = True - channelMode = 1 - except: - pass - - if channelEnabled: - self.specOpChannel.setText(value) - self.specOpChannel.setEnabled(True) - self.specOpCebChannel.setCheckState(QtCore.Qt.Checked) - self.specOpComChannel.setCurrentIndex(channelMode) - - opObj = puObj.getOperationObj(name="selectHeights") - if opObj == None: - self.specOpHeights.clear() - self.specOpCebHeights.setCheckState(0) - else: - value1 = int(opObj.getParameterValue(parameterName='minHei')) - value1 = str(value1) - value2 = int(opObj.getParameterValue(parameterName='maxHei')) - value2 = str(value2) - value = value1 + "," + value2 - self.specOpHeights.setText(value) - self.specOpHeights.setEnabled(True) - self.specOpCebHeights.setCheckState(QtCore.Qt.Checked) - - opObj = puObj.getOperationObj(name="IncohInt") - if opObj == None: - self.specOpIncoherent.clear() - self.specOpCebIncoherent.setCheckState(0) - else: - for parmObj in opObj.getParameterObjList(): - if parmObj.name == 'timeInterval': - value = opObj.getParameterValue(parameterName='timeInterval') - self.specOpIncoherent.setText(str(value)) - self.specOpIncoherent.setEnabled(True) - self.specOpCebIncoherent.setCheckState(QtCore.Qt.Checked) - self.specOpCobIncInt.setCurrentIndex(0) - - if parmObj.name == 'n': - value = opObj.getParameterValue(parameterName='n') - self.specOpIncoherent.setText(str(value)) - self.specOpIncoherent.setEnabled(True) - self.specOpCebIncoherent.setCheckState(QtCore.Qt.Checked) - self.specOpCobIncInt.setCurrentIndex(1) - - opObj = puObj.getOperationObj(name="removeDC") - if opObj == None: - self.specOpCebRemoveDC.setCheckState(0) - else: - self.specOpCebRemoveDC.setCheckState(QtCore.Qt.Checked) - - parmObj = opObj.getParameterObj(parameterName='mode') - - value = 1 - if parmObj: - value = parmObj.getValue() - - if value == 1: - self.specOpComRemoveDC.setCurrentIndex(0) - elif value == 2: - self.specOpComRemoveDC.setCurrentIndex(1) - - opObj = puObj.getOperationObj(name="removeInterference") - if opObj == None: - self.specOpCebRemoveInt.setCheckState(0) - else: - self.specOpCebRemoveInt.setCheckState(QtCore.Qt.Checked) - - opObj = puObj.getOperationObj(name='getNoise') - if opObj == None: - self.specOpCebgetNoise.setCheckState(0) - self.specOpgetNoise.clear() - else: - self.specOpCebgetNoise.setCheckState(QtCore.Qt.Checked) - parmObj = opObj.getParameterObj(parameterName='minHei') - if parmObj == None: - self.specOpgetNoise.clear() - value1 = None - else: - value1 = opObj.getParameterValue(parameterName='minHei') - value1 = str(value1) - parmObj = opObj.getParameterObj(parameterName='maxHei') - if parmObj == None: - value2 = None - value = value1 - self.specOpgetNoise.setText(value) - self.specOpgetNoise.setEnabled(True) - else: - value2 = opObj.getParameterValue(parameterName='maxHei') - value2 = str(value2) - parmObj = opObj.getParameterObj(parameterName='minVel') - if parmObj == None: - value3 = None - value = value1 + "," + value2 - self.specOpgetNoise.setText(value) - self.specOpgetNoise.setEnabled(True) - else: - value3 = opObj.getParameterValue(parameterName='minVel') - value3 = str(value3) - parmObj = opObj.getParameterObj(parameterName='maxVel') - if parmObj == None: - value4 = None - value = value1 + "," + value2 + "," + value3 - self.specOpgetNoise.setText(value) - self.specOpgetNoise.setEnabled(True) - else: - value4 = opObj.getParameterValue(parameterName='maxVel') - value4 = str(value4) - value = value1 + "," + value2 + "," + value3 + ',' + value4 - self.specOpgetNoise.setText(value) - self.specOpgetNoise.setEnabled(True) - - self.specGraphPath.clear() - self.specGraphPrefix.clear() - self.specGgraphFreq.clear() - self.specGgraphHeight.clear() - self.specGgraphDbsrange.clear() - self.specGgraphmagnitud.clear() - self.specGgraphPhase.clear() - self.specGgraphChannelList.clear() - self.specGgraphTminTmax.clear() - self.specGgraphTimeRange.clear() - self.specGgraphftpratio.clear() - - opObj = puObj.getOperationObj(name='SpectraPlot') - - if opObj == None: - self.specGraphCebSpectraplot.setCheckState(0) - self.specGraphSaveSpectra.setCheckState(0) - self.specGraphftpSpectra.setCheckState(0) - else: -# operationSpectraPlot = "Enable" - self.specGraphCebSpectraplot.setCheckState(QtCore.Qt.Checked) - - parmObj = opObj.getParameterObj(parameterName='show') - if parmObj: - if not parmObj.getValue(): - self.specGraphCebSpectraplot.setCheckState(0) - - parmObj = opObj.getParameterObj(parameterName='channelList') - if parmObj == None: - self.specGgraphChannelList.clear() - else: - value = opObj.getParameterValue(parameterName='channelList') - channelListSpectraPlot = str(value)[1:-1] - self.specGgraphChannelList.setText(channelListSpectraPlot) - self.specGgraphChannelList.setEnabled(True) - - parmObj = opObj.getParameterObj(parameterName='xmin') - if parmObj == None: - self.specGgraphFreq.clear() - else: - value1 = opObj.getParameterValue(parameterName='xmin') - value1 = str(value1) - value2 = opObj.getParameterValue(parameterName='xmax') - value2 = str(value2) - value = value1 + "," + value2 - self.specGgraphFreq.setText(value) - self.specGgraphFreq.setEnabled(True) - - parmObj = opObj.getParameterObj(parameterName='ymin') - if parmObj == None: - self.specGgraphHeight.clear() - else: - value1 = opObj.getParameterValue(parameterName='ymin') - value1 = str(value1) - value2 = opObj.getParameterValue(parameterName='ymax') - value2 = str(value2) - value = value1 + "," + value2 - self.specGgraphHeight.setText(value) - self.specGgraphHeight.setEnabled(True) - - parmObj = opObj.getParameterObj(parameterName='zmin') - if parmObj == None: - self.specGgraphDbsrange.clear() - else: - value1 = opObj.getParameterValue(parameterName='zmin') - value1 = str(value1) - value2 = opObj.getParameterValue(parameterName='zmax') - value2 = str(value2) - value = value1 + "," + value2 - self.specGgraphDbsrange.setText(value) - self.specGgraphDbsrange.setEnabled(True) - - parmObj = opObj.getParameterObj(parameterName="save") - if parmObj == None: - self.specGraphSaveSpectra.setCheckState(0) - else: - self.specGraphSaveSpectra.setCheckState(QtCore.Qt.Checked) - - parmObj = opObj.getParameterObj(parameterName="ftp") - if parmObj == None: - self.specGraphftpSpectra.setCheckState(0) - else: - self.specGraphftpSpectra.setCheckState(QtCore.Qt.Checked) - - parmObj = opObj.getParameterObj(parameterName="figpath") - if parmObj: - value = parmObj.getValue() - self.specGraphPath.setText(value) - - parmObj = opObj.getParameterObj(parameterName="wr_period") - if parmObj: - value = parmObj.getValue() - self.specGgraphftpratio.setText(str(value)) - - opObj = puObj.getOperationObj(name='CrossSpectraPlot') - - if opObj == None: - self.specGraphCebCrossSpectraplot.setCheckState(0) - self.specGraphSaveCross.setCheckState(0) - self.specGraphftpCross.setCheckState(0) - else: -# operationCrossSpectraPlot = "Enable" - self.specGraphCebCrossSpectraplot.setCheckState(QtCore.Qt.Checked) - - parmObj = opObj.getParameterObj(parameterName='show') - if parmObj: - if not parmObj.getValue(): - self.specGraphCebCrossSpectraplot.setCheckState(0) - - parmObj = opObj.getParameterObj(parameterName='xmin') - if parmObj == None: - self.specGgraphFreq.clear() - else: - value1 = opObj.getParameterValue(parameterName='xmin') - value1 = str(value1) - value2 = opObj.getParameterValue(parameterName='xmax') - value2 = str(value2) - value = value1 + "," + value2 - self.specGgraphFreq.setText(value) - self.specGgraphFreq.setEnabled(True) - - parmObj = opObj.getParameterObj(parameterName='ymin') - if parmObj == None: - self.specGgraphHeight.clear() - else: - value1 = opObj.getParameterValue(parameterName='ymin') - value1 = str(value1) - value2 = opObj.getParameterValue(parameterName='ymax') - value2 = str(value2) - value = value1 + "," + value2 - self.specGgraphHeight.setText(value) - self.specGgraphHeight.setEnabled(True) - - parmObj = opObj.getParameterObj(parameterName='zmin') - if parmObj == None: - self.specGgraphDbsrange.clear() - else: - value1 = opObj.getParameterValue(parameterName='zmin') - value1 = str(value1) - value2 = opObj.getParameterValue(parameterName='zmax') - value2 = str(value2) - value = value1 + "," + value2 - self.specGgraphDbsrange.setText(value) - self.specGgraphDbsrange.setEnabled(True) - - parmObj = opObj.getParameterObj(parameterName='coh_min') - if parmObj == None: - self.specGgraphmagnitud.clear() - else: - value1 = opObj.getParameterValue(parameterName='coh_min') - value1 = str(value1) - value2 = opObj.getParameterValue(parameterName='coh_max') - value2 = str(value2) - value = value1 + "," + value2 - self.specGgraphmagnitud.setText(value) - self.specGgraphmagnitud.setEnabled(True) - - parmObj = opObj.getParameterObj(parameterName='phase_min') - if parmObj == None: - self.specGgraphPhase.clear() - else: - value1 = opObj.getParameterValue(parameterName='phase_min') - value1 = str(value1) - value2 = opObj.getParameterValue(parameterName='phase_max') - value2 = str(value2) - value = value1 + "," + value2 - self.specGgraphPhase.setText(value) - self.specGgraphPhase.setEnabled(True) - - parmObj = opObj.getParameterObj(parameterName="save") - if parmObj == None: - self.specGraphSaveCross.setCheckState(0) - else: - self.specGraphSaveCross.setCheckState(QtCore.Qt.Checked) - - parmObj = opObj.getParameterObj(parameterName="ftp") - if parmObj == None: - self.specGraphftpCross.setCheckState(0) - else: - self.specGraphftpCross.setCheckState(QtCore.Qt.Checked) - - parmObj = opObj.getParameterObj(parameterName="figpath") - if parmObj: - value = parmObj.getValue() - self.specGraphPath.setText(value) - - parmObj = opObj.getParameterObj(parameterName="wr_period") - if parmObj: - value = parmObj.getValue() - self.specGgraphftpratio.setText(str(value)) - - opObj = puObj.getOperationObj(name='RTIPlot') - - if opObj == None: - self.specGraphCebRTIplot.setCheckState(0) - self.specGraphSaveRTIplot.setCheckState(0) - self.specGraphftpRTIplot.setCheckState(0) - else: - self.specGraphCebRTIplot.setCheckState(QtCore.Qt.Checked) - - parmObj = opObj.getParameterObj(parameterName='show') - if parmObj: - if not parmObj.getValue(): - self.specGraphCebRTIplot.setCheckState(0) - - parmObj = opObj.getParameterObj(parameterName='channelList') - if parmObj == None: - self.specGgraphChannelList.clear() - else: - value = opObj.getParameterValue(parameterName='channelList') - channelListRTIPlot = str(value)[1:-1] - self.specGgraphChannelList.setText(channelListRTIPlot) - self.specGgraphChannelList.setEnabled(True) - - parmObj = opObj.getParameterObj(parameterName='xmin') - if parmObj == None: - self.specGgraphTminTmax.clear() - else: - value1 = opObj.getParameterValue(parameterName='xmin') - value1 = str(value1) - value2 = opObj.getParameterValue(parameterName='xmax') - value2 = str(value2) - value = value1 + "," + value2 - self.specGgraphTminTmax.setText(value) - self.specGgraphTminTmax.setEnabled(True) - - parmObj = opObj.getParameterObj(parameterName='timerange') - if parmObj == None: - self.specGgraphTimeRange.clear() - else: - value1 = opObj.getParameterValue(parameterName='timerange') - value1 = str(value1) - self.specGgraphTimeRange.setText(value1) - self.specGgraphTimeRange.setEnabled(True) - - parmObj = opObj.getParameterObj(parameterName='ymin') - if parmObj == None: - self.specGgraphHeight.clear() - else: - value1 = opObj.getParameterValue(parameterName='ymin') - value1 = str(value1) - value2 = opObj.getParameterValue(parameterName='ymax') - value2 = str(value2) - value = value1 + "," + value2 - self.specGgraphHeight.setText(value) - self.specGgraphHeight.setEnabled(True) - - parmObj = opObj.getParameterObj(parameterName='zmin') - if parmObj == None: - self.specGgraphDbsrange.clear() - else: - value1 = opObj.getParameterValue(parameterName='zmin') - value1 = str(value1) - value2 = opObj.getParameterValue(parameterName='zmax') - value2 = str(value2) - value = value1 + "," + value2 - self.specGgraphDbsrange.setText(value) - self.specGgraphDbsrange.setEnabled(True) - - parmObj = opObj.getParameterObj(parameterName="save") - if parmObj == None: - self.specGraphSaveRTIplot.setCheckState(0) - else: - self.specGraphSaveRTIplot.setCheckState(QtCore.Qt.Checked) - - parmObj = opObj.getParameterObj(parameterName="ftp") - if parmObj == None: - self.specGraphftpRTIplot.setCheckState(0) - else: - self.specGraphftpRTIplot.setCheckState(QtCore.Qt.Checked) - - parmObj = opObj.getParameterObj(parameterName="figpath") - if parmObj: - value = parmObj.getValue() - self.specGraphPath.setText(value) - - parmObj = opObj.getParameterObj(parameterName="wr_period") - if parmObj: - value = parmObj.getValue() - self.specGgraphftpratio.setText(str(value)) - - opObj = puObj.getOperationObj(name='CoherenceMap') - - if opObj == None: - self.specGraphCebCoherencmap.setCheckState(0) - self.specGraphSaveCoherencemap.setCheckState(0) - self.specGraphftpCoherencemap.setCheckState(0) - else: -# operationCoherenceMap = "Enable" - self.specGraphCebCoherencmap.setCheckState(QtCore.Qt.Checked) - - parmObj = opObj.getParameterObj(parameterName='show') - if parmObj: - if not parmObj.getValue(): - self.specGraphCebCoherencmap.setCheckState(0) - - parmObj = opObj.getParameterObj(parameterName='xmin') - if parmObj == None: - self.specGgraphTminTmax.clear() - else: - value1 = opObj.getParameterValue(parameterName='xmin') - value1 = str(value1) - value2 = opObj.getParameterValue(parameterName='xmax') - value2 = str(value2) - value = value1 + "," + value2 - self.specGgraphTminTmax.setText(value) - self.specGgraphTminTmax.setEnabled(True) - - parmObj = opObj.getParameterObj(parameterName='timerange') - if parmObj == None: - self.specGgraphTimeRange.clear() - else: - value1 = opObj.getParameterValue(parameterName='timerange') - value1 = str(value1) - self.specGgraphTimeRange.setText(value1) - self.specGgraphTimeRange.setEnabled(True) - - parmObj = opObj.getParameterObj(parameterName='ymin') - if parmObj == None: - self.specGgraphHeight.clear() - else: - value1 = opObj.getParameterValue(parameterName='ymin') - value1 = str(value1) - value2 = opObj.getParameterValue(parameterName='ymax') - value2 = str(value2) - value = value1 + "," + value2 - self.specGgraphHeight.setText(value) - self.specGgraphHeight.setEnabled(True) - - parmObj = opObj.getParameterObj(parameterName='zmin') - if parmObj == None: - self.specGgraphmagnitud.clear() - else: - value1 = opObj.getParameterValue(parameterName='zmin') - value1 = str(value1) - value2 = opObj.getParameterValue(parameterName='zmax') - value2 = str(value2) - value = value1 + "," + value2 - self.specGgraphmagnitud.setText(value) - self.specGgraphmagnitud.setEnabled(True) - - parmObj = opObj.getParameterObj(parameterName='coh_min') - if parmObj == None: - self.specGgraphmagnitud.clear() - else: - value1 = opObj.getParameterValue(parameterName='coh_min') - value1 = str(value1) - value2 = opObj.getParameterValue(parameterName='coh_max') - value2 = str(value2) - value = value1 + "," + value2 - self.specGgraphmagnitud.setText(value) - self.specGgraphmagnitud.setEnabled(True) - - parmObj = opObj.getParameterObj(parameterName='phase_min') - if parmObj == None: - self.specGgraphPhase.clear() - else: - value1 = opObj.getParameterValue(parameterName='phase_min') - value1 = str(value1) - value2 = opObj.getParameterValue(parameterName='phase_max') - value2 = str(value2) - value = value1 + "," + value2 - self.specGgraphPhase.setText(value) - self.specGgraphPhase.setEnabled(True) - - parmObj = opObj.getParameterObj(parameterName="save") - if parmObj == None: - self.specGraphSaveCoherencemap.setCheckState(0) - else: - self.specGraphSaveCoherencemap.setCheckState(QtCore.Qt.Checked) - - parmObj = opObj.getParameterObj(parameterName="ftp") - if parmObj == None: - self.specGraphftpCoherencemap.setCheckState(0) - else: - self.specGraphftpCoherencemap.setCheckState(QtCore.Qt.Checked) - - parmObj = opObj.getParameterObj(parameterName="figpath") - if parmObj: - value = parmObj.getValue() - self.specGraphPath.setText(value) - - parmObj = opObj.getParameterObj(parameterName="wr_period") - if parmObj: - value = parmObj.getValue() - self.specGgraphftpratio.setText(str(value)) - - opObj = puObj.getOperationObj(name='PowerProfilePlot') - - if opObj == None: - self.specGraphPowerprofile.setCheckState(0) - self.specGraphSavePowerprofile.setCheckState(0) - self.specGraphftpPowerprofile.setCheckState(0) - operationPowerProfilePlot = "Disabled" - channelList = None - freq_vel = None - heightsrange = None - else: -# operationPowerProfilePlot = "Enable" - self.specGraphPowerprofile.setCheckState(QtCore.Qt.Checked) - - parmObj = opObj.getParameterObj(parameterName='show') - if parmObj: - if not parmObj.getValue(): - self.specGraphPowerprofile.setCheckState(0) - - parmObj = opObj.getParameterObj(parameterName='xmin') - if parmObj == None: - self.specGgraphDbsrange.clear() - else: - value1 = opObj.getParameterValue(parameterName='xmin') - value1 = str(value1) - value2 = opObj.getParameterValue(parameterName='xmax') - value2 = str(value2) - value = value1 + "," + value2 - self.specGgraphDbsrange.setText(value) - self.specGgraphDbsrange.setEnabled(True) - - parmObj = opObj.getParameterObj(parameterName='ymin') - if parmObj == None: - self.specGgraphHeight.clear() - else: - value1 = opObj.getParameterValue(parameterName='ymin') - value1 = str(value1) - value2 = opObj.getParameterValue(parameterName='ymax') - value2 = str(value2) - value = value1 + "," + value2 - self.specGgraphHeight.setText(value) - self.specGgraphHeight.setEnabled(True) - - parmObj = opObj.getParameterObj(parameterName="save") - if parmObj == None: - self.specGraphSavePowerprofile.setCheckState(0) - else: - self.specGraphSavePowerprofile.setCheckState(QtCore.Qt.Checked) - - parmObj = opObj.getParameterObj(parameterName="ftp") - if parmObj == None: - self.specGraphftpPowerprofile.setCheckState(0) - else: - self.specGraphftpPowerprofile.setCheckState(QtCore.Qt.Checked) - - parmObj = opObj.getParameterObj(parameterName="figpath") - if parmObj: - value = parmObj.getValue() - self.specGraphPath.setText(value) - - parmObj = opObj.getParameterObj(parameterName="wr_period") - if parmObj: - value = parmObj.getValue() - self.specGgraphftpratio.setText(str(value)) - - opObj = puObj.getOperationObj(name='Noise') - - if opObj == None: - self.specGraphCebRTInoise.setCheckState(0) - self.specGraphSaveRTInoise.setCheckState(0) - self.specGraphftpRTInoise.setCheckState(0) - else: - self.specGraphCebRTInoise.setCheckState(QtCore.Qt.Checked) - - parmObj = opObj.getParameterObj(parameterName='show') - if parmObj: - if not parmObj.getValue(): - self.specGraphCebRTInoise.setCheckState(0) - - parmObj = opObj.getParameterObj(parameterName='channelList') - if parmObj == None: - self.specGgraphChannelList.clear() - else: - value = opObj.getParameterValue(parameterName='channelList') - channelListRTINoise = str(value)[1:-1] - self.specGgraphChannelList.setText(channelListRTINoise) - self.specGgraphChannelList.setEnabled(True) - - parmObj = opObj.getParameterObj(parameterName='xmin') - if parmObj == None: - self.specGgraphTminTmax.clear() - else: - value1 = opObj.getParameterValue(parameterName='xmin') - value1 = str(value1) - value2 = opObj.getParameterValue(parameterName='xmax') - value2 = str(value2) - value = value1 + "," + value2 - self.specGgraphTminTmax.setText(value) - self.specGgraphTminTmax.setEnabled(True) - - parmObj = opObj.getParameterObj(parameterName='timerange') - if parmObj == None: - self.specGgraphTimeRange.clear() - else: - value1 = opObj.getParameterValue(parameterName='timerange') - value1 = str(value1) - self.specGgraphTimeRange.setText(value1) - self.specGgraphTimeRange.setEnabled(True) - - - parmObj = opObj.getParameterObj(parameterName='ymin') - if parmObj == None: - self.specGgraphDbsrange.clear() - else: - value1 = opObj.getParameterValue(parameterName='ymin') - value1 = str(value1) - value2 = opObj.getParameterValue(parameterName='ymax') - value2 = str(value2) - value = value1 + "," + value2 - self.specGgraphDbsrange.setText(value) - self.specGgraphDbsrange.setEnabled(True) - - parmObj = opObj.getParameterObj(parameterName="save") - if parmObj == None: - self.specGraphSaveRTInoise.setCheckState(0) - else: - self.specGraphSaveRTInoise.setCheckState(QtCore.Qt.Checked) - - parmObj = opObj.getParameterObj(parameterName="ftp") - if parmObj == None: - self.specGraphftpRTInoise.setCheckState(0) - else: - self.specGraphftpRTInoise.setCheckState(QtCore.Qt.Checked) - - parmObj = opObj.getParameterObj(parameterName="figpath") - if parmObj: - value = parmObj.getValue() - self.specGraphPath.setText(value) - - parmObj = opObj.getParameterObj(parameterName="wr_period") - if parmObj: - value = parmObj.getValue() - self.specGgraphftpratio.setText(str(value)) - - opObj = puObj.getOperationObj(name='SpectraWriter') - if opObj == None: - self.specOutputPath.clear() - self.specOutputblocksperfile.clear() - else: - value = opObj.getParameterObj(parameterName='path') - if value == None: - self.specOutputPath.clear() - else: - value = opObj.getParameterValue(parameterName='path') - path = str(value) - self.specOutputPath.setText(path) - value = opObj.getParameterObj(parameterName='blocksPerFile') - if value == None: - self.specOutputblocksperfile.clear() - else: - value = opObj.getParameterValue(parameterName='blocksPerFile') - blocksperfile = str(value) - self.specOutputblocksperfile.setText(blocksperfile) - - return - - def __refreshSpectraHeisWindow(self, puObj): - - opObj = puObj.getOperationObj(name="IncohInt4SpectraHeis") - if opObj == None: - self.specHeisOpIncoherent.clear() - self.specHeisOpCebIncoherent.setCheckState(0) - else: - for parmObj in opObj.getParameterObjList(): - if parmObj.name == 'timeInterval': - value = opObj.getParameterValue(parameterName='timeInterval') - self.specHeisOpIncoherent.setText(str(value)) - self.specHeisOpIncoherent.setEnabled(True) - self.specHeisOpCebIncoherent.setCheckState(QtCore.Qt.Checked) - self.specHeisOpCobIncInt.setCurrentIndex(0) - - # SpectraHeis Graph - - self.specHeisGgraphXminXmax.clear() - self.specHeisGgraphYminYmax.clear() - - self.specHeisGgraphChannelList.clear() - self.specHeisGgraphTminTmax.clear() - self.specHeisGgraphTimeRange.clear() - self.specHeisGgraphftpratio.clear() - - opObj = puObj.getOperationObj(name='SpectraHeisScope') - if opObj == None: - self.specHeisGraphCebSpectraplot.setCheckState(0) - self.specHeisGraphSaveSpectra.setCheckState(0) - self.specHeisGraphftpSpectra.setCheckState(0) - else: - operationSpectraHeisScope = "Enable" - self.specHeisGraphCebSpectraplot.setCheckState(QtCore.Qt.Checked) - - parmObj = opObj.getParameterObj(parameterName='channelList') - if parmObj == None: - self.specHeisGgraphChannelList.clear() - else: - value = opObj.getParameterValue(parameterName='channelList') - channelListSpectraHeisScope = str(value)[1:-1] - self.specHeisGgraphChannelList.setText(channelListSpectraHeisScope) - self.specHeisGgraphChannelList.setEnabled(True) - - parmObj = opObj.getParameterObj(parameterName='xmin') - if parmObj == None: - self.specHeisGgraphXminXmax.clear() - else: - value1 = opObj.getParameterValue(parameterName='xmin') - value1 = str(value1) - value2 = opObj.getParameterValue(parameterName='xmax') - value2 = str(value2) - value = value1 + "," + value2 - self.specHeisGgraphXminXmax.setText(value) - self.specHeisGgraphXminXmax.setEnabled(True) - - parmObj = opObj.getParameterObj(parameterName='ymin') - if parmObj == None: - self.specHeisGgraphYminYmax.clear() - else: - value1 = opObj.getParameterValue(parameterName='ymin') - value1 = str(value1) - value2 = opObj.getParameterValue(parameterName='ymax') - value2 = str(value2) - value = value1 + "," + value2 - self.specHeisGgraphYminYmax.setText(value) - self.specHeisGgraphYminYmax.setEnabled(True) - - parmObj = opObj.getParameterObj(parameterName="save") - if parmObj == None: - self.specHeisGraphSaveSpectra.setCheckState(0) - else: - self.specHeisGraphSaveSpectra.setCheckState(QtCore.Qt.Checked) - - parmObj = opObj.getParameterObj(parameterName="ftp") - if parmObj == None: - self.specHeisGraphftpSpectra.setCheckState(0) - else: - self.specHeisGraphftpSpectra.setCheckState(QtCore.Qt.Checked) - - parmObj = opObj.getParameterObj(parameterName="figpath") - if parmObj: - value = parmObj.getValue() - self.specHeisGraphPath.setText(value) - - parmObj = opObj.getParameterObj(parameterName="wr_period") - if parmObj: - value = parmObj.getValue() - self.specHeisGgraphftpratio.setText(str(value)) - - opObj = puObj.getOperationObj(name='RTIfromSpectraHeis') - - if opObj == None: - self.specHeisGraphCebRTIplot.setCheckState(0) - self.specHeisGraphSaveRTIplot.setCheckState(0) - self.specHeisGraphftpRTIplot.setCheckState(0) - else: - self.specHeisGraphCebRTIplot.setCheckState(QtCore.Qt.Checked) - parmObj = opObj.getParameterObj(parameterName='channelList') - if parmObj == None: - self.specHeisGgraphChannelList.clear() - else: - value = opObj.getParameterValue(parameterName='channelList') - channelListRTIPlot = str(value)[1:-1] - self.specGgraphChannelList.setText(channelListRTIPlot) - self.specGgraphChannelList.setEnabled(True) - - parmObj = opObj.getParameterObj(parameterName='xmin') - if parmObj == None: - self.specHeisGgraphTminTmax.clear() - else: - value1 = opObj.getParameterValue(parameterName='xmin') - value1 = str(value1) - value2 = opObj.getParameterValue(parameterName='xmax') - value2 = str(value2) - value = value1 + "," + value2 - self.specHeisGgraphTminTmax.setText(value) - self.specHeisGgraphTminTmax.setEnabled(True) - - parmObj = opObj.getParameterObj(parameterName='timerange') - if parmObj == None: - self.specGgraphTimeRange.clear() - else: - value1 = opObj.getParameterValue(parameterName='timerange') - value1 = str(value1) - self.specHeisGgraphTimeRange.setText(value1) - self.specHeisGgraphTimeRange.setEnabled(True) - - parmObj = opObj.getParameterObj(parameterName='ymin') - if parmObj == None: - self.specHeisGgraphYminYmax.clear() - else: - value1 = opObj.getParameterValue(parameterName='ymin') - value1 = str(value1) - value2 = opObj.getParameterValue(parameterName='ymax') - value2 = str(value2) - value = value1 + "," + value2 - self.specHeisGgraphYminYmax.setText(value) - self.specHeisGgraphYminYmax.setEnabled(True) - - parmObj = opObj.getParameterObj(parameterName="save") - if parmObj == None: - self.specHeisGraphSaveRTIplot.setCheckState(0) - else: - self.specHeisGraphSaveRTIplot.setCheckState(QtCore.Qt.Checked) - - parmObj = opObj.getParameterObj(parameterName="ftp") - if parmObj == None: - self.specHeisGraphftpRTIplot.setCheckState(0) - else: - self.specHeisGraphftpRTIplot.setCheckState(QtCore.Qt.Checked) - - parmObj = opObj.getParameterObj(parameterName="figpath") - if parmObj: - value = parmObj.getValue() - self.specHeisGraphPath.setText(value) - - parmObj = opObj.getParameterObj(parameterName="wr_period") - if parmObj: - value = parmObj.getValue() - self.specHeisGgraphftpratio.setText(str(value)) - - # outputSpectraHeisWrite - opObj = puObj.getOperationObj(name='FitsWriter') - if opObj == None: - self.specHeisOutputPath.clear() - self.specHeisOutputblocksperfile.clear() - self.specHeisOutputMetada.clear() - else: - value = opObj.getParameterObj(parameterName='path') - if value == None: - self.specHeisOutputPath.clear() - else: - value = opObj.getParameterValue(parameterName='path') - path = str(value) - self.specHeisOutputPath.setText(path) - value = opObj.getParameterObj(parameterName='dataBlocksPerFile') - if value == None: - self.specHeisOutputblocksperfile.clear() - else: - value = opObj.getParameterValue(parameterName='dataBlocksPerFile') - blocksperfile = str(value) - self.specHeisOutputblocksperfile.setText(blocksperfile) - value = opObj.getParameterObj(parameterName='metadatafile') - if value == None: - self.specHeisOutputMetada.clear() - else: - value = opObj.getParameterValue(parameterName='metadatafile') - metadata_file = str(value) - self.specHeisOutputMetada.setText(metadata_file) - - return - - def __refreshCorrelationWindow(self, puObj): - pass - - def refreshPUWindow(self, puObj): - - if puObj.datatype == 'Voltage': - self.__refreshVoltageWindow(puObj) - - if puObj.datatype == 'Spectra': - self.__refreshSpectraWindow(puObj) - - if puObj.datatype == 'SpectraHeis': - self.__refreshSpectraHeisWindow(puObj) - - def refreshProjectProperties(self, projectObjView): - - propertyBuffObj = PropertyBuffer() - name = projectObjView.name - - propertyBuffObj.append("Properties", "Name", projectObjView.name), - propertyBuffObj.append("Properties", "Description", projectObjView.description) - propertyBuffObj.append("Properties", "Workspace", self.pathWorkSpace) - - readUnitObj = projectObjView.getReadUnitObj() - runOperationObj = readUnitObj.getOperationObj(name='run') - - for thisParmObj in runOperationObj.getParameterObjList(): - propertyBuffObj.append("Reading parms", thisParmObj.name, str(thisParmObj.getValue())) - - propertiesModel = propertyBuffObj.getPropertyModel() - - self.treeProjectProperties.setModel(propertiesModel) - self.treeProjectProperties.expandAll() - self.treeProjectProperties.resizeColumnToContents(0) - self.treeProjectProperties.resizeColumnToContents(1) - - def refreshPUProperties(self, puObjView): - - ############ FTP CONFIG ################################ - #Deleting FTP Conf. This processing unit have not got any - #FTP configuration by default - if puObjView.id in self.__puLocalFolder2FTP.keys(): - self.__puLocalFolder2FTP.pop(puObjView.id) - ######################################################## - - propertyBuffObj = PropertyBuffer() - - for thisOp in puObjView.getOperationObjList(): - - operationName = thisOp.name - - if operationName == 'run': - operationName = 'Properties' - - else: - if not thisOp.getParameterObjList(): - propertyBuffObj.append(operationName, '--', '--') - continue - - for thisParmObj in thisOp.getParameterObjList(): - propertyBuffObj.append(operationName, thisParmObj.name, str(thisParmObj.getValue())) - - ############ FTP CONFIG ################################ - if thisParmObj.name == "ftp_wei" and thisParmObj.getValue(): - value = thisParmObj.getValue() - self.temporalFTP.ftp_wei = value - - if thisParmObj.name == "exp_code" and thisParmObj.getValue(): - value = thisParmObj.getValue() - self.temporalFTP.exp_code = value - - if thisParmObj.name == "sub_exp_code" and thisParmObj.getValue(): - value = thisParmObj.getValue() - self.temporalFTP.sub_exp_code = value - - if thisParmObj.name == "plot_pos" and thisParmObj.getValue(): - value = thisParmObj.getValue() - self.temporalFTP.plot_pos = value - - if thisParmObj.name == 'ftp' and thisParmObj.getValue(): - figpathObj = thisOp.getParameterObj('figpath') - if figpathObj: - self.__puLocalFolder2FTP[puObjView.id] = figpathObj.getValue() - - ######################################################## - - propertiesModel = propertyBuffObj.getPropertyModel() - - self.treeProjectProperties.setModel(propertiesModel) - self.treeProjectProperties.expandAll() - self.treeProjectProperties.resizeColumnToContents(0) - self.treeProjectProperties.resizeColumnToContents(1) - - def refreshGraphicsId(self): - - projectObj = self.getSelectedProjectObj() - - if not projectObj: - return - - for idPU, puObj in projectObj.procUnitConfObjDict.items(): - - for opObj in puObj.getOperationObjList(): - - if opObj.name not in ('Scope', 'SpectraPlot', 'CrossSpectraPlot', 'RTIPlot', 'CoherenceMap', 'PowerProfilePlot', 'Noise', 'SpectraHeisScope', 'RTIfromSpectraHeis'): - continue - - opObj.changeParameter(name='id', value=opObj.id, format='int') - - def on_click(self, index): - - self._disable_save_button() - self._disable_play_button() - - self.console.clear() - - self.selectedItemTree = self.projectExplorerModel.itemFromIndex(index) - - projectObjView = self.getSelectedProjectObj() - - if not projectObjView: - return - - self.create = False - selectedObjView = self.getSelectedItemObj() - - self.refreshProjectWindow(projectObjView) - self.refreshProjectProperties(projectObjView) - - #A project has been selected - if projectObjView == selectedObjView: - - self.tabProject.setEnabled(True) - self.tabVoltage.setEnabled(False) - self.tabSpectra.setEnabled(False) - self.tabCorrelation.setEnabled(False) - self.tabSpectraHeis.setEnabled(False) - self.tabWidgetProject.setCurrentWidget(self.tabProject) - - if self.dateList: - self._enable_save_button() - self._enable_play_button() - - return - - #A processing unit has been selected - voltEnable = False - specEnable = False - corrEnable = False - specHeisEnable = False - tabSelected = self.tabProject - - puObj = selectedObjView - - self.refreshPUWindow(puObj) - self.refreshPUProperties(puObj) - self.showtabPUCreated(puObj.datatype) - - if self.dateList: - self._enable_save_button() - self._enable_play_button() - - def on_right_click(self, pos): - - self.menu = QtGui.QMenu() - quitAction0 = self.menu.addAction("Create a New Project") - quitAction1 = self.menu.addAction("Create a New Processing Unit") - quitAction2 = self.menu.addAction("Delete Item") - quitAction3 = self.menu.addAction("Quit") - - if len(self.__itemTreeDict) == 0: - quitAction2.setEnabled(False) - else: - quitAction2.setEnabled(True) - - action = self.menu.exec_(self.mapToGlobal(pos)) - - if action == quitAction0: - self. setInputsProject_View() - self.create = True - - if action == quitAction1: - if len(self.__projectObjDict) == 0: - outputstr = "You need to create a Project before adding a Processing Unit" - self.console.clear() - self.console.append(outputstr) - return 0 - else: - self.addPUWindow() - self.console.clear() - self.console.append("Please, Choose the type of Processing Unit") -# self.console.append("If your Datatype is rawdata, you will start with processing unit Type Voltage") -# self.console.append("If your Datatype is pdata, you will choose between processing unit Type Spectra or Correlation") -# self.console.append("If your Datatype is fits, you will start with processing unit Type SpectraHeis") - - if action == quitAction2: - index = self.selectedItemTree - try: - index.parent() - except: - self.console.append('Please, first at all select a Project or Processing Unit') - return 0 - # print index.parent(),index - if index.parent() == None: - self.projectExplorerModel.removeRow(index.row()) - else: - index.parent().removeRow(index.row()) - self.removeItemTreeFromProject() - self.console.clear() - # for i in self.projectExplorerTree.selectionModel().selection().indexes(): - # print i.row() - - if action == quitAction3: - self.close() - return 0 - - def createProjectView(self, id): - -# project_name, description, datatype, data_path, starDate, endDate, startTime, endTime, online, delay, walk, set = self.getParmsFromProjectWindow() - id = str(id) - projectParms = self.__getParmsFromProjectWindow() - - if not projectParms.isValid(): - return None - - projectObjView = Project() - projectObjView.setup(id=id, name=projectParms.name, description=projectParms.description) - - self.__projectObjDict[id] = projectObjView - self.addProject2ProjectExplorer(id=id, name=projectObjView.name) - - return projectObjView - - def updateProjectView(self): - -# project_name, description, datatype, data_path, starDate, endDate, startTime, endTime, online, delay, walk, set = self.getParmsFromProjectWindow() - - projectParms = self.__getParmsFromProjectWindow() - - if not projectParms.isValid(): - return None - - projectObjView = self.getSelectedProjectObj() - - if not projectObjView: - self.console.append("Please select a project before update it") - return None - - projectObjView.update(name=projectParms.name, description=projectParms.description) - - return projectObjView - - def createReadUnitView(self, projectObjView, idReadUnit=None): - - projectParms = self.__getParmsFromProjectWindow() - - if not projectParms.isValid(): - return None - - if projectParms.datatype in ("Voltage", "Spectra", "Fits"): - readUnitConfObj = projectObjView.addReadUnit(id=idReadUnit, - datatype=projectParms.datatype, - path=projectParms.dpath, - startDate=projectParms.startDate, - endDate=projectParms.endDate, - startTime=projectParms.startTime, - endTime=projectParms.endTime, - online=projectParms.online, - walk=projectParms.walk - ) - - if projectParms.set: - readUnitConfObj.addParameter(name="set", value=projectParms.set, format="int") - - if projectParms.delay: - readUnitConfObj.addParameter(name="delay", value=projectParms.delay, format="int") - - if projectParms.expLabel: - readUnitConfObj.addParameter(name="expLabel", value=projectParms.expLabel) - - readUnitConfObj.addOperation(name="printInfo") - - if projectParms.datatype == "USRP": - readUnitConfObj = projectObjView.addReadUnit(id=idReadUnit, - datatype=projectParms.datatype, - path=projectParms.dpath, - startDate=projectParms.startDate, - endDate=projectParms.endDate, - startTime=projectParms.startTime, - endTime=projectParms.endTime, - online=projectParms.online, - ippKm=projectParms.ippKm - ) - - if projectParms.delay: - readUnitConfObj.addParameter(name="delay", value=projectParms.delay, format="int") - - return readUnitConfObj - - def updateReadUnitView(self, projectObjView, idReadUnit): - - projectObjView.removeProcUnit(idReadUnit) - - readUnitConfObj = self.createReadUnitView(projectObjView, idReadUnit) - - return readUnitConfObj - - def createProcUnitView(self, projectObjView, datatype, inputId): - - procUnitConfObj = projectObjView.addProcUnit(datatype=datatype, inputId=inputId) - - self.__puObjDict[procUnitConfObj.getId()] = procUnitConfObj - - return procUnitConfObj - - def updateProcUnitView(self, id): - - pass - - def addPUWindow(self): - - self.configUPWindowObj = UnitProcessWindow(self) - fatherObj = self.getSelectedItemObj() - try: - fatherObj.getElementName() - except: - self.console.append("First left click on Project or Processing Unit") - return 0 - - if fatherObj.getElementName() == 'Project': - readUnitConfObj = fatherObj.getReadUnitObj() - self.configUPWindowObj.dataTypeProject = str(readUnitConfObj.datatype) - - self.configUPWindowObj.getfromWindowList.append(fatherObj) - self.configUPWindowObj.loadTotalList() - self.configUPWindowObj.show() - self.configUPWindowObj.closed.connect(self.createPUWindow) - - def createPUWindow(self): - - if not self.configUPWindowObj.create: - return - - fatherObj = self.configUPWindowObj.getFromWindow - datatype = self.configUPWindowObj.typeofUP - - if fatherObj.getElementName() == 'Project': - inputId = fatherObj.getReadUnitId() - projectObjView = fatherObj - else: - inputId = fatherObj.getId() - projectObjView = self.getSelectedProjectObj() - - if not projectObjView: - return - - puObj = self.createProcUnitView(projectObjView, datatype, inputId) - - self.addPU2ProjectExplorer(puObj) - - self.showtabPUCreated(datatype) - - self.clearPUWindow(datatype) - - self.showPUinitView() - - def addFTPConf2Operation(self, puObj, opObj): - - if not self.temporalFTP.create: - self.temporalFTP.setwithoutconfiguration() - -# opObj.addParameter(name='server', value=self.temporalFTP.server, format='str') -# opObj.addParameter(name='remotefolder', value=self.temporalFTP.remotefolder, format='str') -# opObj.addParameter(name='username', value=self.temporalFTP.username, format='str') -# opObj.addParameter(name='password', value=self.temporalFTP.password, format='str') - - if self.temporalFTP.ftp_wei: - opObj.addParameter(name='ftp_wei', value=int(self.temporalFTP.ftp_wei), format='int') - if self.temporalFTP.exp_code: - opObj.addParameter(name='exp_code', value=int(self.temporalFTP.exp_code), format='int') - if self.temporalFTP.sub_exp_code: - opObj.addParameter(name='sub_exp_code', value=int(self.temporalFTP.sub_exp_code), format='int') - if self.temporalFTP.plot_pos: - opObj.addParameter(name='plot_pos', value=int(self.temporalFTP.plot_pos), format='int') - -# def __checkFTPProcUnit(self, projectObj, localfolder): -# -# puId = None -# puObj = None -# -# for thisPuId, thisPuObj in projectObj.procUnitItems(): -# -# if not thisPuObj.name == "SendToServer": -# continue -# -# opObj = thisPuObj.getOperationObj(name='run') -# -# parmObj = opObj.getParameterObj('localfolder') -# -# #localfolder parameter should always be set, if it is not set then ProcUnit should be removed -# if not parmObj: -# projectObj.removeProcUnit(thisPuId) -# continue -# -# thisLocalfolder = parmObj.getValue() -# -# if localfolder != thisLocalfolder: -# continue -# -# puId = thisPuId -# puObj = thisPuObj -# break -# -# return puObj - - def createFTPProcUnitView(self): - - if not self.temporalFTP.create: - self.temporalFTP.setwithoutconfiguration() - - projectObj = self.getSelectedProjectObj() - - if not projectObj: - return - - self.removeAllFTPProcUnitView(projectObj) - - if not self.__puLocalFolder2FTP: - return - - folderList = ",".join(self.__puLocalFolder2FTP.values()) - - procUnitConfObj = projectObj.addProcUnit(name="SendToServer") - - procUnitConfObj.addParameter(name='server', value=self.temporalFTP.server, format='str') - procUnitConfObj.addParameter(name='username', value=self.temporalFTP.username, format='str') - procUnitConfObj.addParameter(name='password', value=self.temporalFTP.password, format='str') - procUnitConfObj.addParameter(name='localfolder', value=folderList, format='list') - procUnitConfObj.addParameter(name='remotefolder', value=self.temporalFTP.remotefolder, format='str') - procUnitConfObj.addParameter(name='ext', value=self.temporalFTP.extension, format='str') - procUnitConfObj.addParameter(name='period', value=self.temporalFTP.period, format='int') - procUnitConfObj.addParameter(name='protocol', value=self.temporalFTP.protocol, format='str') - - procUnitConfObj.addParameter(name='ftp_wei', value=self.temporalFTP.ftp_wei, format='int') - procUnitConfObj.addParameter(name='exp_code', value=self.temporalFTP.exp_code, format='int') - procUnitConfObj.addParameter(name='sub_exp_code', value=self.temporalFTP.sub_exp_code, format='int') - procUnitConfObj.addParameter(name='plot_pos', value=self.temporalFTP.plot_pos, format='int') - - self.__puObjDict[procUnitConfObj.getId()] = procUnitConfObj - - def removeAllFTPProcUnitView(self, projectObj): - - for thisPuId, thisPuObj in projectObj.procUnitItems(): - - if not thisPuObj.name == "SendToServer": - continue - - projectObj.removeProcUnit(thisPuId) - - if thisPuId not in self.__puObjDict.keys(): - continue - - self.__puObjDict.pop(thisPuId) - - def showPUinitView(self): - - self.propertiesModel = TreeModel() - self.propertiesModel.initPUVoltageView() - self.treeProjectProperties.setModel(self.propertiesModel) - self.treeProjectProperties.expandAll() - self.treeProjectProperties.allColumnsShowFocus() - self.treeProjectProperties.resizeColumnToContents(1) - - def saveFTPFromOpObj(self, operationObj): - - if operationObj.name != "SendByFTP": - return - - server = operationObj.getParameterValue("server") - username = operationObj.getParameterValue("username") - password = operationObj.getParameterValue("password") - localfolder = operationObj.getParameterValue("localfolder") - remotefolder = operationObj.getParameterValue("remotefolder") - ext = operationObj.getParameterValue("ext") - period = operationObj.getParameterValue("period") - - self.temporalFTP.save(server=server, - remotefolder=remotefolder, - username=username, - password=password, - localfolder=localfolder, - extension=ext) - - return - - def saveFTPFromProcUnitObj(self, puObj): - - opObj = puObj.getOperationObj(name="run") - - parmObj = opObj.getParameterObj(parameterName="server") - if parmObj == None: - server = 'jro-app.igp.gob.pe' - else: - server = parmObj.getValue() - - parmObj = opObj.getParameterObj(parameterName="remotefolder") - if parmObj == None: - remotefolder = '/home/wmaster/graficos' - else: - remotefolder = parmObj.getValue() - - parmObj = opObj.getParameterObj(parameterName="username") - if parmObj == None: - username = 'wmaster' - else: - username = parmObj.getValue() - - parmObj = opObj.getParameterObj(parameterName="password") - if parmObj == None: - password = 'mst2010vhf' - else: - password = parmObj.getValue() - - parmObj = opObj.getParameterObj(parameterName="ftp_wei") - if parmObj == None: - ftp_wei = 0 - else: - ftp_wei = parmObj.getValue() - - parmObj = opObj.getParameterObj(parameterName="exp_code") - if parmObj == None: - exp_code = 0 - else: - exp_code = parmObj.getValue() - - parmObj = opObj.getParameterObj(parameterName="sub_exp_code") - if parmObj == None: - sub_exp_code = 0 - else: - sub_exp_code = parmObj.getValue() - - parmObj = opObj.getParameterObj(parameterName="plot_pos") - if parmObj == None: - plot_pos = 0 - else: - plot_pos = parmObj.getValue() - - parmObj = opObj.getParameterObj(parameterName="localfolder") - if parmObj == None: - localfolder = None - else: - localfolder = parmObj.getValue() - - parmObj = opObj.getParameterObj(parameterName="ext") - if parmObj == None: - extension = '.png' - else: - extension = parmObj.getValue() - - self.temporalFTP.save(server=server, - remotefolder=remotefolder, - username=username, - password=password, - ftp_wei=ftp_wei, - exp_code=exp_code, - sub_exp_code=sub_exp_code, - plot_pos=plot_pos, - localfolder=localfolder, - extension=extension) - - def addProject2ProjectExplorer(self, id, name): - - itemTree = QtGui.QStandardItem(QtCore.QString(str(name))) - - parentItem = self.projectExplorerModel.invisibleRootItem() - parentItem.appendRow(itemTree) - - self.projectExplorerTree.setCurrentIndex(itemTree.index()) - - self.selectedItemTree = itemTree - - self.__itemTreeDict[id] = itemTree - - def addPU2ProjectExplorer(self, puObj): - - id, name = puObj.id, puObj.datatype - - itemTree = QtGui.QStandardItem(QtCore.QString(str(name))) - - parentItem = self.selectedItemTree - parentItem.appendRow(itemTree) - self.projectExplorerTree.expandAll() - - self.projectExplorerTree.setCurrentIndex(itemTree.index()) - - self.selectedItemTree = itemTree - - self.__itemTreeDict[id] = itemTree - - def addPU2PELoadXML(self, puObj): - - id, name, inputId = puObj.id, puObj.datatype, puObj.inputId - - itemTree = QtGui.QStandardItem(QtCore.QString(str(name))) - - if self.__itemTreeDict.has_key(inputId): - parentItem = self.__itemTreeDict[inputId] - else: - #If parent is a Reader object - parentItem = self.__itemTreeDict[id[:-1]] - - parentItem.appendRow(itemTree) - self.projectExplorerTree.expandAll() - parentItem = itemTree - self.projectExplorerTree.setCurrentIndex(parentItem.index()) - - self.__itemTreeDict[id] = itemTree - self.selectedItemTree = itemTree - - def getSelectedProjectObj(self): - """ - Return the current project object selected. If a processing unit is - actually selected this function returns associated project. - - None if any project or processing unit is selected - """ - for key in self.__itemTreeDict.keys(): - if self.__itemTreeDict[key] != self.selectedItemTree: - continue - - if self.__projectObjDict.has_key(key): - projectObj = self.__projectObjDict[key] - return projectObj - - puObj = self.__puObjDict[key] - - if puObj.parentId == None: - projectId = puObj.getId()[0] - else: - projectId = puObj.parentId - - projectObj = self.__projectObjDict[projectId] - return projectObj - - return None - - def getSelectedItemObj(self): - """ - Return the current project or processing unit object selected - - None if any project or processing unit is selected - """ - for key in self.__itemTreeDict.keys(): - if self.__itemTreeDict[key] != self.selectedItemTree: - continue - - if self.__projectObjDict.has_key(key) == True: - fatherObj = self.__projectObjDict[key] - else: - fatherObj = self.__puObjDict[key] - - return fatherObj - - return None - - def _WarningWindow(self, text, information): - - msgBox = QtGui.QMessageBox() - msgBox.setText(text) - msgBox.setInformativeText(information) - msgBox.setStandardButtons(QtGui.QMessageBox.Ok | QtGui.QMessageBox.Cancel) - msgBox.setDefaultButton(QtGui.QMessageBox.Ok) - ret = msgBox.exec_() - - answer = False - - if ret == QtGui.QMessageBox.Ok: - answer = True - - return answer - - def __getNewProjectId(self): - - loadProject = False - - for thisId in range(1,10): - newId = str(thisId) - if newId in self.__projectObjDict.keys(): - continue - - loadProject = True - projectId = newId - break - - if not loadProject: - self.console.clear() - self.console.append("The maximum number of projects has been loaded, a new project can not be loaded") - return None - - return projectId - - def openProject(self): - - self._disable_save_button() - self._disable_play_button() - - self.console.clear() - - self.frame_2.setEnabled(True) - - # print self.dir - filename = str(QtGui.QFileDialog.getOpenFileName(self, "Open a project file", self.pathWorkSpace, self.tr("Html Files (*.xml)"))) - - projectObjLoad = Project() - - if not projectObjLoad.readXml(filename): - self.console.append("The selected xml file could not be loaded ...") - return 0 - - self.create = False - self.refreshProjectWindow(projectObjLoad) - self.refreshProjectProperties(projectObjLoad) - - projectId = projectObjLoad.id - - if projectId in self.__projectObjDict.keys(): - - projectId = self.__getNewProjectId() - - if not projectId: - return 0 - - projectObjLoad.updateId(projectId) - - self.__projectObjDict[projectId] = projectObjLoad - - self.addProject2ProjectExplorer(id=projectId, name=projectObjLoad.name) - - self.tabWidgetProject.setEnabled(True) - self.tabWidgetProject.setCurrentWidget(self.tabProject) - # Disable tabProject after finish the creation - self.tabProject.setEnabled(True) - puObjorderList = OrderedDict(sorted(projectObjLoad.procUnitConfObjDict.items(), key=lambda x: x[0])) - - for puId, puObj in puObjorderList.items(): - - self.__puObjDict[puId] = puObj - - if puObj.name == "SendToServer": - self.saveFTPFromProcUnitObj(puObj) - - ############## COMPATIBLE WITH OLD VERSIONS ################ - operationObj = puObj.getOperationObj("SendByFTP") - - if operationObj: - self.saveFTPFromOpObj(operationObj) - ############################################################ - - if puObj.inputId == '0': - continue - - self.addPU2PELoadXML(puObj) - - self.refreshPUWindow(puObj) - self.refreshPUProperties(puObj) - self.showtabPUCreated(datatype=puObj.datatype) - -# self.console.clear() - self.console.append("\nThe selected xml file has been loaded successfully") - - if self.dateList: - self._disable_save_button() - self._enable_play_button() - - def create_updating_timer(self): - - self.comm_data_timer = QtCore.QTimer(self) - self.comm_data_timer.timeout.connect(self.on_comm_updating_timer) - self.comm_data_timer.start(1000) - - def on_comm_updating_timer(self): - # Verifica si algun proceso ha sido inicializado y sigue ejecutandose - # Si el proceso se ha parado actualizar el GUI (stopProject) - if not self.threadStarted: - return - - if self.controllerThread.isFinished(): - self.stopProject() - return - - def use_plotmanager(self, controllerThread): - - self.plotManager = controllerThread.useExternalPlotter() - - self.plot_timer = QtCore.QTimer() - self.plot_timer.timeout.connect(self.on_plotmanager_timer) - self.plot_timer.start(10) - - def on_plotmanager_timer(self): - - if not self.plotManager: - return - - self.plotManager.run() - - if self.plotManager.isErrorDetected(): - self.stopProject() - return - - def playProject(self, ext=".xml", save=1): - - self._disable_play_button() - self._disable_save_button() - - if self.controllerThread: - if self.controllerThread.isRunning(): - self.console.append("There is already another process running") - self._enable_stop_button() - return - - if not self.dateList: - self.console.append("No data found, check datapath") - - projectObj = self.getSelectedProjectObj() - - if not projectObj: - self.console.append("Please, select a project to start it") - return - - if save: - filename = self.saveProject() - if filename == None: - self.console.append("Process not initialized.") - return - else: - filename = TEMPORAL_FILE - projectObj.writeXml( os.path.join(self.pathWorkSpace,filename) ) - - self.console.clear() - self.console.append("Please wait...") - - self.controllerThread = ControllerThread() - self.controllerThread.readXml(filename) - - self.use_plotmanager(self.controllerThread) - - self.console.clear() - - self.controllerThread.start() - - sleep(0.5) - - - self.threadStarted = True - - self._disable_play_button() - self._disable_save_button() - self._enable_stop_button() - - def stopProject(self): - - self.threadStarted = False - self.controllerThread.stop() - self.plot_timer.stop() - - self.plotManager.join() - self.plotManager = None - - while self.controllerThread.isRunning(): - sleep(0.5) - - self._disable_stop_button() - self._enable_play_button() - - def pauseProject(self): - -# self.commCtrlPThread.cmd_q.put(ProcessCommand(ProcessCommand.PAUSE, data=True)) - paused = self.controllerThread.pause() - - self.changePauseIcon(paused) - - def saveProject(self, filename=None): - - self._disable_save_button() - self._disable_play_button() - - projectObj = self.getSelectedProjectObj() - - if not projectObj: - - if self.create: - self.console.append("Please press Ok before save it") - else: - self.console.append("Please select a project before save it") - return - - self.refreshGraphicsId() - - sts = True - selectedItemObj = self.getSelectedItemObj() - - #A Processing Unit has been selected - if projectObj == selectedItemObj: - if not self.on_proOk_clicked(): - return None - - #A Processing Unit has been selected - if projectObj != selectedItemObj: - puObj = selectedItemObj - - if puObj.name == 'VoltageProc': - sts = self.on_volOpOk_clicked() - if puObj.name == 'SpectraProc': - sts = self.on_specOpOk_clicked() - if puObj.name == 'SpectraHeisProc': - sts = self.on_specHeisOpOk_clicked() - - if not sts: - return None - - self.createFTPProcUnitView() - - if not filename: - filename = os.path.join( str(self.pathWorkSpace), "%s%s" %(str(projectObj.name), '.xml') ) - - projectObj.writeXml(filename) - self.console.clear() - self.console.append("Project saved") - self.console.append("Press Play button to start data processing ...") - - self._disable_save_button() - self._enable_play_button() - - return filename - - def removeItemTreeFromProject(self): - """ - Metodo para eliminar el proyecto en el dictionario de proyectos y en el dictionario de vista de arbol - """ - for key in self.__itemTreeDict.keys(): - - #Check again because an item can delete multiple items (childs) - if key not in self.__itemTreeDict.keys(): - continue - - if self.__itemTreeDict[key] != self.selectedItemTree: - continue - - if self.__projectObjDict.has_key(key) == True: - - del self.__projectObjDict[key] - del self.__itemTreeDict[key] - - else: - puObj = self.__puObjDict[key] - idProjectParent = puObj.parentId - projectObj = self.__projectObjDict[idProjectParent] - - del self.__puObjDict[key] - del self.__itemTreeDict[key] - del projectObj.procUnitConfObjDict[key] - - for key in projectObj.procUnitConfObjDict.keys(): - if projectObj.procUnitConfObjDict[key].inputId != puObj.getId(): - continue - del self.__puObjDict[projectObj.procUnitConfObjDict[key].getId()] - del self.__itemTreeDict[projectObj.procUnitConfObjDict[key].getId()] - del projectObj.procUnitConfObjDict[key] - # print projectObj.procUnitConfObjDict - # print self.__itemTreeDict,self.__projectObjDict,self.__puObjDict - - def setInputsProject_View(self): - - self.tabWidgetProject.setEnabled(True) - self.tabWidgetProject.setCurrentWidget(self.tabProject) - self.tabProject.setEnabled(True) - self.frame_2.setEnabled(False) - self.proName.clear() - self.proName.setFocus() - self.proName.setSelection(0, 0) - self.proName.setCursorPosition(0) - self.proDataType.setText('.r') - self.proDataPath.clear() - self.proComDataType.clear() - self.proComDataType.addItem("Voltage") - self.proComDataType.addItem("Spectra") - self.proComDataType.addItem("Fits") - self.proComDataType.addItem("USRP") - - self.proComStartDate.clear() - self.proComEndDate.clear() - - startTime = "00:00:00" - endTime = "23:59:59" - starlist = startTime.split(":") - endlist = endTime.split(":") - self.proDelay.setText("60") - self.proSet.setText("") - - self.labelSet.show() - self.proSet.show() - - self.labelIPPKm.hide() - self.proIPPKm.hide() - - self.time.setHMS(int(starlist[0]), int(starlist[1]), int(starlist[2])) - self.proStartTime.setTime(self.time) - self.time.setHMS(int(endlist[0]), int(endlist[1]), int(endlist[2])) - self.proEndTime.setTime(self.time) - self.proDescription.clear() - self.proOk.setEnabled(False) -# self.console.append("Please, Write a name Project") -# self.console.append("Introduce Project Parameters")DC -# self.console.append("Select data type Voltage( .rawdata) or Spectra(.pdata)") - - def clearPUWindow(self, datatype): - - projectObjView = self.getSelectedProjectObj() - - if not projectObjView: - return - - puObj = self.getSelectedItemObj() - inputId = puObj.getInputId() - inputPUObj = projectObjView.getProcUnitObj(inputId) - - if datatype == 'Voltage': - self.volOpComChannels.setEnabled(False) - self.volOpComHeights.setEnabled(False) - self.volOpFilter.setEnabled(False) - self.volOpComProfile.setEnabled(False) - self.volOpComCode.setEnabled(False) - self.volOpCohInt.setEnabled(False) - self.volOpChannel.setEnabled(False) - self.volOpHeights.setEnabled(False) - self.volOpProfile.setEnabled(False) - self.volOpRadarfrequency.setEnabled(False) - self.volOpSplitter.setEnabled(False) - self.volOpCombiner.setEnabled(False) - - self.volOpCebChannels.setCheckState(0) - self.volOpCebRadarfrequency.setCheckState(0) - self.volOpCebHeights.setCheckState(0) - self.volOpCebFilter.setCheckState(0) - self.volOpCebProfile.setCheckState(0) - self.volOpCebDecodification.setCheckState(0) - self.volOpCebCohInt.setCheckState(0) - self.volOpCebSplitter.setCheckState(0) - self.volOpCebCombiner.setCheckState(0) - - self.volOpChannel.clear() - self.volOpHeights.clear() - self.volOpProfile.clear() - self.volOpFilter.clear() - self.volOpCohInt.clear() - self.volOpRadarfrequency.clear() - self.volOpSplitter.clear() - self.volOpCombiner.clear() - - if datatype == 'Spectra': - - if inputPUObj.datatype == 'Spectra': - self.specOpnFFTpoints.setEnabled(False) - self.specOpProfiles.setEnabled(False) - self.specOpippFactor.setEnabled(False) - else: - self.specOpnFFTpoints.setEnabled(True) - self.specOpProfiles.setEnabled(True) - self.specOpippFactor.setEnabled(True) - - self.specOpCebCrossSpectra.setCheckState(0) - self.specOpCebChannel.setCheckState(0) - self.specOpCebHeights.setCheckState(0) - self.specOpCebIncoherent.setCheckState(0) - self.specOpCebRemoveDC.setCheckState(0) - self.specOpCebRemoveInt.setCheckState(0) - self.specOpCebgetNoise.setCheckState(0) - self.specOpCebRadarfrequency.setCheckState(0) - - self.specOpRadarfrequency.setEnabled(False) - self.specOppairsList.setEnabled(False) - self.specOpChannel.setEnabled(False) - self.specOpHeights.setEnabled(False) - self.specOpIncoherent.setEnabled(False) - self.specOpgetNoise.setEnabled(False) - - self.specOpRadarfrequency.clear() - self.specOpnFFTpoints.clear() - self.specOpProfiles.clear() - self.specOpippFactor.clear - self.specOppairsList.clear() - self.specOpChannel.clear() - self.specOpHeights.clear() - self.specOpIncoherent.clear() - self.specOpgetNoise.clear() - - self.specGraphCebSpectraplot.setCheckState(0) - self.specGraphCebCrossSpectraplot.setCheckState(0) - self.specGraphCebRTIplot.setCheckState(0) - self.specGraphCebRTInoise.setCheckState(0) - self.specGraphCebCoherencmap.setCheckState(0) - self.specGraphPowerprofile.setCheckState(0) - - self.specGraphSaveSpectra.setCheckState(0) - self.specGraphSaveCross.setCheckState(0) - self.specGraphSaveRTIplot.setCheckState(0) - self.specGraphSaveRTInoise.setCheckState(0) - self.specGraphSaveCoherencemap.setCheckState(0) - self.specGraphSavePowerprofile.setCheckState(0) - - self.specGraphftpRTIplot.setCheckState(0) - self.specGraphftpRTInoise.setCheckState(0) - self.specGraphftpCoherencemap.setCheckState(0) - - self.specGraphPath.clear() - self.specGraphPrefix.clear() - - self.specGgraphftpratio.clear() - - self.specGgraphChannelList.clear() - self.specGgraphFreq.clear() - self.specGgraphHeight.clear() - self.specGgraphDbsrange.clear() - self.specGgraphmagnitud.clear() - self.specGgraphTminTmax.clear() - self.specGgraphTimeRange.clear() - - if datatype == 'SpectraHeis': - self.specHeisOpCebIncoherent.setCheckState(0) - self.specHeisOpIncoherent.setEnabled(False) - self.specHeisOpIncoherent.clear() - - self.specHeisGraphCebSpectraplot.setCheckState(0) - self.specHeisGraphCebRTIplot.setCheckState(0) - - self.specHeisGraphSaveSpectra.setCheckState(0) - self.specHeisGraphSaveRTIplot.setCheckState(0) - - self.specHeisGraphftpSpectra.setCheckState(0) - self.specHeisGraphftpRTIplot.setCheckState(0) - - self.specHeisGraphPath.clear() - self.specHeisGraphPrefix.clear() - self.specHeisGgraphChannelList.clear() - self.specHeisGgraphXminXmax.clear() - self.specHeisGgraphYminYmax.clear() - self.specHeisGgraphTminTmax.clear() - self.specHeisGgraphTimeRange.clear() - self.specHeisGgraphftpratio.clear() - - def showtabPUCreated(self, datatype): - - if datatype == "Voltage": - self.tabVoltage.setEnabled(True) - self.tabProject.setEnabled(False) - self.tabSpectra.setEnabled(False) - self.tabCorrelation.setEnabled(False) - self.tabSpectraHeis.setEnabled(False) - self.tabWidgetProject.setCurrentWidget(self.tabVoltage) - - if datatype == "Spectra": - self.tabVoltage.setEnabled(False) - self.tabProject.setEnabled(False) - self.tabSpectra.setEnabled(True) - self.tabCorrelation.setEnabled(False) - self.tabSpectraHeis.setEnabled(False) - self.tabWidgetProject.setCurrentWidget(self.tabSpectra) - - if datatype == "SpectraHeis": - self.tabVoltage.setEnabled(False) - self.tabProject.setEnabled(False) - self.tabSpectra.setEnabled(False) - self.tabCorrelation.setEnabled(False) - self.tabSpectraHeis.setEnabled(True) - self.tabWidgetProject.setCurrentWidget(self.tabSpectraHeis) - - def checkInputsProject(self): - """ - Check Inputs Project: - - project_name - - datatype - - ext - - data_path - - readmode - - delay - - set - - walk - """ - parms_ok = True - project_name = str(self.proName.text()) - if project_name == '' or project_name == None: - outputstr = "Enter the Project Name" - self.console.append(outputstr) - parms_ok = False - project_name = None - - datatype = str(self.proComDataType.currentText()) - if not(datatype in ['Voltage', 'Spectra', 'Fits', 'USRP']): - outputstr = 'datatype = %s, this must be either Voltage, Spectra, SpectraHeis or USRP' % datatype - self.console.append(outputstr) - parms_ok = False - datatype = None - - ext = str(self.proDataType.text()) - if not(ext in ['.r', '.pdata', '.fits', '.hdf5']): - outputstr = "extension files must be .r , .pdata, .fits or .hdf5" - self.console.append(outputstr) - parms_ok = False - ext = None - - data_path = str(self.proDataPath.text()) - - if data_path == '': - outputstr = 'Datapath is empty' - self.console.append(outputstr) - parms_ok = False - data_path = None - - if data_path != None: - if not os.path.isdir(data_path): - outputstr = 'Datapath:%s does not exist' % data_path - self.console.append(outputstr) - parms_ok = False - data_path = None - - read_mode = str(self.proComReadMode.currentText()) - if not(read_mode in ['Online', 'Offline']): - outputstr = 'Read Mode: %s, this must be either Online or Offline' % read_mode - self.console.append(outputstr) - parms_ok = False - read_mode = None - - delay = None - if read_mode == "Online": - parms_ok = False - try: - delay = int(str(self.proDelay.text())) - parms_ok = True - except: - outputstr = 'Delay: %s, this must be a integer number' % str(self.proDelay.text()) - self.console.append(outputstr) - - try: - set = int(str(self.proSet.text())) - except: - # outputstr = 'Set: %s, this must be a integer number' % str(self.proName.text()) - # self.console.append(outputstr) - # parms_ok = False - set = None - - walk = int(self.proComWalk.currentIndex()) - expLabel = str(self.proExpLabel.text()) - - return parms_ok, project_name, datatype, ext, data_path, read_mode, delay, walk, set, expLabel - - def checkInputsPUSave(self, datatype): - """ - Check Inputs Spectra Save: - - path - - blocks Per File - - sufix - - dataformat - """ - parms_ok = True - - if datatype == "Voltage": - output_path = str(self.volOutputPath.text()) - blocksperfile = str(self.volOutputblocksperfile.text()) - profilesperblock = str(self.volOutputprofilesperblock.text()) - - if datatype == "Spectra": - output_path = str(self.specOutputPath.text()) - blocksperfile = str(self.specOutputblocksperfile.text()) - profilesperblock = 0 - - if datatype == "SpectraHeis": - output_path = str(self.specHeisOutputPath.text()) - blocksperfile = str(self.specHeisOutputblocksperfile.text()) - metadata_file = str(self.specHeisOutputMetada.text()) - - message = '' - - if not os.path.isdir(output_path): - message += 'OutputPath:%s does not exist\n' % output_path - parms_ok = False - - try: - profilesperblock = int(profilesperblock) - except: - if datatype == "Voltage": - message += 'Profilesperblock: %s, this must be a integer number\n' % str(self.volOutputprofilesperblock.text()) - parms_ok = False - profilesperblock = None - - try: - blocksperfile = int(blocksperfile) - except: - if datatype == "Voltage": - message += 'Blocksperfile: %s, this must be a integer number\n' % str(self.volOutputblocksperfile.text()) - elif datatype == "Spectra": - message += 'Blocksperfile: %s, this must be a integer number\n' % str(self.specOutputblocksperfile.text()) - elif datatype == "SpectraHeis": - message += 'Blocksperfile: %s, this must be a integer number\n' % str(self.specHeisOutputblocksperfile.text()) - - parms_ok = False - blocksperfile = None - - if datatype == "SpectraHeis": - if metadata_file != '': - if not os.path.isfile(metadata_file): - message += 'Metadata file %s does not exist\n' % metadata_file - parms_ok = False - - if str.strip(output_path) != '': - self.console.append(message) - - if datatype == "Voltage": - return parms_ok, output_path, blocksperfile, profilesperblock - - - if datatype == "Spectra": - return parms_ok, output_path, blocksperfile, profilesperblock - - - if datatype == "SpectraHeis": - return parms_ok, output_path, blocksperfile, metadata_file - - def findDatafiles(self, data_path, ext, walk, expLabel=''): - - dateList = [] - fileList = [] - - if ext == ".r": - from schainpy.model.io.jroIO_base import JRODataReader - - readerObj = JRODataReader() - dateList = readerObj.findDatafiles(path=data_path, - expLabel=expLabel, - ext=ext, - walk=walk) - - if ext == ".pdata": - from schainpy.model.io.jroIO_base import JRODataReader - - readerObj = JRODataReader() - dateList = readerObj.findDatafiles(path=data_path, - expLabel=expLabel, - ext=ext, - walk=walk) - - if ext == ".fits": - from schainpy.model.io.jroIO_base import JRODataReader - - readerObj = JRODataReader() - dateList = readerObj.findDatafiles(path=data_path, - expLabel=expLabel, - ext=ext, - walk=walk) - - if ext == ".hdf5": - from schainpy.model.io.jroIO_usrp import USRPReader - - readerObj = USRPReader() - dateList = readerObj.findDatafiles(path=data_path) - - return dateList - - def loadDays(self, data_path, ext, walk, expLabel=''): - """ - Method to loads day - """ -# self._disable_save_button() -# self._disable_play_button() -# self.proOk.setEnabled(False) - - self.proComStartDate.clear() - self.proComEndDate.clear() - - self.dateList = [] - - if not data_path: - self.console.append("Datapath has not been set") - return [] - - if not os.path.isdir(data_path): - self.console.append("Directory %s does not exist" %data_path) - return [] - - self.dataPath = data_path - - dateList = self.findDatafiles(data_path, ext=ext, walk=walk, expLabel=expLabel) - - if not dateList: -# self.console.clear() - if walk: - if expLabel: - outputstr = "No files (*%s) were found on %s/DOYPATH/%s" % (ext, data_path, expLabel) - else: - outputstr = "No files (*%s) were found on %s" % (ext, data_path) - else: - outputstr = "No files (*%s) were found on %s" % (ext, data_path) - - self.console.append(outputstr) - return [] - - dateStrList = [] - for thisDate in dateList: - dateStr = thisDate.strftime("%Y/%m/%d") - - self.proComStartDate.addItem(dateStr) - self.proComEndDate.addItem(dateStr) - dateStrList.append(dateStr) - - self.proComStartDate.setCurrentIndex(0) - self.proComEndDate.setCurrentIndex(self.proComEndDate.count() - 1) - - self.dateList = dateStrList - - self.console.clear() - self.console.append("Successful load") - -# self.proOk.setEnabled(True) -# self._enable_play_button() -# self._enable_save_button() - - return self.dateList - - def setWorkSpaceGUI(self, pathWorkSpace=None): - - if pathWorkSpace == None: - home = os.path.expanduser("~") - pathWorkSpace = os.path.join(home,'schain_workspace') - - self.pathWorkSpace = pathWorkSpace - - """ - Comandos Usados en Console - """ - def __del__(self): - sys.stdout = sys.__stdout__ - sys.stderr = sys.__stderr__ - - def normalOutputWritten(self, text): - color_black = QtGui.QColor(0,0,0) - self.console.setTextColor(color_black) - self.console.append(text) - - def errorOutputWritten(self, text): - color_red = QtGui.QColor(255,0,0) - color_black = QtGui.QColor(0,0,0) - - self.console.setTextColor(color_red) - self.console.append(text) - self.console.setTextColor(color_black) - - def _enable_save_button(self): - - self.actionSaveToolbar.setEnabled(True) - self.actionSave.setEnabled(True) - - def _disable_save_button(self): - - self.actionSaveToolbar.setEnabled(False) - self.actionSave.setEnabled(False) - - def _enable_play_button(self): - - if self.controllerThread: - if self.controllerThread.isRunning(): - return - - self.actionStart.setEnabled(True) - self.actionStarToolbar.setEnabled(True) - - self.changeStartIcon(started=False) - - def _disable_play_button(self): - - self.actionStart.setEnabled(False) - self.actionStarToolbar.setEnabled(False) - - self.changeStartIcon(started=True) - - def _enable_stop_button(self): - - self.actionPause.setEnabled(True) - self.actionStop.setEnabled(True) - - self.actionPauseToolbar.setEnabled(True) - self.actionStopToolbar.setEnabled(True) - - self.changePauseIcon(paused=False) - self.changeStopIcon(started=True) - - def _disable_stop_button(self): - - self.actionPause.setEnabled(False) - self.actionStop.setEnabled(False) - - self.actionPauseToolbar.setEnabled(False) - self.actionStopToolbar.setEnabled(False) - - self.changePauseIcon(paused=False) - self.changeStopIcon(started=False) - - def setGUIStatus(self): - - self.setWindowTitle("ROJ-Signal Chain") - self.setWindowIcon(QtGui.QIcon( os.path.join(FIGURES_PATH,"logo.png") )) - - self.tabWidgetProject.setEnabled(False) - self.tabVoltage.setEnabled(False) - self.tabSpectra.setEnabled(False) - self.tabCorrelation.setEnabled(False) - self.frame_2.setEnabled(False) - - self.actionCreate.setShortcut('Ctrl+N') - self.actionOpen.setShortcut('Ctrl+O') - self.actionSave.setShortcut('Ctrl+S') - self.actionClose.setShortcut('Ctrl+X') - - self.actionStart.setShortcut('Ctrl+1') - self.actionPause.setShortcut('Ctrl+2') - self.actionStop.setShortcut('Ctrl+3') - - self.actionFTP.setShortcut('Ctrl+F') - - self.actionStart.setEnabled(False) - self.actionPause.setEnabled(False) - self.actionStop.setEnabled(False) - - self.actionStarToolbar.setEnabled(False) - self.actionPauseToolbar.setEnabled(False) - self.actionStopToolbar.setEnabled(False) - - self.proName.clear() - self.proDataPath.setText('') - self.console.setReadOnly(True) - self.console.append("Welcome to Signal Chain\n\n") - self.console.append("Open a project or Create a new one\n") - self.proStartTime.setDisplayFormat("hh:mm:ss") - self.proDataType.setEnabled(False) - self.time = QtCore.QTime() - self.proEndTime.setDisplayFormat("hh:mm:ss") - startTime = "00:00:00" - endTime = "23:59:59" - starlist = startTime.split(":") - endlist = endTime.split(":") - self.time.setHMS(int(starlist[0]), int(starlist[1]), int(starlist[2])) - self.proStartTime.setTime(self.time) - self.time.setHMS(int(endlist[0]), int(endlist[1]), int(endlist[2])) - self.proEndTime.setTime(self.time) - self.proOk.setEnabled(False) - # set model Project Explorer - self.projectExplorerModel = QtGui.QStandardItemModel() - self.projectExplorerModel.setHorizontalHeaderLabels(("Project Explorer",)) - layout = QtGui.QVBoxLayout() - layout.addWidget(self.projectExplorerTree) - self.projectExplorerTree.setModel(self.projectExplorerModel) - self.projectExplorerTree.setContextMenuPolicy(QtCore.Qt.CustomContextMenu) - self.projectExplorerTree.customContextMenuRequested.connect(self.on_right_click) - self.projectExplorerTree.clicked.connect(self.on_click) - self.projectExplorerTree.expandAll() - # set model Project Properties - - self.propertiesModel = TreeModel() - self.propertiesModel.initProjectView() - self.treeProjectProperties.setModel(self.propertiesModel) - self.treeProjectProperties.expandAll() - self.treeProjectProperties.allColumnsShowFocus() - self.treeProjectProperties.resizeColumnToContents(1) - - # set Project - self.pronTxs.setEnabled(False) - self.proComByBlock.setEnabled(False) - self.proExpLabel.setEnabled(False) - self.proDelay.setEnabled(False) - self.proSet.setEnabled(True) - self.proDataType.setReadOnly(True) - - # set Operation Voltage - self.volOpComChannels.setEnabled(False) - self.volOpComHeights.setEnabled(False) - self.volOpFilter.setEnabled(False) - self.volOpComProfile.setEnabled(False) - self.volOpComCode.setEnabled(False) - self.volOpFlip.setEnabled(False) - self.volOpCohInt.setEnabled(False) - self.volOpRadarfrequency.setEnabled(False) - - self.volOpChannel.setEnabled(False) - self.volOpHeights.setEnabled(False) - self.volOpProfile.setEnabled(False) - self.volOpComMode.setEnabled(False) - - self.volOpSplitter.setEnabled(False) - self.volOpCombiner.setEnabled(False) - self.volOpAdjustHei.setEnabled(False) - - self.volOpCebAdjustHei.setEnabled(False) - - self.volGraphPath.setEnabled(False) - self.volGraphPrefix.setEnabled(False) - self.volGraphToolPath.setEnabled(False) - - # set Graph Voltage - self.volGraphChannelList.setEnabled(False) - self.volGraphIntensityRange.setEnabled(False) - self.volGraphHeightrange.setEnabled(False) - - # set Operation Spectra - self.specOpnFFTpoints.setEnabled(False) - self.specOpProfiles.setEnabled(False) - self.specOpippFactor.setEnabled(False) - self.specOppairsList.setEnabled(False) - - self.specOpComCrossSpectra.setEnabled(False) - self.specOpComChannel.setEnabled(False) - self.specOpComHeights.setEnabled(False) - self.specOpIncoherent.setEnabled(False) - self.specOpgetNoise.setEnabled(False) - self.specOpRadarfrequency.setEnabled(False) - - - self.specOpChannel.setEnabled(False) - self.specOpHeights.setEnabled(False) - # set Graph Spectra - self.specGgraphChannelList.setEnabled(False) - self.specGgraphFreq.setEnabled(False) - self.specGgraphHeight.setEnabled(False) - self.specGgraphDbsrange.setEnabled(False) - self.specGgraphmagnitud.setEnabled(False) - self.specGgraphTminTmax.setEnabled(False) - self.specGgraphTimeRange.setEnabled(False) - self.specGraphPath.setEnabled(False) - self.specGraphToolPath.setEnabled(False) - self.specGraphPrefix.setEnabled(False) - - self.specGgraphftpratio.setEnabled(False) - # set Operation SpectraHeis - self.specHeisOpIncoherent.setEnabled(False) - self.specHeisOpCobIncInt.setEnabled(False) - # set Graph SpectraHeis - self.specHeisGgraphChannelList.setEnabled(False) - self.specHeisGgraphXminXmax.setEnabled(False) - self.specHeisGgraphYminYmax.setEnabled(False) - self.specHeisGgraphTminTmax.setEnabled(False) - self.specHeisGgraphTimeRange.setEnabled(False) - self.specHeisGgraphftpratio.setEnabled(False) - self.specHeisGraphPath.setEnabled(False) - self.specHeisGraphPrefix.setEnabled(False) - self.specHeisGraphToolPath.setEnabled(False) - - self.proComWalk.setCurrentIndex(0) - - # tool tip gui - QtGui.QToolTip.setFont(QtGui.QFont('SansSerif', 10)) - self.projectExplorerTree.setToolTip('Right clik to add Project or Unit Process') - # tool tip gui project - - # tool tip gui volOp -# self.volOpChannel.setToolTip('Example: 1,2,3,4,5') -# self.volOpHeights.setToolTip('Example: 90,180') -# self.volOpFilter.setToolTip('Example: 2') -# self.volOpProfile.setToolTip('Example:0,127') -# self.volOpCohInt.setToolTip('Example: 128') -# self.volOpFlip.setToolTip('ChannelList where flip will be applied. Example: 0,2,3') -# self.volOpOk.setToolTip('If you have finished, please Ok ') -# # tool tip gui volGraph -# self.volGraphIntensityRange.setToolTip('Height range. Example: 50,100') -# self.volGraphHeightrange.setToolTip('Amplitude. Example: 0,10000') - # tool tip gui specOp -# self.specOpnFFTpoints.setToolTip('Example: 128') -# self.specOpProfiles.setToolTip('Example: 128') -# self.specOpippFactor.setToolTip('Example:1.0') -# self.specOpIncoherent.setToolTip('Example: 10') -# self.specOpgetNoise.setToolTip('Example:20,180,30,120 (minHei,maxHei,minVel,maxVel)') -# -# self.specOpChannel.setToolTip('Example: 0,1,2,3') -# self.specOpHeights.setToolTip('Example: 90,180') -# self.specOppairsList.setToolTip('Example: (0,1),(2,3)') -# # tool tip gui specGraph -# -# self.specGgraphChannelList.setToolTip('Example: 0,3,4') -# self.specGgraphFreq.setToolTip('Example: -20,20') -# self.specGgraphHeight.setToolTip('Example: 100,400') -# self.specGgraphDbsrange.setToolTip('Example: 30,170') -# -# self.specGraphPrefix.setToolTip('Example: EXPERIMENT_NAME') -# -# -# self.specHeisOpIncoherent.setToolTip('Example: 10') -# -# self.specHeisGgraphChannelList.setToolTip('Example: 0,2,3') -# self.specHeisGgraphXminXmax.setToolTip('Example (Hz): -1000, 1000') -# self.specHeisGgraphYminYmax.setToolTip('Example (dB): 5, 35') -# self.specHeisGgraphTminTmax.setToolTip('Example (hours): 0, 24') -# self.specHeisGgraphTimeRange.setToolTip('Example (hours): 8') - - self.labelSet.show() - self.proSet.show() - - self.labelIPPKm.hide() - self.proIPPKm.hide() - - sys.stdout = ShowMeConsole(textWritten=self.normalOutputWritten) -# sys.stderr = ShowMeConsole(textWritten=self.errorOutputWritten) - - -class UnitProcessWindow(QMainWindow, Ui_UnitProcess): - """ - Class documentation goes here. - """ - closed = pyqtSignal() - create = False - - def __init__(self, parent=None): - """ - Constructor - """ - QMainWindow.__init__(self, parent) - self.setupUi(self) - self.getFromWindow = None - self.getfromWindowList = [] - self.dataTypeProject = None - - self.listUP = None - - @pyqtSignature("") - def on_unitPokbut_clicked(self): - """ - Slot documentation goes here. - """ - self.create = True - self.getFromWindow = self.getfromWindowList[int(self.comboInputBox.currentIndex())] - # self.nameofUP= str(self.nameUptxt.text()) - self.typeofUP = str(self.comboTypeBox.currentText()) - self.close() - - - @pyqtSignature("") - def on_unitPcancelbut_clicked(self): - """ - Slot documentation goes here. - """ - self.create = False - self.close() - - def loadTotalList(self): - self.comboInputBox.clear() - for i in self.getfromWindowList: - - name = i.getElementName() - if name == 'Project': - id = i.id - name = i.name - if self.dataTypeProject == 'Voltage': - self.comboTypeBox.clear() - self.comboTypeBox.addItem("Voltage") - - if self.dataTypeProject == 'Spectra': - self.comboTypeBox.clear() - self.comboTypeBox.addItem("Spectra") - self.comboTypeBox.addItem("Correlation") - if self.dataTypeProject == 'Fits': - self.comboTypeBox.clear() - self.comboTypeBox.addItem("SpectraHeis") - - - if name == 'ProcUnit': - id = int(i.id) - 1 - name = i.datatype - if name == 'Voltage': - self.comboTypeBox.clear() - self.comboTypeBox.addItem("Spectra") - self.comboTypeBox.addItem("SpectraHeis") - self.comboTypeBox.addItem("Correlation") - if name == 'Spectra': - self.comboTypeBox.clear() - self.comboTypeBox.addItem("Spectra") - self.comboTypeBox.addItem("SpectraHeis") - self.comboTypeBox.addItem("Correlation") - if name == 'SpectraHeis': - self.comboTypeBox.clear() - self.comboTypeBox.addItem("SpectraHeis") - - self.comboInputBox.addItem(str(name)) - # self.comboInputBox.addItem(str(name)+str(id)) - - def closeEvent(self, event): - self.closed.emit() - event.accept() - -class Ftp(QMainWindow, Ui_Ftp): - """ - Class documentation goes here. - """ - create = False - closed = pyqtSignal() - server = None - remotefolder = None - username = None - password = None - ftp_wei = None - exp_code = None - sub_exp_code = None - plot_pos = None - - def __init__(self, parent=None): - """ - Constructor - """ - QMainWindow.__init__(self, parent) - self.setupUi(self) - self.setGUIStatus() - - def setGUIStatus(self): - self.setWindowTitle("ROJ-Signal Chain") - self.serverFTP.setToolTip('Example: jro-app.igp.gob.pe') - self.folderFTP.setToolTip('Example: /home/wmaster/graficos') - self.usernameFTP.setToolTip('Example: myusername') - self.passwordFTP.setToolTip('Example: mypass ') - self.weightFTP.setToolTip('Example: 0') - self.expcodeFTP.setToolTip('Example: 0') - self.subexpFTP.setToolTip('Example: 0') - self.plotposFTP.setToolTip('Example: 0') - - def setParmsfromTemporal(self, server, remotefolder, username, password, ftp_wei, exp_code, sub_exp_code, plot_pos): - self.serverFTP.setText(str(server)) - self.folderFTP.setText(str(remotefolder)) - self.usernameFTP.setText(str(username)) - self.passwordFTP.setText(str(password)) - self.weightFTP.setText(str(ftp_wei)) - self.expcodeFTP.setText(str(exp_code)) - self.subexpFTP.setText(str(sub_exp_code)) - self.plotposFTP.setText(str(plot_pos)) - - def getParmsFromFtpWindow(self): - """ - Return Inputs Project: - - server - - remotefolder - - username - - password - - ftp_wei - - exp_code - - sub_exp_code - - plot_pos - """ - name_server_ftp = str(self.serverFTP.text()) - if not name_server_ftp: - self.console.clear() - self.console.append("Please Write a FTP Server") - return 0 - - folder_server_ftp = str(self.folderFTP.text()) - if not folder_server_ftp: - self.console.clear() - self.console.append("Please Write a Folder") - return 0 - - username_ftp = str(self.usernameFTP.text()) - if not username_ftp: - self.console.clear() - self.console.append("Please Write a User Name") - return 0 - - password_ftp = str(self.passwordFTP.text()) - if not password_ftp: - self.console.clear() - self.console.append("Please Write a passwordFTP") - return 0 - - ftp_wei = str(self.weightFTP.text()) - if not ftp_wei == "": - try: - ftp_wei = int(self.weightFTP.text()) - except: - self.console.clear() - self.console.append("Please Write a ftp_wei number") - return 0 - - exp_code = str(self.expcodeFTP.text()) - if not exp_code == "": - try: - exp_code = int(self.expcodeFTP.text()) - except: - self.console.clear() - self.console.append("Please Write a exp_code number") - return 0 - - - sub_exp_code = str(self.subexpFTP.text()) - if not sub_exp_code == "": - try: - sub_exp_code = int(self.subexpFTP.text()) - except: - self.console.clear() - self.console.append("Please Write a sub_exp_code number") - return 0 - - plot_pos = str(self.plotposFTP.text()) - if not plot_pos == "": - try: - plot_pos = int(self.plotposFTP.text()) - except: - self.console.clear() - self.console.append("Please Write a plot_pos number") - return 0 - - return name_server_ftp, folder_server_ftp, username_ftp, password_ftp, ftp_wei, exp_code, sub_exp_code, plot_pos - - @pyqtSignature("") - def on_ftpOkButton_clicked(self): - server, remotefolder, username, password, ftp_wei, exp_code, sub_exp_code, plot_pos = self.getParmsFromFtpWindow() - self.create = True - self.close() - - @pyqtSignature("") - def on_ftpCancelButton_clicked(self): - self.create = False - self.close() - - def closeEvent(self, event): - self.closed.emit() - event.accept() - -class ftpBuffer(): - - server = None - remotefolder = None - username = None - password = None - ftp_wei = None - exp_code = None - sub_exp_code = None - plot_pos = None - create = False - withoutconfig = False - createforView = False - localfolder = None - extension = None - period = None - protocol = None - - def __init__(self): - - self.create = False - self.server = None - self.remotefolder = None - self.username = None - self.password = None - self.ftp_wei = None - self.exp_code = None - self.sub_exp_code = None - self.plot_pos = None - # self.create = False - self.localfolder = None - self.extension = None - self.period = None - self.protocol = None - - def setwithoutconfiguration(self): - - self.create = False - self.server = "jro-app.igp.gob.pe" - self.remotefolder = "/home/wmaster/graficos" - self.username = "wmaster" - self.password = "mst2010vhf" - self.withoutconfig = True - self.localfolder = './' - self.extension = '.png' - self.period = 60 - self.protocol = 'ftp' - self.createforView = True - - if not self.ftp_wei: - self.ftp_wei = 0 - - if not self.exp_code: - self.exp_code = 0 - - if not self.sub_exp_code: - self.sub_exp_code = 0 - - if not self.plot_pos: - self.plot_pos = 0 - - def save(self, server, remotefolder, username, password, ftp_wei=0, exp_code=0, sub_exp_code=0, plot_pos=0, localfolder='./', extension='.png', period=60, protocol='ftp'): - - self.server = server - self.remotefolder = remotefolder - self.username = username - self.password = password - self.ftp_wei = ftp_wei - self.exp_code = exp_code - self.sub_exp_code = sub_exp_code - self.plot_pos = plot_pos - self.create = True - self.withoutconfig = False - self.createforView = True - self.localfolder = localfolder - self.extension = extension - self.period = period - self.protocol = protocol - - def recover(self): - - return self.server, self.remotefolder, self.username, self.password, self.ftp_wei, self.exp_code, self.sub_exp_code, self.plot_pos, self.extension, self.period, self.protocol - -class ShowMeConsole(QtCore.QObject): - - textWritten = QtCore.pyqtSignal(str) - - def write(self, text): - - if len(text) == 0: - self.textWritten.emit("\n") - return - - if text[-1] == "\n": - text = text[:-1] - - self.textWritten.emit(str(text)) - - def flush(self): - pass diff --git a/schainpy/gui/viewcontroller/ftp.py b/schainpy/gui/viewcontroller/ftp.py deleted file mode 100644 index 5b08bbc..0000000 --- a/schainpy/gui/viewcontroller/ftp.py +++ /dev/null @@ -1,21 +0,0 @@ -# -*- coding: utf-8 -*- - -""" -Module implementing Ftp. -""" - -from PyQt4.QtGui import QMainWindow -from PyQt4.QtCore import pyqtSignature - -from schainpy.gui.viewer.ftp import Ui_Ftp - -class Ftp(QMainWindow, Ui_MainWindow): - """ - Class documentation goes here. - """ - def __init__(self, parent = None): - """ - Constructor - """ - QMainWindow.__init__(self, parent) - self.setupUi(self) diff --git a/schainpy/gui/viewcontroller/initwindow.py b/schainpy/gui/viewcontroller/initwindow.py deleted file mode 100644 index 3ee1396..0000000 --- a/schainpy/gui/viewcontroller/initwindow.py +++ /dev/null @@ -1,46 +0,0 @@ -# -*- coding: utf-8 -*- - -""" -Module implementing InitWindow. -""" -import os - -from PyQt4.QtGui import QDialog -from PyQt4.QtCore import pyqtSignature -from PyQt4 import QtCore -from PyQt4 import QtGui - -from schainpy.gui.viewer.ui_initwindow import Ui_InitWindow -from schainpy.gui.figures import tools - -FIGURES_PATH = tools.get_path() - -class InitWindow(QDialog, Ui_InitWindow): - """ - Class documentation goes here. - """ - def __init__(self, parent = None): - """ - Constructor - """ - QDialog.__init__(self, parent) - self.setupUi(self) - self.setWindowTitle("ROJ-Signal Chain") - self.setWindowIcon(QtGui.QIcon( os.path.join(FIGURES_PATH,"logo.png") )) - - @pyqtSignature("") - def on_ExitBtn_clicked(self): - """ - Exit cierra la ventana de Bienvenida - """ - self.close() - - @pyqtSignature("") - def on_ContinueBtn_clicked(self): - """ - Continue cierra la ventana de Bienvenida, a este evento se le complementa con la accion - conectar con la ventana de configuracion de Workspace - """ - # TODO: not implemented yet - #raise NotImplementedError - self.accept() diff --git a/schainpy/gui/viewcontroller/parametersModel.py b/schainpy/gui/viewcontroller/parametersModel.py deleted file mode 100644 index 665c01f..0000000 --- a/schainpy/gui/viewcontroller/parametersModel.py +++ /dev/null @@ -1,92 +0,0 @@ -""" -Classes to save parameters from Windows. - --Project window --Voltage window --Spectra window --SpectraHeis window --Correlation window - -""" - -class ProjectParms(): - - parmsOk = False - name = None - description = None - datatype = None - ext = None - dpath = None - startDate = None - endDate = None - startTime = None - endTime = None - online = None - delay = None - walk = None - expLabel = None - set = None - ippKm = None - - def __init__(self): - - self.parmsOk = True - self.description = '' - self.expLabel = '' - self.set = '' - self.ippKm = '' - self.walk = None - self.delay = '' - - def getDatatypeIndex(self): - - indexDatatype = None - - if 'voltage' in self.datatype.lower(): - indexDatatype = 0 - if 'spectra' in self.datatype.lower(): - indexDatatype = 1 - if 'fits' in self.datatype.lower(): - indexDatatype = 2 - if 'usrp' in self.datatype.lower(): - indexDatatype = 3 - return indexDatatype - - def getExt(self): - - ext = None - - if self.datatype.lower() == 'voltage': - ext = '.r' - if self.datatype.lower() == 'spectra': - ext = '.pdata' - if self.datatype.lower() == 'fits': - ext = '.fits' - if self.datatype.lower() == 'usrp': - ext = '.hdf5' - return ext - - def set(self, project_name, datatype, ext, dpath, online, - startDate=None, endDate=None, startTime=None, endTime=None, - delay=None, walk=None, set=None, ippKm=None, parmsOk=True, expLabel=''): - - name = project_name - datatype = datatype - ext = ext - dpath = dpath - startDate = startDate - endDate = endDate - startTime = startTime - endTime = endTime - online = online - delay = delay - walk = walk - set = set - ippKm = ippKm - expLabel = expLabel - - self.parmsOk = parmsOk - - def isValid(self): - - return self.parmsOk \ No newline at end of file diff --git a/schainpy/gui/viewcontroller/propertiesViewModel.py b/schainpy/gui/viewcontroller/propertiesViewModel.py deleted file mode 100644 index 1a278ee..0000000 --- a/schainpy/gui/viewcontroller/propertiesViewModel.py +++ /dev/null @@ -1,211 +0,0 @@ -# -*- coding: utf-8 -*- -""" -This module contains every model class to create, modify and show a property tree on a GUI. -""" - -from PyQt4 import QtCore -import itertools - -HORIZONTAL_HEADERS = ("Property","Value " ) - -HORIZONTAL = ("RAMA :",) - -class PropertyBuffer(): - - def __init__(self): - - self.clear() - - def clear(self): - - self.headerList = [] - self.parmList = [] - self.valueList = [] - - def append(self, header, parm, value): - - self.headerList.append(header) - self.parmList.append(parm) - self.valueList.append(value) - - return - - def get(self): - - return self.headerList, self.parmList, self.valueList - - def getPropertyModel(self): - - propertiesModel = TreeModel() - propertiesModel.showProperties(self.headerList, self.parmList, self.valueList) - - return propertiesModel - - -class TreeModel(QtCore.QAbstractItemModel): - ''' - a model to display a few names, ordered by encabezado - - ''' - def __init__(self ,parent=None): - super(TreeModel, self).__init__(parent) - self.people = [] - - def initProjectView(self): - """ - Reemplazo del método showtree - """ - HORIZONTAL_HEADERS = ("Property","Value " ) - HORIZONTAL = ("RAMA :",) - self.rootItem = TreeItem(None, "ALL", None) - self.parents = {0 : self.rootItem} - self.__setupModelData() - - def initPUVoltageView(self): - HORIZONTAL_HEADERS = ("Operation"," Parameter Value " ) - HORIZONTAL = ("RAMA :",) - self.rootItem = TreeItem(None, "ALL", None) - self.parents = {0 : self.rootItem} - self.__setupModelData() - - def showProperties(self,headerList, parmList, valueList): - """ - set2Obje - """ - for header, parameter, value in itertools.izip(headerList, parmList, valueList): - person = person_class(header, parameter, value) - self.people.append(person) - - self.rootItem = TreeItem(None, "ALL", None) - self.parents = {0 : self.rootItem} - self.__setupModelData() - - def columnCount(self, parent=None): - if parent and parent.isValid(): - return parent.internalPointer().columnCount() - else: - return len(HORIZONTAL_HEADERS) - - def data(self, index, role): - if not index.isValid(): - return QtCore.QVariant() - - item = index.internalPointer() - if role == QtCore.Qt.DisplayRole: - return item.data(index.column()) - if role == QtCore.Qt.UserRole: - if item: - return item.person - - return QtCore.QVariant() - - def index(self, row, column, parent): - if not self.hasIndex(row, column, parent): - return QtCore.QModelIndex() - - if not parent.isValid(): - parentItem = self.rootItem - else: - parentItem = parent.internalPointer() - - childItem = parentItem.child(row) - if childItem: - return self.createIndex(row, column, childItem) - else: - return QtCore.QModelIndex() - - def parent(self, index): - if not index.isValid(): - return QtCore.QModelIndex() - - childItem = index.internalPointer() - if not childItem: - return QtCore.QModelIndex() - - parentItem = childItem.parent() - - if parentItem == self.rootItem: - return QtCore.QModelIndex() - - return self.createIndex(parentItem.row(), 0, parentItem) - - def rowCount(self, parent=QtCore.QModelIndex()): - if parent.column() > 0: - return 0 - if not parent.isValid(): - p_Item = self.rootItem - else: - p_Item = parent.internalPointer() - return p_Item.childCount() - - def __setupModelData(self): - for person in self.people: - if person.value: - encabezado = person.header - - if not self.parents.has_key(encabezado): - newparent = TreeItem(None, encabezado, self.rootItem) - self.rootItem.appendChild(newparent) - - self.parents[encabezado] = newparent - - parentItem = self.parents[encabezado] - newItem = TreeItem(person, "", parentItem) - parentItem.appendChild(newItem) - -class person_class(object): - ''' - a trivial custom data object - ''' - def __init__(self, header, parameter, value): - self.header = header - self.parameter = parameter - self.value = value - - def __repr__(self): - return "PERSON - %s %s"% (self.parameter, self.header) - -class TreeItem(object): - ''' - a python object used to return row/column data, and keep note of - it's parents and/or children - ''' - def __init__(self, person, header, parentItem): - self.person = person - self.parentItem = parentItem - self.header = header - self.childItems = [] - - def appendChild(self, item): - self.childItems.append(item) - - def child(self, row): - return self.childItems[row] - - def childCount(self): - return len(self.childItems) - - def columnCount(self): - return 2 - - def data(self, column): - if self.person == None: - if column == 0: - return QtCore.QVariant(self.header) - if column == 1: - return QtCore.QVariant("") - else: - if column == 0: - return QtCore.QVariant(self.person.parameter) - if column == 1: - return QtCore.QVariant(self.person.value) - return QtCore.QVariant() - - def parent(self): - return self.parentItem - - def row(self): - if self.parentItem: - return self.parentItem.childItems.index(self) - return 0 - \ No newline at end of file diff --git a/schainpy/gui/viewcontroller/workspace.py b/schainpy/gui/viewcontroller/workspace.py deleted file mode 100644 index 4ad7729..0000000 --- a/schainpy/gui/viewcontroller/workspace.py +++ /dev/null @@ -1,70 +0,0 @@ -# -*- coding: utf-8 -*- -import os -from os.path import expanduser - -from PyQt4.QtGui import QDialog -from PyQt4.QtCore import pyqtSignature -from PyQt4.QtCore import pyqtSignal -from PyQt4 import QtGui, QtCore - -from schainpy.gui.viewer.ui_workspace import Ui_Workspace -from schainpy.gui.figures import tools - -FIGURES_PATH = tools.get_path() - -class Workspace(QDialog, Ui_Workspace): - """ - Class documentation goes here. - """ - - def __init__(self, parent = None): - """ - Constructor - """ - QDialog.__init__(self, parent) - self.dirList=[] - self.setupUi(self) - self.setWindowTitle("ROJ-Signal Chain") - self.setWindowIcon(QtGui.QIcon( os.path.join(FIGURES_PATH,"logo.png") )) - #*####### DIRECTORIO DE TRABAJO #########*# - #self.dirCmbBox.setItemText(0, QtGui.QApplication.translate("MainWindow", "C:\WorkSpaceGui", None, QtGui.QApplication.UnicodeUTF8)) - home=expanduser("~") - self.dir=os.path.join(home,'schain_workspace') - if not os.path.exists(self.dir): - os.makedirs(self.dir) - self.dirComBox.addItem(self.dir) - self.i=0 - - - @pyqtSignature("") - def on_dirToolPath_clicked(self): - """ - Slot documentation goes here. - """ - self.i +=1 - self.dirBrowse = str(QtGui.QFileDialog.getExistingDirectory(self, 'Open Directory', './', QtGui.QFileDialog.ShowDirsOnly)) - self.dirComBox.addItem(self.dirBrowse) - self.dirComBox.setCurrentIndex(self.i) - - - - @pyqtSignature("") - def on_dirOkBtn_clicked(self): - """ - VISTA DE INTERFAZ GRÁFICA - """ - self.accept() - # self.close() -# - @pyqtSignature("") - def on_dirCancelBtn_clicked(self): - """ - Cerrar - """ - self.close() - - - - - - \ No newline at end of file diff --git a/schainpy/gui/viewer/__init__.py b/schainpy/gui/viewer/__init__.py deleted file mode 100644 index 416f53f..0000000 --- a/schainpy/gui/viewer/__init__.py +++ /dev/null @@ -1,5 +0,0 @@ -# import ui_initwindow -# import ui_workspace -# import ui_mainwindow -# import ui_window -# import ui_ftp \ No newline at end of file diff --git a/schainpy/gui/viewer/ui_ftp.py b/schainpy/gui/viewer/ui_ftp.py deleted file mode 100644 index b785509..0000000 --- a/schainpy/gui/viewer/ui_ftp.py +++ /dev/null @@ -1,140 +0,0 @@ -# -*- coding: utf-8 -*- - -# Form implementation generated from reading ui file '/home/alex/ui/ftpConfig4.ui' -# -# Created: Tue Aug 20 08:24:35 2013 -# by: PyQt4 UI code generator 4.10 -# -# WARNING! All changes made in this file will be lost! - -from PyQt4 import QtCore, QtGui - -try: - _fromUtf8 = QtCore.QString.fromUtf8 -except AttributeError: - def _fromUtf8(s): - return s - -try: - _encoding = QtGui.QApplication.UnicodeUTF8 - def _translate(context, text, disambig): - return QtGui.QApplication.translate(context, text, disambig, _encoding) -except AttributeError: - def _translate(context, text, disambig): - return QtGui.QApplication.translate(context, text, disambig) - -class Ui_Ftp(object): - def setupUi(self, MainWindow): - MainWindow.setObjectName(_fromUtf8("MainWindow")) - MainWindow.resize(374, 399) - MainWindow.setMinimumSize(QtCore.QSize(374, 399)) - MainWindow.setMaximumSize(QtCore.QSize(374, 399)) - self.centralWidget = QtGui.QWidget(MainWindow) - self.centralWidget.setObjectName(_fromUtf8("centralWidget")) - self.label = QtGui.QLabel(self.centralWidget) - self.label.setGeometry(QtCore.QRect(9, 38, 47, 17)) - self.label.setObjectName(_fromUtf8("label")) - self.label_2 = QtGui.QLabel(self.centralWidget) - self.label_2.setGeometry(QtCore.QRect(9, 133, 77, 17)) - self.label_2.setObjectName(_fromUtf8("label_2")) - self.label_3 = QtGui.QLabel(self.centralWidget) - self.label_3.setGeometry(QtCore.QRect(9, 166, 68, 17)) - self.label_3.setObjectName(_fromUtf8("label_3")) - self.label_4 = QtGui.QLabel(self.centralWidget) - self.label_4.setGeometry(QtCore.QRect(9, 9, 101, 17)) - font = QtGui.QFont() - font.setBold(True) - font.setWeight(75) - self.label_4.setFont(font) - self.label_4.setObjectName(_fromUtf8("label_4")) - self.label_5 = QtGui.QLabel(self.centralWidget) - self.label_5.setGeometry(QtCore.QRect(9, 104, 87, 17)) - font = QtGui.QFont() - font.setBold(True) - font.setWeight(75) - self.label_5.setFont(font) - self.label_5.setObjectName(_fromUtf8("label_5")) - self.label_6 = QtGui.QLabel(self.centralWidget) - self.label_6.setGeometry(QtCore.QRect(9, 71, 47, 17)) - self.label_6.setObjectName(_fromUtf8("label_6")) - self.serverFTP = QtGui.QLineEdit(self.centralWidget) - self.serverFTP.setGeometry(QtCore.QRect(130, 40, 231, 27)) - self.serverFTP.setObjectName(_fromUtf8("serverFTP")) - self.folderFTP = QtGui.QLineEdit(self.centralWidget) - self.folderFTP.setGeometry(QtCore.QRect(130, 70, 231, 27)) - self.folderFTP.setObjectName(_fromUtf8("folderFTP")) - self.usernameFTP = QtGui.QLineEdit(self.centralWidget) - self.usernameFTP.setGeometry(QtCore.QRect(130, 130, 231, 27)) - self.usernameFTP.setObjectName(_fromUtf8("usernameFTP")) - self.passwordFTP = QtGui.QLineEdit(self.centralWidget) - self.passwordFTP.setEchoMode(QtGui.QLineEdit.Password) - self.passwordFTP.setGeometry(QtCore.QRect(130, 160, 231, 27)) - self.passwordFTP.setObjectName(_fromUtf8("passwordFTP")) - self.ftpCancelButton = QtGui.QPushButton(self.centralWidget) - self.ftpCancelButton.setGeometry(QtCore.QRect(130, 360, 111, 27)) - self.ftpCancelButton.setObjectName(_fromUtf8("ftpCancelButton")) - self.ftpOkButton = QtGui.QPushButton(self.centralWidget) - self.ftpOkButton.setGeometry(QtCore.QRect(250, 360, 111, 27)) - self.ftpOkButton.setObjectName(_fromUtf8("ftpOkButton")) - self.label_7 = QtGui.QLabel(self.centralWidget) - self.label_7.setGeometry(QtCore.QRect(10, 200, 66, 17)) - font = QtGui.QFont() - font.setBold(True) - font.setWeight(75) - self.label_7.setFont(font) - self.label_7.setObjectName(_fromUtf8("label_7")) - self.label_8 = QtGui.QLabel(self.centralWidget) - self.label_8.setGeometry(QtCore.QRect(10, 230, 81, 17)) - self.label_8.setObjectName(_fromUtf8("label_8")) - self.label_9 = QtGui.QLabel(self.centralWidget) - self.label_9.setGeometry(QtCore.QRect(10, 260, 81, 17)) - self.label_9.setObjectName(_fromUtf8("label_9")) - self.label_10 = QtGui.QLabel(self.centralWidget) - self.label_10.setGeometry(QtCore.QRect(10, 290, 81, 17)) - self.label_10.setObjectName(_fromUtf8("label_10")) - self.label_11 = QtGui.QLabel(self.centralWidget) - self.label_11.setGeometry(QtCore.QRect(10, 320, 81, 17)) - self.label_11.setObjectName(_fromUtf8("label_11")) - self.weightFTP = QtGui.QLineEdit(self.centralWidget) - self.weightFTP.setGeometry(QtCore.QRect(130, 230, 231, 27)) - self.weightFTP.setObjectName(_fromUtf8("weightFTP")) - self.expcodeFTP = QtGui.QLineEdit(self.centralWidget) - self.expcodeFTP.setGeometry(QtCore.QRect(130, 260, 231, 27)) - self.expcodeFTP.setObjectName(_fromUtf8("expcodeFTP")) - self.subexpFTP = QtGui.QLineEdit(self.centralWidget) - self.subexpFTP.setGeometry(QtCore.QRect(130, 290, 231, 27)) - self.subexpFTP.setObjectName(_fromUtf8("subexpFTP")) - self.plotposFTP = QtGui.QLineEdit(self.centralWidget) - self.plotposFTP.setGeometry(QtCore.QRect(130, 320, 231, 27)) - self.plotposFTP.setObjectName(_fromUtf8("plotposFTP")) - MainWindow.setCentralWidget(self.centralWidget) - - self.retranslateUi(MainWindow) - QtCore.QMetaObject.connectSlotsByName(MainWindow) - - def retranslateUi(self, MainWindow): - MainWindow.setWindowTitle(_translate("MainWindow", "MainWindow", None)) - self.label.setText(_translate("MainWindow", "Server:", None)) - self.label_2.setText(_translate("MainWindow", "User Name:", None)) - self.label_3.setText(_translate("MainWindow", "Password:", None)) - self.label_4.setText(_translate("MainWindow", "Server Details", None)) - self.label_5.setText(_translate("MainWindow", "User Details", None)) - self.label_6.setText(_translate("MainWindow", "Remote Folder:", None)) - self.ftpCancelButton.setText(_translate("MainWindow", "Cancel", None)) - self.ftpOkButton.setText(_translate("MainWindow", "Ok", None)) - self.label_7.setText(_translate("MainWindow", "Project Details", None)) - self.label_8.setText(_translate("MainWindow", "Ftp_wei:", None)) - self.label_9.setText(_translate("MainWindow", "Exp_code:", None)) - self.label_10.setText(_translate("MainWindow", "Sub_exp:", None)) - self.label_11.setText(_translate("MainWindow", "Plot_pos:", None)) - - -if __name__ == "__main__": - import sys - app = QtGui.QApplication(sys.argv) - MainWindow = QtGui.QMainWindow() - ui = Ui_Ftp() - ui.setupUi(MainWindow) - MainWindow.show() - sys.exit(app.exec_()) - diff --git a/schainpy/gui/viewer/ui_initwindow.py b/schainpy/gui/viewer/ui_initwindow.py deleted file mode 100644 index d6bc947..0000000 --- a/schainpy/gui/viewer/ui_initwindow.py +++ /dev/null @@ -1,95 +0,0 @@ -# -*- coding: utf-8 -*- - -# Form implementation generated from reading ui file '/home/roj-idl71/SignalChain/initwindowv2.ui' -# -# Created: Wed Mar 6 15:32:39 2013 -# by: PyQt4 UI code generator 4.8.6 -# -# WARNING! All changes made in this file will be lost! - -from PyQt4 import QtCore, QtGui - -try: - _fromUtf8 = QtCore.QString.fromUtf8 -except AttributeError: - _fromUtf8 = lambda s: s - -import os -from schainpy.gui.figures import tools -from schainpy import __version__ - -INITIAL_MSG = "Signal Chain GUI - v%s" %__version__ -FIGURES_PATH = tools.get_path() - -class Ui_InitWindow(object): - def setupUi(self, Dialog): - Dialog.setObjectName(_fromUtf8("Dialog")) - Dialog.resize(652, 496) - Dialog.setWindowTitle(QtGui.QApplication.translate("Dialog", "Dialog", None, QtGui.QApplication.UnicodeUTF8)) - self.gridLayout = QtGui.QGridLayout(Dialog) - self.gridLayout.setObjectName(_fromUtf8("gridLayout")) - self.verticalLayout_3 = QtGui.QVBoxLayout() - self.verticalLayout_3.setObjectName(_fromUtf8("verticalLayout_3")) - self.verticalLayout_4 = QtGui.QVBoxLayout() - self.verticalLayout_4.setObjectName(_fromUtf8("verticalLayout_4")) - self.label_3 = QtGui.QLabel(Dialog) - font = QtGui.QFont() - font.setFamily(_fromUtf8("Cambria")) - font.setPointSize(22) - font.setBold(False) - font.setWeight(50) - self.label_3.setFont(font) - self.label_3.setText(QtGui.QApplication.translate("Dialog", INITIAL_MSG, None, QtGui.QApplication.UnicodeUTF8)) - self.label_3.setObjectName(_fromUtf8("label_3")) - self.verticalLayout_4.addWidget(self.label_3) - self.line_2 = QtGui.QFrame(Dialog) - self.line_2.setFrameShape(QtGui.QFrame.HLine) - self.line_2.setFrameShadow(QtGui.QFrame.Sunken) - self.line_2.setObjectName(_fromUtf8("line_2")) - self.verticalLayout_4.addWidget(self.line_2) - self.label_4 = QtGui.QLabel(Dialog) - self.label_4.setText(_fromUtf8("")) - self.label_4.setPixmap(QtGui.QPixmap(_fromUtf8( os.path.join(FIGURES_PATH,"w.jpg") ))) - self.label_4.setScaledContents(True) - self.label_4.setObjectName(_fromUtf8("label_4")) - self.verticalLayout_4.addWidget(self.label_4) - self.verticalLayout_3.addLayout(self.verticalLayout_4) - self.horizontalLayout_3 = QtGui.QHBoxLayout() - self.horizontalLayout_3.setObjectName(_fromUtf8("horizontalLayout_3")) - self.horizontalLayout_4 = QtGui.QHBoxLayout() - self.horizontalLayout_4.setObjectName(_fromUtf8("horizontalLayout_4")) - spacerItem = QtGui.QSpacerItem(40, 20, QtGui.QSizePolicy.Expanding, QtGui.QSizePolicy.Minimum) - self.horizontalLayout_4.addItem(spacerItem) - self.ExitBtn = QtGui.QPushButton(Dialog) - self.ExitBtn.setText(QtGui.QApplication.translate("Dialog", "Exit", None, QtGui.QApplication.UnicodeUTF8)) - self.ExitBtn.setObjectName(_fromUtf8("ExitBtn")) - self.horizontalLayout_4.addWidget(self.ExitBtn) - self.ContinueBtn = QtGui.QPushButton(Dialog) - self.ContinueBtn.setText(QtGui.QApplication.translate("Dialog", "Continue", None, QtGui.QApplication.UnicodeUTF8)) - self.ContinueBtn.setObjectName(_fromUtf8("ContinueBtn")) - self.horizontalLayout_4.addWidget(self.ContinueBtn) - spacerItem1 = QtGui.QSpacerItem(40, 20, QtGui.QSizePolicy.Expanding, QtGui.QSizePolicy.Minimum) - self.horizontalLayout_4.addItem(spacerItem1) - self.horizontalLayout_3.addLayout(self.horizontalLayout_4) - self.verticalLayout_3.addLayout(self.horizontalLayout_3) - self.gridLayout.addLayout(self.verticalLayout_3, 0, 0, 1, 1) - - self.retranslateUi(Dialog) - QtCore.QMetaObject.connectSlotsByName(Dialog) - - def retranslateUi(self, Dialog): - pass - - -if __name__ == "__main__": - import sys - app = QtGui.QApplication(sys.argv) - Dialog = QtGui.QDialog() - ui = Ui_InitWindow() - ui.setupUi(Dialog) - Dialog.show() - sys.exit(app.exec_()) - - - - diff --git a/schainpy/gui/viewer/ui_mainwindow.py b/schainpy/gui/viewer/ui_mainwindow.py deleted file mode 100644 index fb097e4..0000000 --- a/schainpy/gui/viewer/ui_mainwindow.py +++ /dev/null @@ -1,352 +0,0 @@ -# -*- coding: utf-8 -*- - -# Form implementation generated from reading ui file '/home/alex/ui/MainWindow_21_02_13_v49.ui' -# -# Created: Mon Mar 24 13:28:36 2014 -# by: PyQt4 UI code generator 4.10 -# -# WARNING! All changes made in this file will be lost! - -from PyQt4 import QtCore, QtGui -from windows import * - -try: - _fromUtf8 = QtCore.QString.fromUtf8 -except AttributeError: - def _fromUtf8(s): - return s - -try: - _encoding = QtGui.QApplication.UnicodeUTF8 - def _translate(context, text, disambig): - return QtGui.QApplication.translate(context, text, disambig, _encoding) -except AttributeError: - def _translate(context, text, disambig): - return QtGui.QApplication.translate(context, text, disambig) - -import os -from schainpy.gui.figures import tools -from schainpy import __version__ - -FIGURES_PATH = tools.get_path() - -ICON_START = "start_green.png" -ICON_PAUSE = "pause_yellow.png" -ICON_STOP = "stop_red.png" - -ICON_PAUSE_ENA = "restart_yellow.png" - -ICON_START_DIS = "start.png" -ICON_PAUSE_DIS = "pause.png" -ICON_STOP_DIS = "stop.png" - -class Ui_EnvWindow(object): - - def changeStartIcon(self, started=False): - - if started == False: - icon_name = ICON_START - else: - icon_name = ICON_START_DIS - - iconStart = QtGui.QIcon() - iconStart.addPixmap(QtGui.QPixmap(_fromUtf8( os.path.join(FIGURES_PATH, icon_name) )), QtGui.QIcon.Normal, QtGui.QIcon.Off) - self.actionStarToolbar.setIcon(iconStart) - - return - - def changePauseIcon(self, paused=False): - - if paused == True: - icon_name = ICON_PAUSE_ENA - else: - icon_name = ICON_PAUSE - - iconPause = QtGui.QIcon() - iconPause.addPixmap(QtGui.QPixmap(_fromUtf8( os.path.join(FIGURES_PATH, icon_name) )), QtGui.QIcon.Normal, QtGui.QIcon.Off) - self.actionPauseToolbar.setIcon(iconPause) - - return - - def changeStopIcon(self, started=False): - - if started == True: - icon_name = ICON_STOP - else: - icon_name = ICON_STOP_DIS - - iconStop = QtGui.QIcon() - iconStop.addPixmap(QtGui.QPixmap(_fromUtf8( os.path.join(FIGURES_PATH, icon_name) )), QtGui.QIcon.Normal, QtGui.QIcon.Off) - self.actionStopToolbar.setIcon(iconStop) - - return - - def setupUi(self, MainWindow): - - self.paused=False - - MainWindow.setObjectName(_fromUtf8("MainWindow")) - MainWindow.resize(1200, 800) - - self.centralWidget = QtGui.QWidget(MainWindow) - self.centralWidget.setObjectName(_fromUtf8("centralWidget")) - self.gridLayout_16 = QtGui.QGridLayout(self.centralWidget) - self.gridLayout_16.setObjectName(_fromUtf8("gridLayout_16")) - self.splitter_2 = QtGui.QSplitter(self.centralWidget) - self.splitter_2.setOrientation(QtCore.Qt.Horizontal) - self.splitter_2.setObjectName(_fromUtf8("splitter_2")) - self.projectExplorerTree = QtGui.QTreeView(self.splitter_2) - self.projectExplorerTree.setObjectName(_fromUtf8("projectExplorerTree")) - self.splitter = QtGui.QSplitter(self.splitter_2) - self.splitter.setOrientation(QtCore.Qt.Vertical) - self.splitter.setObjectName(_fromUtf8("splitter")) - self.tabWidgetProject = QtGui.QTabWidget(self.splitter) - self.tabWidgetProject.setMinimumSize(QtCore.QSize(0, 278)) - self.tabWidgetProject.setMaximumSize(QtCore.QSize(16777215, 16777215)) - self.tabWidgetProject.setObjectName(_fromUtf8("tabWidgetProject")) - - self.tabConsole = QtGui.QTabWidget(self.splitter) - self.tabConsole.setMinimumSize(QtCore.QSize(0, 0)) - self.tabConsole.setObjectName(_fromUtf8("tabConsole")) - self.tab_5 = QtGui.QWidget() - self.tab_5.setObjectName(_fromUtf8("tab_5")) - self.gridLayout_4 = QtGui.QGridLayout(self.tab_5) - self.gridLayout_4.setObjectName(_fromUtf8("gridLayout_4")) - self.console = QtGui.QTextEdit(self.tab_5) - self.console.setObjectName(_fromUtf8("console")) - self.gridLayout_4.addWidget(self.console, 0, 0, 1, 1) - self.tabConsole.addTab(self.tab_5, _fromUtf8("")) - self.tabWidget = QtGui.QTabWidget(self.splitter_2) - self.tabWidget.setObjectName(_fromUtf8("tabWidget")) - self.tabProjectProperty = QtGui.QWidget() - self.tabProjectProperty.setObjectName(_fromUtf8("tabProjectProperty")) - self.gridLayout_8 = QtGui.QGridLayout(self.tabProjectProperty) - self.gridLayout_8.setObjectName(_fromUtf8("gridLayout_8")) - self.treeProjectProperties = QtGui.QTreeView(self.tabProjectProperty) - self.treeProjectProperties.setObjectName(_fromUtf8("treeProjectProperties")) - self.gridLayout_8.addWidget(self.treeProjectProperties, 0, 0, 1, 1) - self.tabWidget.addTab(self.tabProjectProperty, _fromUtf8("")) - self.gridLayout_16.addWidget(self.splitter_2, 1, 0, 1, 1) - - MainWindow.setCentralWidget(self.centralWidget) - self.toolBar = QtGui.QToolBar(MainWindow) - self.toolBar.setObjectName(_fromUtf8("toolBar")) - MainWindow.addToolBar(QtCore.Qt.TopToolBarArea, self.toolBar) - - self.menuBar = QtGui.QMenuBar(MainWindow) - self.menuBar.setGeometry(QtCore.QRect(0, 0, 1065, 25)) - self.menuBar.setObjectName(_fromUtf8("menuBar")) - self.menuProject = QtGui.QMenu(self.menuBar) - self.menuProject.setObjectName(_fromUtf8("menuProject")) - self.menuRun = QtGui.QMenu(self.menuBar) - self.menuRun.setObjectName(_fromUtf8("menuRun")) - self.menuOptions = QtGui.QMenu(self.menuBar) - self.menuOptions.setObjectName(_fromUtf8("menuOptions")) - self.menuHelp = QtGui.QMenu(self.menuBar) - self.menuHelp.setObjectName(_fromUtf8("menuHelp")) - MainWindow.setMenuBar(self.menuBar) - - iconOpen = QtGui.QIcon() - iconOpen.addPixmap(QtGui.QPixmap(_fromUtf8( os.path.join(FIGURES_PATH,"open.png") )), QtGui.QIcon.Normal, QtGui.QIcon.Off) - iconCreate = QtGui.QIcon() - iconCreate.addPixmap(QtGui.QPixmap(_fromUtf8( os.path.join(FIGURES_PATH,"new.png") )), QtGui.QIcon.Normal, QtGui.QIcon.Off) - iconSave = QtGui.QIcon() - iconSave.addPixmap(QtGui.QPixmap(_fromUtf8( os.path.join(FIGURES_PATH,"save.png") )), QtGui.QIcon.Normal, QtGui.QIcon.Off) - iconStart = QtGui.QIcon() - iconStart.addPixmap(QtGui.QPixmap(_fromUtf8( os.path.join(FIGURES_PATH, ICON_START_DIS) )), QtGui.QIcon.Normal, QtGui.QIcon.Off) - iconStop = QtGui.QIcon() - iconStop.addPixmap(QtGui.QPixmap(_fromUtf8( os.path.join(FIGURES_PATH, ICON_STOP_DIS) )), QtGui.QIcon.Normal, QtGui.QIcon.Off) - iconPause = QtGui.QIcon() - iconPause.addPixmap(QtGui.QPixmap(_fromUtf8( os.path.join(FIGURES_PATH, ICON_PAUSE_DIS) )), QtGui.QIcon.Normal, QtGui.QIcon.Off) - iconAddPU = QtGui.QIcon() - iconAddPU.addPixmap(QtGui.QPixmap(_fromUtf8( os.path.join(FIGURES_PATH,"branch.png") )), QtGui.QIcon.Normal, QtGui.QIcon.Off) - iconClose = QtGui.QIcon() - iconClose.addPixmap(QtGui.QPixmap(_fromUtf8( os.path.join(FIGURES_PATH,"close.png") )), QtGui.QIcon.Normal, QtGui.QIcon.Off) - - - self.actionOpen = QtGui.QAction(MainWindow) - self.actionOpen.setIcon(iconOpen) - self.actionOpen.setObjectName(_fromUtf8("actionOpen")) - self.actionCreate = QtGui.QAction(MainWindow) - self.actionCreate.setIcon(iconCreate) - self.actionCreate.setObjectName(_fromUtf8("actionCreate")) - self.actionSave = QtGui.QAction(MainWindow) - self.actionSave.setIcon(iconSave) - self.actionSave.setObjectName(_fromUtf8("actionSave")) - self.actionClose = QtGui.QAction(MainWindow) - self.actionClose.setIcon(iconClose) - self.actionClose.setObjectName(_fromUtf8("actionClose")) - self.actionStart = QtGui.QAction(MainWindow) - self.actionStart.setIcon(iconStart) - self.actionStart.setObjectName(_fromUtf8("actionStart")) - self.actionPause = QtGui.QAction(MainWindow) - self.actionPause.setIcon(iconPause) - self.actionPause.setObjectName(_fromUtf8("actionPause")) - self.actionStop = QtGui.QAction(MainWindow) - self.actionStop.setIcon(iconStop) - self.actionStop.setObjectName(_fromUtf8("actionStop")) - self.actionAbout = QtGui.QAction(MainWindow) - self.actionAbout.setObjectName(_fromUtf8("actionAbout")) - - self.actionOpenToolbar = QtGui.QAction(MainWindow) - self.actionOpenToolbar.setIcon(iconOpen) - self.actionOpenToolbar.setObjectName(_fromUtf8("actionOpenToolbar")) - self.actionCreateToolbar = QtGui.QAction(MainWindow) - self.actionCreateToolbar.setIcon(iconCreate) - self.actionCreateToolbar.setObjectName(_fromUtf8("actionCreateToolbar")) - self.actionSaveToolbar = QtGui.QAction(MainWindow) - self.actionSaveToolbar.setIcon(iconSave) - self.actionSaveToolbar.setObjectName(_fromUtf8("actionSaveToolbar")) - self.actionStarToolbar = QtGui.QAction(MainWindow) - self.actionStarToolbar.setIcon(iconStart) - self.actionStarToolbar.setObjectName(_fromUtf8("actionStarToolbar")) - self.actionStopToolbar = QtGui.QAction(MainWindow) - self.actionStopToolbar.setIcon(iconStop) - self.actionStopToolbar.setObjectName(_fromUtf8("actionStopToolbar")) - self.actionPauseToolbar = QtGui.QAction(MainWindow) - self.actionPauseToolbar.setIcon(iconPause) - self.actionPauseToolbar.setObjectName(_fromUtf8("actionPauseToolbar")) - self.actionAddPU = QtGui.QAction(MainWindow) - self.actionAddPU.setIcon(iconAddPU) - self.actionAddPU.setObjectName(_fromUtf8("actionAddPU")) - self.actionFTP = QtGui.QAction(MainWindow) - self.actionFTP.setObjectName(_fromUtf8("actionFTP")) - self.toolBar.addAction(self.actionOpenToolbar) - self.toolBar.addSeparator() - self.toolBar.addAction(self.actionCreateToolbar) - self.toolBar.addSeparator() - self.toolBar.addAction(self.actionAddPU) - self.toolBar.addSeparator() - self.toolBar.addAction(self.actionSaveToolbar) - self.toolBar.addSeparator() - self.toolBar.addAction(self.actionStarToolbar) - self.toolBar.addSeparator() - self.toolBar.addAction(self.actionPauseToolbar) - self.toolBar.addSeparator() - self.toolBar.addAction(self.actionStopToolbar) - self.toolBar.addSeparator() - -# self.actionPause.triggered.connect(self.changePauseIcon) -# self.actionPauseToolbar.triggered.connect(self.changePauseIcon) - - self.menuProject.addAction(self.actionOpen) - self.menuProject.addAction(self.actionCreate) - self.menuProject.addAction(self.actionSave) - self.menuProject.addAction(self.actionClose) - self.menuRun.addAction(self.actionStart) - self.menuRun.addAction(self.actionPause) - self.menuRun.addAction(self.actionStop) - self.menuOptions.addAction(self.actionFTP) - self.menuHelp.addAction(self.actionAbout) - self.menuBar.addAction(self.menuProject.menuAction()) - self.menuBar.addAction(self.menuRun.menuAction()) - self.menuBar.addAction(self.menuOptions.menuAction()) - self.menuBar.addAction(self.menuHelp.menuAction()) - - self.tabConsole.setCurrentIndex(0) - self.tabWidget.setCurrentIndex(0) - - def retranslateUi(self, MainWindow): - - MainWindow.setWindowTitle(_translate("MainWindow", "MainWindow", None)) - - self.tabConsole.setTabText(self.tabConsole.indexOf(self.tab_5), _translate("MainWindow", "Console", None)) - - self.tabWidget.setTabText(self.tabWidget.indexOf(self.tabProjectProperty), _translate("MainWindow", "Project Property", None)) - self.toolBar.setWindowTitle(_translate("MainWindow", "toolBar", None)) - self.menuProject.setTitle(_translate("MainWindow", "Project", None)) - self.menuRun.setTitle(_translate("MainWindow", "Run", None)) - self.menuOptions.setTitle(_translate("MainWindow", "Options", None)) - self.menuHelp.setTitle(_translate("MainWindow", "Help", None)) - self.actionOpen.setText(_translate("MainWindow", "Open", None)) - self.actionCreate.setText(_translate("MainWindow", "Create", None)) - self.actionSave.setText(_translate("MainWindow", "Save", None)) - self.actionClose.setText(_translate("MainWindow", "Close", None)) - self.actionStart.setText(_translate("MainWindow", "Start", None)) - self.actionPause.setText(_translate("MainWindow", "Pause", None)) - self.actionStop.setText(_translate("MainWindow", "Stop", None)) - self.actionAbout.setText(_translate("MainWindow", "About SChain", None)) - self.actionOpenToolbar.setText(_translate("MainWindow", "openToolbar", None)) - self.actionOpenToolbar.setToolTip(_translate("MainWindow", "Open a project", None)) - self.actionCreateToolbar.setText(_translate("MainWindow", "createToolbar", None)) - self.actionCreateToolbar.setToolTip(_translate("MainWindow", "Create a new project", None)) - self.actionSaveToolbar.setText(_translate("MainWindow", "saveToolbar", None)) - self.actionSaveToolbar.setToolTip(_translate("MainWindow", "Save a project", None)) - self.actionStarToolbar.setText(_translate("MainWindow", "starToolbar", None)) - self.actionStarToolbar.setToolTip(_translate("MainWindow", "Start process", None)) - self.actionStopToolbar.setText(_translate("MainWindow", "stopToolbar", None)) - self.actionStopToolbar.setToolTip(_translate("MainWindow", "Stop process", None)) - self.actionPauseToolbar.setText(_translate("MainWindow", "pauseToolbar", None)) - self.actionPauseToolbar.setToolTip(_translate("MainWindow", "Pause process", None)) - self.actionAddPU.setText(_translate("MainWindow", "Add Processing Unit", None)) - self.actionFTP.setText(_translate("MainWindow", "FTP", None)) - - def closeEvent(self, event): - - reply = QtGui.QMessageBox.question(self, 'Message', - "Are you sure to quit?", QtGui.QMessageBox.Yes | - QtGui.QMessageBox.No, QtGui.QMessageBox.No) - if reply == QtGui.QMessageBox.Yes: - event.accept() - else: - event.ignore() - - def aboutEvent(self): - title = "Signal Chain Processing Software v%s" %__version__ - message = """ - Developed by: - Miguel Urco - miguel.urco@jro.igp.gob.pe -Jicamarca Radio Observatory -""" - QtGui.QMessageBox.about(self, title, message) - - -class Ui_BasicWindow(Ui_EnvWindow, Ui_ProjectTab, Ui_VoltageTab, Ui_SpectraTab, Ui_SpectraHeisTab, Ui_CorrelationTab): - - def setupUi(self, MainWindow): - - Ui_EnvWindow.setupUi(self, MainWindow) - - Ui_ProjectTab.setupUi(self) - Ui_VoltageTab.setupUi(self) - Ui_SpectraTab.setupUi(self) - Ui_SpectraHeisTab.setupUi(self) - Ui_CorrelationTab.setupUi(self) - - self.retranslateUi(MainWindow) - - QtCore.QMetaObject.connectSlotsByName(MainWindow) - - def retranslateUi(self, MainWindow): - - Ui_EnvWindow.retranslateUi(self, MainWindow) - - Ui_ProjectTab.retranslateUi(self) - Ui_VoltageTab.retranslateUi(self) - Ui_SpectraTab.retranslateUi(self) - Ui_SpectraHeisTab.retranslateUi(self) - Ui_CorrelationTab.retranslateUi(self) - - -class Ui_AdvancedWindow(Ui_EnvWindow): - - def setupUi(self, AdvancedWindow): - - Ui_MainWindow.setupUi(self, AdvancedWindow) - - def retranslateUi(self, AdvancedWindow): - - Ui_MainWindow.retranslateUi(self, AdvancedWindow) - - - -if __name__ == "__main__": - import sys - app = QtGui.QApplication(sys.argv) - MainWindow = QtGui.QMainWindow() - ui = Ui_BasicWindow() - ui.setupUi(MainWindow) - MainWindow.show() - sys.exit(app.exec_()) - diff --git a/schainpy/gui/viewer/ui_unitprocess.py b/schainpy/gui/viewer/ui_unitprocess.py deleted file mode 100644 index 63b61c8..0000000 --- a/schainpy/gui/viewer/ui_unitprocess.py +++ /dev/null @@ -1,123 +0,0 @@ -# -*- coding: utf-8 -*- - -# Form implementation generated from reading ui file '/home/alex/ui/unitProcess4.ui' -# -# Created: Fri May 24 05:23:03 2013 -# by: PyQt4 UI code generator 4.10 -# -# WARNING! All changes made in this file will be lost! - -from PyQt4 import QtCore, QtGui - -try: - _fromUtf8 = QtCore.QString.fromUtf8 -except AttributeError: - def _fromUtf8(s): - return s - -try: - _encoding = QtGui.QApplication.UnicodeUTF8 - def _translate(context, text, disambig): - return QtGui.QApplication.translate(context, text, disambig, _encoding) -except AttributeError: - def _translate(context, text, disambig): - return QtGui.QApplication.translate(context, text, disambig) - -class Ui_UnitProcess(object): - def setupUi(self, MainWindow): - MainWindow.setObjectName(_fromUtf8("MainWindow")) - MainWindow.resize(312, 195) - self.centralWidget = QtGui.QWidget(MainWindow) - self.centralWidget.setObjectName(_fromUtf8("centralWidget")) - self.gridLayout = QtGui.QGridLayout(self.centralWidget) - self.gridLayout.setObjectName(_fromUtf8("gridLayout")) - self.inputLabel = QtGui.QLabel(self.centralWidget) - font = QtGui.QFont() - font.setPointSize(12) - font.setBold(False) - font.setWeight(50) - self.inputLabel.setFont(font) - self.inputLabel.setObjectName(_fromUtf8("inputLabel")) - self.gridLayout.addWidget(self.inputLabel, 2, 0, 1, 1) - self.unitPcancelbut = QtGui.QPushButton(self.centralWidget) - font = QtGui.QFont() - font.setBold(False) - font.setWeight(50) - self.unitPcancelbut.setFont(font) - self.unitPcancelbut.setObjectName(_fromUtf8("unitPcancelbut")) - self.gridLayout.addWidget(self.unitPcancelbut, 5, 2, 1, 2) - self.unitPokbut = QtGui.QPushButton(self.centralWidget) - font = QtGui.QFont() - font.setBold(False) - font.setWeight(50) - self.unitPokbut.setFont(font) - self.unitPokbut.setObjectName(_fromUtf8("unitPokbut")) - self.gridLayout.addWidget(self.unitPokbut, 5, 0, 1, 2) - self.typeLabel = QtGui.QLabel(self.centralWidget) - font = QtGui.QFont() - font.setPointSize(12) - font.setBold(False) - font.setWeight(50) - self.typeLabel.setFont(font) - self.typeLabel.setObjectName(_fromUtf8("typeLabel")) - self.gridLayout.addWidget(self.typeLabel, 3, 0, 1, 1) - self.nameUP = QtGui.QLabel(self.centralWidget) - font = QtGui.QFont() - font.setPointSize(13) - font.setBold(False) - font.setWeight(50) - self.nameUP.setFont(font) - self.nameUP.setObjectName(_fromUtf8("nameUP")) - self.gridLayout.addWidget(self.nameUP, 0, 1, 1, 2) - spacerItem = QtGui.QSpacerItem(40, 20, QtGui.QSizePolicy.Expanding, QtGui.QSizePolicy.Minimum) - self.gridLayout.addItem(spacerItem, 0, 3, 1, 1) - self.comboInputBox = QtGui.QComboBox(self.centralWidget) - font = QtGui.QFont() - font.setPointSize(12) - self.comboInputBox.setFont(font) - self.comboInputBox.setObjectName(_fromUtf8("comboInputBox")) - self.gridLayout.addWidget(self.comboInputBox, 2, 1, 1, 3) - self.comboTypeBox = QtGui.QComboBox(self.centralWidget) - font = QtGui.QFont() - font.setPointSize(12) - self.comboTypeBox.setFont(font) - self.comboTypeBox.setObjectName(_fromUtf8("comboTypeBox")) - self.comboTypeBox.addItem(_fromUtf8("")) - self.comboTypeBox.addItem(_fromUtf8("")) - self.comboTypeBox.addItem(_fromUtf8("")) - self.gridLayout.addWidget(self.comboTypeBox, 3, 1, 1, 3) - spacerItem1 = QtGui.QSpacerItem(40, 20, QtGui.QSizePolicy.Expanding, QtGui.QSizePolicy.Minimum) - self.gridLayout.addItem(spacerItem1, 0, 0, 1, 1) - spacerItem2 = QtGui.QSpacerItem(20, 40, QtGui.QSizePolicy.Minimum, QtGui.QSizePolicy.Expanding) - self.gridLayout.addItem(spacerItem2, 4, 0, 1, 1) - self.line = QtGui.QFrame(self.centralWidget) - self.line.setFrameShape(QtGui.QFrame.HLine) - self.line.setFrameShadow(QtGui.QFrame.Sunken) - self.line.setObjectName(_fromUtf8("line")) - self.gridLayout.addWidget(self.line, 1, 0, 1, 4) - MainWindow.setCentralWidget(self.centralWidget) - - self.retranslateUi(MainWindow) - QtCore.QMetaObject.connectSlotsByName(MainWindow) - - def retranslateUi(self, MainWindow): - MainWindow.setWindowTitle(_translate("MainWindow", "MainWindow", None)) - self.inputLabel.setText(_translate("MainWindow", "Input:", None)) - self.unitPcancelbut.setText(_translate("MainWindow", "Cancel", None)) - self.unitPokbut.setText(_translate("MainWindow", "Ok", None)) - self.typeLabel.setText(_translate("MainWindow", "Type:", None)) - self.nameUP.setText(_translate("MainWindow", "Processing Unit", None)) - self.comboTypeBox.setItemText(0, _translate("MainWindow", "Voltage", None)) - self.comboTypeBox.setItemText(1, _translate("MainWindow", "Spectra", None)) - self.comboTypeBox.setItemText(2, _translate("MainWindow", "Correlation", None)) - - -if __name__ == "__main__": - import sys - app = QtGui.QApplication(sys.argv) - MainWindow = QtGui.QMainWindow() - ui = Ui_UnitProcess() - ui.setupUi(MainWindow) - MainWindow.show() - sys.exit(app.exec_()) - diff --git a/schainpy/gui/viewer/ui_window.py b/schainpy/gui/viewer/ui_window.py deleted file mode 100644 index efe81e3..0000000 --- a/schainpy/gui/viewer/ui_window.py +++ /dev/null @@ -1,72 +0,0 @@ -# -*- coding: utf-8 -*- - -# Form implementation generated from reading ui file 'C:\Users\alex\ericworkspace\UIDOS\window.ui' -# -# Created: Thu Dec 06 08:56:59 2012 -# by: PyQt4 UI code generator 4.9.4 -# -# WARNING! All changes made in this file will be lost! - -from PyQt4 import QtCore, QtGui - -try: - _fromUtf8 = QtCore.QString.fromUtf8 -except AttributeError: - _fromUtf8 = lambda s: s - -class Ui_window(object): - def setupUi(self, MainWindow): - MainWindow.setObjectName(_fromUtf8("MainWindow")) - MainWindow.resize(220, 198) - self.centralWidget = QtGui.QWidget(MainWindow) - self.centralWidget.setObjectName(_fromUtf8("centralWidget")) - self.label = QtGui.QLabel(self.centralWidget) - self.label.setGeometry(QtCore.QRect(20, 10, 131, 20)) - font = QtGui.QFont() - font.setPointSize(12) - self.label.setFont(font) - self.label.setObjectName(_fromUtf8("label")) - self.label_2 = QtGui.QLabel(self.centralWidget) - self.label_2.setGeometry(QtCore.QRect(20, 60, 131, 20)) - font = QtGui.QFont() - font.setPointSize(12) - self.label_2.setFont(font) - self.label_2.setObjectName(_fromUtf8("label_2")) - self.cancelButton = QtGui.QPushButton(self.centralWidget) - self.cancelButton.setGeometry(QtCore.QRect(150, 160, 51, 23)) - self.cancelButton.setObjectName(_fromUtf8("cancelButton")) - self.okButton = QtGui.QPushButton(self.centralWidget) - self.okButton.setGeometry(QtCore.QRect(80, 160, 61, 23)) - self.okButton.setObjectName(_fromUtf8("okButton")) - self.proyectNameLine = QtGui.QLineEdit(self.centralWidget) - self.proyectNameLine.setGeometry(QtCore.QRect(20, 30, 181, 20)) - self.proyectNameLine.setObjectName(_fromUtf8("proyectNameLine")) - self.descriptionTextEdit = QtGui.QTextEdit(self.centralWidget) - self.descriptionTextEdit.setGeometry(QtCore.QRect(20, 80, 181, 71)) - self.descriptionTextEdit.setObjectName(_fromUtf8("descriptionTextEdit")) - self.saveButton = QtGui.QPushButton(self.centralWidget) - self.saveButton.setGeometry(QtCore.QRect(20, 160, 51, 23)) - self.saveButton.setObjectName(_fromUtf8("saveButton")) - MainWindow.setCentralWidget(self.centralWidget) - - self.retranslateUi(MainWindow) - QtCore.QMetaObject.connectSlotsByName(MainWindow) - - def retranslateUi(self, MainWindow): - MainWindow.setWindowTitle(QtGui.QApplication.translate("MainWindow", "MainWindow", None, QtGui.QApplication.UnicodeUTF8)) - self.label.setText(QtGui.QApplication.translate("MainWindow", "Project Name:", None, QtGui.QApplication.UnicodeUTF8)) - self.label_2.setText(QtGui.QApplication.translate("MainWindow", "Description :", None, QtGui.QApplication.UnicodeUTF8)) - self.cancelButton.setText(QtGui.QApplication.translate("MainWindow", "Cancel", None, QtGui.QApplication.UnicodeUTF8)) - self.okButton.setText(QtGui.QApplication.translate("MainWindow", "Ok", None, QtGui.QApplication.UnicodeUTF8)) - self.saveButton.setText(QtGui.QApplication.translate("MainWindow", "Save", None, QtGui.QApplication.UnicodeUTF8)) - - -if __name__ == "__main__": - import sys - app = QtGui.QApplication(sys.argv) - MainWindow = QtGui.QMainWindow() - ui = Ui_window() - ui.setupUi(MainWindow) - MainWindow.show() - sys.exit(app.exec_()) - diff --git a/schainpy/gui/viewer/ui_workspace.py b/schainpy/gui/viewer/ui_workspace.py deleted file mode 100644 index 3775c09..0000000 --- a/schainpy/gui/viewer/ui_workspace.py +++ /dev/null @@ -1,115 +0,0 @@ -# -*- coding: utf-8 -*- - -# Form implementation generated from reading ui file '/home/alex/ui/workspacev5.ui' -# -# Created: Sun May 12 16:45:47 2013 -# by: PyQt4 UI code generator 4.10 -# -# WARNING! All changes made in this file will be lost! - -from PyQt4 import QtCore, QtGui - -try: - _fromUtf8 = QtCore.QString.fromUtf8 -except AttributeError: - def _fromUtf8(s): - return s - -try: - _encoding = QtGui.QApplication.UnicodeUTF8 - def _translate(context, text, disambig): - return QtGui.QApplication.translate(context, text, disambig, _encoding) -except AttributeError: - def _translate(context, text, disambig): - return QtGui.QApplication.translate(context, text, disambig) - -class Ui_Workspace(object): - def setupUi(self, Dialog): - Dialog.setObjectName(_fromUtf8("Dialog")) - Dialog.setEnabled(True) - Dialog.resize(730, 295) - Dialog.setMinimumSize(QtCore.QSize(730, 295)) - Dialog.setMaximumSize(QtCore.QSize(730, 295)) - self.dirLabel = QtGui.QTextEdit(Dialog) - self.dirLabel.setGeometry(QtCore.QRect(0, 0, 731, 71)) - self.dirLabel.setReadOnly(True) - self.dirLabel.setObjectName(_fromUtf8("dirLabel")) - self.dirWork = QtGui.QLabel(Dialog) - self.dirWork.setGeometry(QtCore.QRect(10, 90, 91, 16)) - font = QtGui.QFont() - font.setPointSize(11) - self.dirWork.setFont(font) - self.dirWork.setObjectName(_fromUtf8("dirWork")) - self.dirComBox = QtGui.QComboBox(Dialog) - self.dirComBox.setGeometry(QtCore.QRect(110, 80, 501, 31)) - palette = QtGui.QPalette() - brush = QtGui.QBrush(QtGui.QColor(255, 255, 255)) - brush.setStyle(QtCore.Qt.SolidPattern) - palette.setBrush(QtGui.QPalette.Active, QtGui.QPalette.Button, brush) - brush = QtGui.QBrush(QtGui.QColor(255, 255, 255)) - brush.setStyle(QtCore.Qt.SolidPattern) - palette.setBrush(QtGui.QPalette.Inactive, QtGui.QPalette.Button, brush) - brush = QtGui.QBrush(QtGui.QColor(255, 255, 255)) - brush.setStyle(QtCore.Qt.SolidPattern) - palette.setBrush(QtGui.QPalette.Disabled, QtGui.QPalette.Button, brush) - self.dirComBox.setPalette(palette) - self.dirComBox.setObjectName(_fromUtf8("dirComBox")) - self.dirToolPath = QtGui.QToolButton(Dialog) - self.dirToolPath.setGeometry(QtCore.QRect(620, 80, 101, 31)) - font = QtGui.QFont() - font.setPointSize(11) - self.dirToolPath.setFont(font) - self.dirToolPath.setObjectName(_fromUtf8("dirToolPath")) - self.widget = QtGui.QWidget(Dialog) - self.widget.setGeometry(QtCore.QRect(10, 120, 711, 121)) - self.widget.setObjectName(_fromUtf8("widget")) - self.horizontalLayout_3 = QtGui.QHBoxLayout(self.widget) - self.horizontalLayout_3.setMargin(0) - self.horizontalLayout_3.setObjectName(_fromUtf8("horizontalLayout_3")) - spacerItem = QtGui.QSpacerItem(20, 40, QtGui.QSizePolicy.Minimum, QtGui.QSizePolicy.Expanding) - self.horizontalLayout_3.addItem(spacerItem) - spacerItem1 = QtGui.QSpacerItem(20, 40, QtGui.QSizePolicy.Minimum, QtGui.QSizePolicy.Expanding) - self.horizontalLayout_3.addItem(spacerItem1) - spacerItem2 = QtGui.QSpacerItem(20, 40, QtGui.QSizePolicy.Minimum, QtGui.QSizePolicy.Expanding) - self.horizontalLayout_3.addItem(spacerItem2) - spacerItem3 = QtGui.QSpacerItem(20, 40, QtGui.QSizePolicy.Minimum, QtGui.QSizePolicy.Expanding) - self.horizontalLayout_3.addItem(spacerItem3) - spacerItem4 = QtGui.QSpacerItem(20, 40, QtGui.QSizePolicy.Minimum, QtGui.QSizePolicy.Expanding) - self.horizontalLayout_3.addItem(spacerItem4) - self.dirCancelBtn = QtGui.QPushButton(Dialog) - self.dirCancelBtn.setGeometry(QtCore.QRect(490, 250, 111, 31)) - self.dirCancelBtn.setObjectName(_fromUtf8("dirCancelBtn")) - self.dirOkBtn = QtGui.QPushButton(Dialog) - self.dirOkBtn.setGeometry(QtCore.QRect(610, 250, 111, 31)) - font = QtGui.QFont() - font.setPointSize(11) - self.dirOkBtn.setFont(font) - self.dirOkBtn.setObjectName(_fromUtf8("dirOkBtn")) - - self.retranslateUi(Dialog) - QtCore.QMetaObject.connectSlotsByName(Dialog) - - def retranslateUi(self, Dialog): - Dialog.setWindowTitle(_translate("Dialog", "Dialog", None)) - self.dirLabel.setHtml(_translate("Dialog", "\n" -"\n" -"

Select a workspace

\n" -"

Signal Chain stores your projects in a folder called a workspace.

\n" -"

Choose a workspace folder to use for this session.

", None)) - self.dirWork.setText(_translate("Dialog", "Workspace :", None)) - self.dirToolPath.setText(_translate("Dialog", "Browse...", None)) - self.dirCancelBtn.setText(_translate("Dialog", "Cancel", None)) - self.dirOkBtn.setText(_translate("Dialog", "OK", None)) - - -if __name__ == "__main__": - import sys - app = QtGui.QApplication(sys.argv) - Dialog = QtGui.QDialog() - ui = Ui_Workspace() - ui.setupUi(Dialog) - Dialog.show() - sys.exit(app.exec_()) - diff --git a/schainpy/gui/viewer/viewer_ui/MainWindow_21_02_13_v44.ui b/schainpy/gui/viewer/viewer_ui/MainWindow_21_02_13_v44.ui deleted file mode 100644 index 02e8a90..0000000 --- a/schainpy/gui/viewer/viewer_ui/MainWindow_21_02_13_v44.ui +++ /dev/null @@ -1,1765 +0,0 @@ - - - MainWindow - - - - 0 - 0 - 1065 - 711 - - - - MainWindow - - - - - - - Qt::Horizontal - - - - - Qt::Vertical - - - - - 0 - 278 - - - - - 16777215 - 16777215 - - - - 0 - - - - Project - - - - - - QFrame::StyledPanel - - - QFrame::Raised - - - - - - Project Name : - - - - - - - - - - DataType : - - - - - - - - Voltage - - - - - Spectra - - - - - - - - - - - DataPath : - - - - - - - ... - - - - - - - - - - Read Mode: - - - - - - - - Off Line - - - - - On Line - - - - - - - - Delay: - - - - - - - - - - Walk : - - - - - - - - Search 0 - - - - - Search 1 - - - - - - - - Load - - - - - - - Set: - - - - - - - - - - - - - QFrame::StyledPanel - - - QFrame::Raised - - - - - - Star Date: - - - - - - - - - - End Date: - - - - - - - - - - Start Time: - - - - - - - - - - End Time: - - - - - - - - - - Description: - - - - - - - - - - - - - QFrame::StyledPanel - - - QFrame::Raised - - - - - - Ok - - - - - - - Clear - - - - - - - - - - - Voltage - - - - - - QFrame::StyledPanel - - - QFrame::Raised - - - - - - Ok - - - - - - - Clear - - - - - - - - - - 0 - - - - Operation - - - - - - - - - - Value - - - - - Index - - - - - - - - - Value - - - - - Index - - - - - - - - Profile Selector - - - - - - - - Profile List - - - - - Profile Range List - - - - - - - - Decoder - - - - - - - Coherent Integration - - - - - - - - - - - - - Qt::Horizontal - - - - 40 - 20 - - - - - - - - Qt::Horizontal - - - - 40 - 20 - - - - - - - - Qt::Horizontal - - - - 40 - 20 - - - - - - - - - - - Code: - - - - - - - Select Channels - - - - - - - Select Heights - - - - - - - Filter - - - - - - - - - - Radar Frequency - - - - - - - Mode: - - - - - - - Qt::Horizontal - - - - 40 - 20 - - - - - - - - - - - - Barker 3 - - - - - Barker 4 - - - - - Barker 5 - - - - - Barker 7 - - - - - Barker 11 - - - - - Barker 13 - - - - - Barker 3 + Comp. - - - - - Barker 4 + Comp. - - - - - Barker 5 + Comp. - - - - - Barker 7 + Comp. - - - - - Barker 11+ Comp. - - - - - Barker 13+ Comp. - - - - - None - - - - - - - - - Time - - - - - Freq 1 - - - - - - - - - Graphics - - - - - - Qt::Vertical - - - - 20 - 40 - - - - - - - - - - - - - - ... - - - - - - - - - - Scope - - - - - - - Qt::Vertical - - - - 20 - 40 - - - - - - - - Channel List - - - - - - - Show - - - - - - - Freq/Vel - - - - - - - - - - - - - - Path : - - - - - - - Prefix: - - - - - - - Height range - - - - - - - Qt::Horizontal - - - - 40 - 12 - - - - - - - - Qt::Horizontal - - - - 18 - 12 - - - - - - - - - - - - - - Save - - - - - - - - - - - - - - - Output - - - - - - Type: - - - - - - - Path: - - - - - - - - - - ... - - - - - - - - .rawdata - - - - - - - - Qt::Vertical - - - - 20 - 40 - - - - - - - - - - - Blocks per File : - - - - - - - Profiles per Block: - - - - - - - - - - - - - - - Spectra - - - - - - QFrame::StyledPanel - - - QFrame::Raised - - - - - - Ok - - - - - - - Clear - - - - - - - - - - 0 - - - - Operation - - - - - - Select Cross Spectra - - - - - - - - Value - - - - - Index - - - - - - - - - - - - Value - - - - - Index - - - - - - - - - - - - - - Remove DC - - - - - - - Select Heights - - - - - - - Select Channel - - - - - - - - - - - - - x-y pairs - - - - - - - nFFTPoints - - - - - - - Incoherent Integration - - - - - - - - Time Interval - - - - - Profiles - - - - - - - - Qt::Horizontal - - - - 40 - 20 - - - - - - - - Radar Frequency - - - - - - - Qt::Horizontal - - - - 40 - 20 - - - - - - - - Qt::Horizontal - - - - 40 - 20 - - - - - - - - Qt::Horizontal - - - - 40 - 20 - - - - - - - - - - - Qt::Horizontal - - - - 40 - 20 - - - - - - - - Profiles - - - - - - - - - - - Graphics - - - - - - Coherence Map - - - - - - - Qt::Vertical - - - - 20 - 40 - - - - - - - - Tmin, Tmax: - - - - - - - - - - - - - - - - - - - - - - - - - - - Prefix - - - - - - - - - - Qt::Horizontal - - - - 28 - 15 - - - - - - - - Qt::Vertical - - - - 20 - 40 - - - - - - - - RTI Plot - - - - - - - Height range: - - - - - - - dBs range: - - - - - - - magnitud [0-1] - - - - - - - - - - - - - Qt::Horizontal - - - - 49 - 15 - - - - - - - - Path - - - - - - - - - - - - - Power Profile - - - - - - - Freq/Vel: - - - - - - - - - - Cross Spectra Plot - - - - - - - ... - - - - - - - Channel List: - - - - - - - Spectra Plot - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Qt::Horizontal - - - - 40 - 20 - - - - - - - - - - - - - - - - - - - - - - Qt::Horizontal - - - - 39 - 15 - - - - - - - - - - - Show - - - - - - - - - - - - - - Wr Period: - - - - - - - Save - - - - - - - - - - - - - - - - - - - - - - - - - - - - ftp - - - - - - - - - - - - - - RTI Noise - - - - - - - - - - - - - - - - - - - - - Time Range: - - - - - - - - - - - Output - - - - - - Type: - - - - - - - - .pdata - - - - - - - - Path: - - - - - - - - - - Qt::Vertical - - - - 20 - 40 - - - - - - - - ... - - - - - - - - - - Blocks per File: - - - - - - - Profile per Block: - - - - - - - - - - - - - - - Correlation - - - - - - 0 - - - - Operation - - - - - Graphics - - - - - - - - - - - 0 - 0 - - - - 0 - - - - Console - - - - - - - - - - - - 0 - - - - Project Property - - - - - - - - - - - - - - - toolBar - - - TopToolBarArea - - - false - - - - - - - - - - - - - - - - - - - - 0 - 0 - 1065 - 25 - - - - - Project - - - - - - - - - Run - - - - - - - Options - - - - - - Help - - - - - - - - - - - - ../Documents/figure/open.png../Documents/figure/open.png - - - Open - - - - - - ../Documents/figure/create_class_obj.png../Documents/figure/create_class_obj.png - - - Create - - - - - - ../Documents/figure/save.gif../Documents/figure/save.gif - - - Save - - - - - Close - - - - - - ../Documents/figure/startServer.png../Documents/figure/startServer.png - - - Start - - - - - Pause - - - - - About - - - - - - ../Documents/figure/open.png../Documents/figure/open.png - - - openToolbar - - - Open - - - - - - ../Documents/figure/create_class_obj.png../Documents/figure/create_class_obj.png - - - createToolbar - - - Create - - - - - - ../Documents/figure/save.gif../Documents/figure/save.gif - - - saveToolbar - - - Save - - - - - - ../Documents/figure/startServer.png../Documents/figure/startServer.png - - - starToolbar - - - Start - - - - - - ../Documents/figure/stopServer.png../Documents/figure/stopServer.png - - - stopToolbar - - - Stop - - - - - - ../workspace/branch_gui_may/gui/figure/pause.png../workspace/branch_gui_may/gui/figure/pause.png - - - pauseToolbar - - - Pause - - - - - - ../addpu.jpg../addpu.jpg - - - addPU - - - - - FTP - - - - - - diff --git a/schainpy/gui/viewer/viewer_ui/MainWindow_21_02_13_v46.ui b/schainpy/gui/viewer/viewer_ui/MainWindow_21_02_13_v46.ui deleted file mode 100644 index 789d5d2..0000000 --- a/schainpy/gui/viewer/viewer_ui/MainWindow_21_02_13_v46.ui +++ /dev/null @@ -1,2179 +0,0 @@ - - - MainWindow - - - - 0 - 0 - 1065 - 711 - - - - MainWindow - - - - - - - Qt::Horizontal - - - - - Qt::Vertical - - - - - 0 - 278 - - - - - 16777215 - 16777215 - - - - 0 - - - - Project - - - - - - QFrame::StyledPanel - - - QFrame::Raised - - - - - - Project Name : - - - - - - - - - - DataType : - - - - - - - - Voltage - - - - - Spectra - - - - - Fits - - - - - - - - - - - DataPath : - - - - - - - ... - - - - - - - - - - Read Mode: - - - - - - - - Off Line - - - - - On Line - - - - - - - - Delay: - - - - - - - - - - Walk : - - - - - - - - Search 0 - - - - - Search 1 - - - - - - - - Load - - - - - - - Set: - - - - - - - - - - - - - QFrame::StyledPanel - - - QFrame::Raised - - - - - - Star Date: - - - - - - - - - - End Date: - - - - - - - - - - Start Time: - - - - - - - - - - End Time: - - - - - - - - - - Description: - - - - - - - - - - - - - QFrame::StyledPanel - - - QFrame::Raised - - - - - - Ok - - - - - - - Clear - - - - - - - - - - - Voltage - - - - - - QFrame::StyledPanel - - - QFrame::Raised - - - - - - Ok - - - - - - - Clear - - - - - - - - - - 0 - - - - Operation - - - - - - - - - - Value - - - - - Index - - - - - - - - - Value - - - - - Index - - - - - - - - Profile Selector - - - - - - - - Profile List - - - - - Profile Range List - - - - - - - - Decoder - - - - - - - Coherent Integration - - - - - - - - - - - - - Qt::Horizontal - - - - 40 - 20 - - - - - - - - Qt::Horizontal - - - - 40 - 20 - - - - - - - - Qt::Horizontal - - - - 40 - 20 - - - - - - - - - - - Code: - - - - - - - Select Channels - - - - - - - Select Heights - - - - - - - Filter - - - - - - - - - - Radar Frequency - - - - - - - Mode: - - - - - - - Qt::Horizontal - - - - 40 - 20 - - - - - - - - - - - - Barker 3 - - - - - Barker 4 - - - - - Barker 5 - - - - - Barker 7 - - - - - Barker 11 - - - - - Barker 13 - - - - - Barker 3 + Comp. - - - - - Barker 4 + Comp. - - - - - Barker 5 + Comp. - - - - - Barker 7 + Comp. - - - - - Barker 11+ Comp. - - - - - Barker 13+ Comp. - - - - - None - - - - - - - - - Time - - - - - Freq 1 - - - - - - - - - Graphics - - - - - - Qt::Vertical - - - - 20 - 40 - - - - - - - - - - - - - - ... - - - - - - - - - - Scope - - - - - - - Qt::Vertical - - - - 20 - 40 - - - - - - - - Channel List - - - - - - - Show - - - - - - - Freq/Vel - - - - - - - - - - - - - - Path : - - - - - - - Prefix: - - - - - - - Height range - - - - - - - Qt::Horizontal - - - - 40 - 12 - - - - - - - - Qt::Horizontal - - - - 18 - 12 - - - - - - - - - - - - - - Save - - - - - - - - - - - - - - - Output - - - - - - Type: - - - - - - - Path: - - - - - - - - - - ... - - - - - - - - .rawdata - - - - - - - - Qt::Vertical - - - - 20 - 40 - - - - - - - - - - - Blocks per File : - - - - - - - Profiles per Block: - - - - - - - - - - - - - - - Spectra - - - - - - QFrame::StyledPanel - - - QFrame::Raised - - - - - - Ok - - - - - - - Clear - - - - - - - - - - 0 - - - - Operation - - - - - - Select Cross Spectra - - - - - - - - Value - - - - - Index - - - - - - - - - - - - Value - - - - - Index - - - - - - - - - - - - - - Remove DC - - - - - - - Select Heights - - - - - - - Select Channel - - - - - - - - - - - - - x-y pairs - - - - - - - nFFTPoints - - - - - - - Incoherent Integration - - - - - - - - Time Interval - - - - - Profiles - - - - - - - - Qt::Horizontal - - - - 40 - 20 - - - - - - - - Radar Frequency - - - - - - - Qt::Horizontal - - - - 40 - 20 - - - - - - - - Qt::Horizontal - - - - 40 - 20 - - - - - - - - - - - Qt::Horizontal - - - - 40 - 20 - - - - - - - - Profiles - - - - - - - - - - Remove Interference - - - - - - - Qt::Horizontal - - - - 40 - 20 - - - - - - - - - Mode 1 - - - - - Mode 2 - - - - - - - - - Graphics - - - - - - Coherence Map - - - - - - - Qt::Vertical - - - - 20 - 40 - - - - - - - - Tmin, Tmax: - - - - - - - - - - - - - - - - - - - - - - - - - - - Prefix : - - - - - - - - - - Qt::Horizontal - - - - 28 - 15 - - - - - - - - Qt::Vertical - - - - 20 - 40 - - - - - - - - RTI Plot - - - - - - - Height range: - - - - - - - dBs range: - - - - - - - magnitud [0-1] - - - - - - - - - - - - - Qt::Horizontal - - - - 49 - 15 - - - - - - - - Path : - - - - - - - - - - - - - Power Profile - - - - - - - Freq/Vel: - - - - - - - - - - Cross Spectra Plot - - - - - - - ... - - - - - - - Channel List: - - - - - - - Spectra Plot - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Qt::Horizontal - - - - 40 - 20 - - - - - - - - - - - - - - - - - - - - - - Qt::Horizontal - - - - 39 - 15 - - - - - - - - - - - Show - - - - - - - - - - - - - - Wr Period: - - - - - - - Save - - - - - - - - - - - - - - - - - - - - - - - - - - - - ftp - - - - - - - - - - - - - - RTI Noise - - - - - - - - - - - - - - - - - - - - - Time Range: - - - - - - - - - - - Output - - - - - - Type: - - - - - - - - .pdata - - - - - - - - Path: - - - - - - - - - - Qt::Vertical - - - - 20 - 40 - - - - - - - - ... - - - - - - - - - - Blocks per File: - - - - - - - Profile per Block: - - - - - - - - - - - - - - - SpectraHeis - - - - - - QFrame::StyledPanel - - - QFrame::Raised - - - - - - Clear - - - - - - - Ok - - - - - - - - - - 0 - - - - Operation - - - - - - Qt::Horizontal - - - - 40 - 20 - - - - - - - - - Time Interval - - - - - - - - Incoherent Intergration - - - - - - - - - - Qt::Vertical - - - - 20 - 40 - - - - - - - - - Graphics - - - - - - Prefix - - - - - - - ... - - - - - - - - - - - - - - ymin - ymax - - - - - - - Tmin - Tmax: - - - - - - - Time Range: - - - - - - - Wr Period - - - - - - - Qt::Horizontal - - - - 134 - 20 - - - - - - - - - - - - - - - - - - - - - Channel List: - - - - - - - xmin - xmax - - - - - - - - - - Save - - - - - - - ftp - - - - - - - Spectra Plot - - - - - - - - - - - - - - - - - Path - - - - - - - Show - - - - - - - - - - - - - - Qt::Horizontal - - - - 40 - 20 - - - - - - - - - - - - - - - - - Qt::Horizontal - - - - 106 - 20 - - - - - - - - - - - - - - - - - - - - - - RTI PLot - - - - - - - Qt::Horizontal - - - - 40 - 20 - - - - - - - - - - - - Output - - - - - - Path: - - - - - - - Blocks per File: - - - - - - - Type: - - - - - - - Qt::Vertical - - - - 20 - 40 - - - - - - - - ... - - - - - - - - - - - .fits - - - - - - - - Profile per Block: - - - - - - - - - - - - - - - - - - Correlation - - - - - - 0 - - - - Operation - - - - - Graphics - - - - - - - - - - - 0 - 0 - - - - 0 - - - - Console - - - - - - - - - - - - 0 - - - - Project Property - - - - - - - - - - - - - - - toolBar - - - TopToolBarArea - - - false - - - - - - - - - - - - - - - - - - - - 0 - 0 - 1065 - 25 - - - - - Project - - - - - - - - - Run - - - - - - - Options - - - - - - Help - - - - - - - - - - - - ../Documents/figure/open.png../Documents/figure/open.png - - - Open - - - - - - ../Documents/figure/create_class_obj.png../Documents/figure/create_class_obj.png - - - Create - - - - - - ../Documents/figure/save.gif../Documents/figure/save.gif - - - Save - - - - - Close - - - - - - ../Documents/figure/startServer.png../Documents/figure/startServer.png - - - Start - - - - - Pause - - - - - About - - - - - - ../Documents/figure/open.png../Documents/figure/open.png - - - openToolbar - - - Open - - - - - - ../Documents/figure/create_class_obj.png../Documents/figure/create_class_obj.png - - - createToolbar - - - Create - - - - - - ../Documents/figure/save.gif../Documents/figure/save.gif - - - saveToolbar - - - Save - - - - - - ../Documents/figure/startServer.png../Documents/figure/startServer.png - - - starToolbar - - - Start - - - - - - ../Documents/figure/stopServer.png../Documents/figure/stopServer.png - - - stopToolbar - - - Stop - - - - - - ../workspace/branch_gui_may/gui/figure/pause.png../workspace/branch_gui_may/gui/figure/pause.png - - - pauseToolbar - - - Pause - - - - - - ../addpu.jpg../addpu.jpg - - - addPU - - - - - FTP - - - - - - diff --git a/schainpy/gui/viewer/viewer_ui/MainWindow_21_02_13_v47.ui b/schainpy/gui/viewer/viewer_ui/MainWindow_21_02_13_v47.ui deleted file mode 100644 index 2035e1a..0000000 --- a/schainpy/gui/viewer/viewer_ui/MainWindow_21_02_13_v47.ui +++ /dev/null @@ -1,2186 +0,0 @@ - - - MainWindow - - - - 0 - 0 - 1065 - 711 - - - - MainWindow - - - - - - - Qt::Horizontal - - - - - Qt::Vertical - - - - - 0 - 278 - - - - - 16777215 - 16777215 - - - - 0 - - - - Project - - - - - - QFrame::StyledPanel - - - QFrame::Raised - - - - - - Project Name : - - - - - - - - - - DataType : - - - - - - - - Voltage - - - - - Spectra - - - - - Fits - - - - - - - - - - - DataPath : - - - - - - - ... - - - - - - - - - - Read Mode: - - - - - - - - Off Line - - - - - On Line - - - - - - - - Delay: - - - - - - - - - - Walk : - - - - - - - - Search 0 - - - - - Search 1 - - - - - - - - Load - - - - - - - Set: - - - - - - - - - - - - - QFrame::StyledPanel - - - QFrame::Raised - - - - - - Star Date: - - - - - - - - - - End Date: - - - - - - - - - - Start Time: - - - - - - - - - - End Time: - - - - - - - - - - Description: - - - - - - - - - - - - - QFrame::StyledPanel - - - QFrame::Raised - - - - - - Ok - - - - - - - Clear - - - - - - - - - - - Voltage - - - - - - QFrame::StyledPanel - - - QFrame::Raised - - - - - - Ok - - - - - - - Clear - - - - - - - - - - 0 - - - - Operation - - - - - - - - - - Value - - - - - Index - - - - - - - - - Value - - - - - Index - - - - - - - - Profile Selector - - - - - - - - Profile List - - - - - Profile Range List - - - - - - - - Decoder - - - - - - - Coherent Integration - - - - - - - - - - - - - Qt::Horizontal - - - - 40 - 20 - - - - - - - - Qt::Horizontal - - - - 40 - 20 - - - - - - - - Qt::Horizontal - - - - 40 - 20 - - - - - - - - - - - Code: - - - - - - - Select Channels - - - - - - - Select Heights - - - - - - - Filter - - - - - - - - - - Radar Frequency - - - - - - - Mode: - - - - - - - Qt::Horizontal - - - - 40 - 20 - - - - - - - - - - - - Barker 3 - - - - - Barker 4 - - - - - Barker 5 - - - - - Barker 7 - - - - - Barker 11 - - - - - Barker 13 - - - - - Barker 3 + Comp. - - - - - Barker 4 + Comp. - - - - - Barker 5 + Comp. - - - - - Barker 7 + Comp. - - - - - Barker 11+ Comp. - - - - - Barker 13+ Comp. - - - - - None - - - - - - - - - Time - - - - - Freq 1 - - - - - - - - - Graphics - - - - - - Qt::Vertical - - - - 20 - 40 - - - - - - - - - - - - - - ... - - - - - - - - - - Scope - - - - - - - Qt::Vertical - - - - 20 - 40 - - - - - - - - Channel List - - - - - - - Show - - - - - - - Freq/Vel - - - - - - - - - - - - - - Path : - - - - - - - Prefix: - - - - - - - Height range - - - - - - - Qt::Horizontal - - - - 40 - 12 - - - - - - - - Qt::Horizontal - - - - 18 - 12 - - - - - - - - - - - - - - Save - - - - - - - - - - - - - - - Output - - - - - - Type: - - - - - - - Path: - - - - - - - - - - ... - - - - - - - - .rawdata - - - - - - - - Qt::Vertical - - - - 20 - 40 - - - - - - - - - - - Blocks per File : - - - - - - - Profiles per Block: - - - - - - - - - - - - - - - Spectra - - - - - - QFrame::StyledPanel - - - QFrame::Raised - - - - - - Ok - - - - - - - Clear - - - - - - - - - - 0 - - - - Operation - - - - - - Select Cross Spectra - - - - - - - - Value - - - - - Index - - - - - - - - - - - - Value - - - - - Index - - - - - - - - - - - - - - Remove DC - - - - - - - Select Heights - - - - - - - Select Channel - - - - - - - - - - - - - x-y pairs - - - - - - - nFFTPoints - - - - - - - Incoherent Integration - - - - - - - - Time Interval - - - - - Profiles - - - - - - - - Qt::Horizontal - - - - 40 - 20 - - - - - - - - Radar Frequency - - - - - - - Qt::Horizontal - - - - 40 - 20 - - - - - - - - Qt::Horizontal - - - - 40 - 20 - - - - - - - - - - - Qt::Horizontal - - - - 40 - 20 - - - - - - - - Profiles - - - - - - - - - - Remove Interference - - - - - - - Qt::Horizontal - - - - 40 - 20 - - - - - - - - - Mode 1 - - - - - Mode 2 - - - - - - - - - Graphics - - - - - - Coherence Map - - - - - - - Qt::Vertical - - - - 20 - 40 - - - - - - - - Tmin, Tmax: - - - - - - - - - - - - - - - - - - - - - - - - - - - Prefix : - - - - - - - - - - Qt::Horizontal - - - - 28 - 15 - - - - - - - - Qt::Vertical - - - - 20 - 40 - - - - - - - - RTI Plot - - - - - - - Height range: - - - - - - - dBs range: - - - - - - - magnitud [0-1] - - - - - - - - - - - - - Qt::Horizontal - - - - 49 - 15 - - - - - - - - Path : - - - - - - - - - - - - - Power Profile - - - - - - - Freq/Vel: - - - - - - - - - - Cross Spectra Plot - - - - - - - ... - - - - - - - Channel List: - - - - - - - Spectra Plot - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Qt::Horizontal - - - - 40 - 20 - - - - - - - - - - - - - - - - - - - - - - Qt::Horizontal - - - - 39 - 15 - - - - - - - - - - - Show - - - - - - - - - - - - - - Wr Period: - - - - - - - Save - - - - - - - - - - - - - - - - - - - - - - - - - - - - ftp - - - - - - - - - - - - - - RTI Noise - - - - - - - - - - - - - - - - - - - - - Time Range: - - - - - - - - - - - Output - - - - - - Type: - - - - - - - - .pdata - - - - - - - - Path: - - - - - - - - - - Qt::Vertical - - - - 20 - 40 - - - - - - - - ... - - - - - - - - - - Blocks per File: - - - - - - - Profile per Block: - - - - - - - - - - - - - - - SpectraHeis - - - - - - QFrame::StyledPanel - - - QFrame::Raised - - - - - - Clear - - - - - - - Ok - - - - - - - - - - 0 - - - - Operation - - - - - - Qt::Horizontal - - - - 40 - 20 - - - - - - - - - Time Interval - - - - - - - - Incoherent Intergration - - - - - - - - - - Qt::Vertical - - - - 20 - 40 - - - - - - - - - Graphics - - - - - - Prefix - - - - - - - ... - - - - - - - - - - - - - - ymin - ymax - - - - - - - Tmin - Tmax: - - - - - - - Time Range: - - - - - - - Wr Period - - - - - - - Qt::Horizontal - - - - 134 - 20 - - - - - - - - - - - - - - - - - - - - - Channel List: - - - - - - - xmin - xmax - - - - - - - - - - Save - - - - - - - ftp - - - - - - - Spectra Plot - - - - - - - - - - - - - - - - - Path - - - - - - - Show - - - - - - - - - - - - - - Qt::Horizontal - - - - 40 - 20 - - - - - - - - - - - - - - - - - Qt::Horizontal - - - - 106 - 20 - - - - - - - - - - - - - - - - - - - - - - RTI PLot - - - - - - - Qt::Horizontal - - - - 40 - 20 - - - - - - - - - - - - Output - - - - - - Path: - - - - - - - Blocks per File: - - - - - - - Type: - - - - - - - Qt::Vertical - - - - 20 - 40 - - - - - - - - ... - - - - - - - - - - - .fits - - - - - - - - Metada - - - - - - - - - - - - - ... - - - - - - - - - - - - Correlation - - - - - - 0 - - - - Operation - - - - - Graphics - - - - - - - - - - - 0 - 0 - - - - 0 - - - - Console - - - - - - - - - - - - 0 - - - - Project Property - - - - - - - - - - - - - - - toolBar - - - TopToolBarArea - - - false - - - - - - - - - - - - - - - - - - - - 0 - 0 - 1065 - 25 - - - - - Project - - - - - - - - - Run - - - - - - - Options - - - - - - Help - - - - - - - - - - - - ../Documents/figure/open.png../Documents/figure/open.png - - - Open - - - - - - ../Documents/figure/create_class_obj.png../Documents/figure/create_class_obj.png - - - Create - - - - - - ../Documents/figure/save.gif../Documents/figure/save.gif - - - Save - - - - - Close - - - - - - ../Documents/figure/startServer.png../Documents/figure/startServer.png - - - Start - - - - - Pause - - - - - About - - - - - - ../Documents/figure/open.png../Documents/figure/open.png - - - openToolbar - - - Open - - - - - - ../Documents/figure/create_class_obj.png../Documents/figure/create_class_obj.png - - - createToolbar - - - Create - - - - - - ../Documents/figure/save.gif../Documents/figure/save.gif - - - saveToolbar - - - Save - - - - - - ../Documents/figure/startServer.png../Documents/figure/startServer.png - - - starToolbar - - - Start - - - - - - ../Documents/figure/stopServer.png../Documents/figure/stopServer.png - - - stopToolbar - - - Stop - - - - - - ../workspace/branch_gui_may/gui/figure/pause.png../workspace/branch_gui_may/gui/figure/pause.png - - - pauseToolbar - - - Pause - - - - - - ../addpu.jpg../addpu.jpg - - - addPU - - - - - FTP - - - - - - diff --git a/schainpy/gui/viewer/viewer_ui/MainWindow_21_02_13_v48.ui b/schainpy/gui/viewer/viewer_ui/MainWindow_21_02_13_v48.ui deleted file mode 100644 index d34bd2e..0000000 --- a/schainpy/gui/viewer/viewer_ui/MainWindow_21_02_13_v48.ui +++ /dev/null @@ -1,2206 +0,0 @@ - - - MainWindow - - - - 0 - 0 - 1065 - 711 - - - - MainWindow - - - - - - - Qt::Horizontal - - - - - Qt::Vertical - - - - - 0 - 278 - - - - - 16777215 - 16777215 - - - - 0 - - - - Project - - - - - - QFrame::StyledPanel - - - QFrame::Raised - - - - - - Project Name : - - - - - - - - - - DataType : - - - - - - - - Voltage - - - - - Spectra - - - - - Fits - - - - - - - - - - - DataPath : - - - - - - - ... - - - - - - - - - - Read Mode: - - - - - - - - Off Line - - - - - On Line - - - - - - - - Delay: - - - - - - - - - - Walk : - - - - - - - - Search 0 - - - - - Search 1 - - - - - - - - Load - - - - - - - Set: - - - - - - - - - - - - - QFrame::StyledPanel - - - QFrame::Raised - - - - - - Star Date: - - - - - - - - - - End Date: - - - - - - - - - - Start Time: - - - - - - - - - - End Time: - - - - - - - - - - Description: - - - - - - - - - - - - - QFrame::StyledPanel - - - QFrame::Raised - - - - - - Ok - - - - - - - Clear - - - - - - - - - - - Voltage - - - - - - QFrame::StyledPanel - - - QFrame::Raised - - - - - - Ok - - - - - - - Clear - - - - - - - - - - 0 - - - - Operation - - - - - - - - - - Value - - - - - Index - - - - - - - - - Value - - - - - Index - - - - - - - - Profile Selector - - - - - - - - Profile List - - - - - Profile Range List - - - - - - - - Decoder - - - - - - - Coherent Integration - - - - - - - - - - - - - Qt::Horizontal - - - - 40 - 20 - - - - - - - - Qt::Horizontal - - - - 40 - 20 - - - - - - - - Qt::Horizontal - - - - 40 - 20 - - - - - - - - - - - Code: - - - - - - - Select Channels - - - - - - - Select Heights - - - - - - - Filter - - - - - - - - - - Radar Frequency - - - - - - - Mode: - - - - - - - Qt::Horizontal - - - - 40 - 20 - - - - - - - - - - - - Barker 3 - - - - - Barker 4 - - - - - Barker 5 - - - - - Barker 7 - - - - - Barker 11 - - - - - Barker 13 - - - - - Barker 3 + Comp. - - - - - Barker 4 + Comp. - - - - - Barker 5 + Comp. - - - - - Barker 7 + Comp. - - - - - Barker 11+ Comp. - - - - - Barker 13+ Comp. - - - - - None - - - - - - - - - Time - - - - - Freq 1 - - - - - - - - - Graphics - - - - - - Qt::Vertical - - - - 20 - 40 - - - - - - - - - - - - - - ... - - - - - - - - - - Scope - - - - - - - Qt::Vertical - - - - 20 - 40 - - - - - - - - Channel List - - - - - - - Show - - - - - - - Freq/Vel - - - - - - - - - - - - - - Path : - - - - - - - Prefix: - - - - - - - Height range - - - - - - - Qt::Horizontal - - - - 40 - 12 - - - - - - - - Qt::Horizontal - - - - 18 - 12 - - - - - - - - - - - - - - Save - - - - - - - - - - - - - - - Output - - - - - - Type: - - - - - - - Path: - - - - - - - - - - ... - - - - - - - - .rawdata - - - - - - - - Qt::Vertical - - - - 20 - 40 - - - - - - - - - - - Blocks per File : - - - - - - - Profiles per Block: - - - - - - - - - - - - - - - Spectra - - - - - - QFrame::StyledPanel - - - QFrame::Raised - - - - - - Ok - - - - - - - Clear - - - - - - - - - - 0 - - - - Operation - - - - - - Select Cross Spectra - - - - - - - - Value - - - - - Index - - - - - - - - - - - - Value - - - - - Index - - - - - - - - - - - - - - Remove DC - - - - - - - Select Heights - - - - - - - Select Channel - - - - - - - - - - - - - x-y pairs - - - - - - - nFFTPoints - - - - - - - Incoherent Integration - - - - - - - - Time Interval - - - - - Profiles - - - - - - - - Qt::Horizontal - - - - 40 - 20 - - - - - - - - Radar Frequency - - - - - - - Qt::Horizontal - - - - 40 - 20 - - - - - - - - Qt::Horizontal - - - - 40 - 20 - - - - - - - - - - - Qt::Horizontal - - - - 40 - 20 - - - - - - - - Profiles - - - - - - - - - - Remove Interference - - - - - - - Qt::Horizontal - - - - 40 - 20 - - - - - - - - - Mode 1 - - - - - Mode 2 - - - - - - - - IppFactor - - - - - - - - - - Get Noise - - - - - - - - - - - Graphics - - - - - - Coherence Map - - - - - - - Qt::Vertical - - - - 20 - 40 - - - - - - - - Tmin, Tmax: - - - - - - - - - - - - - - - - - - - - - - - - - - - Prefix : - - - - - - - - - - Qt::Horizontal - - - - 28 - 15 - - - - - - - - Qt::Vertical - - - - 20 - 40 - - - - - - - - RTI Plot - - - - - - - Height range: - - - - - - - dBs range: - - - - - - - magnitud [0-1] - - - - - - - - - - - - - Qt::Horizontal - - - - 49 - 15 - - - - - - - - Path : - - - - - - - - - - - - - Power Profile - - - - - - - Freq/Vel: - - - - - - - - - - Cross Spectra Plot - - - - - - - ... - - - - - - - Channel List: - - - - - - - Spectra Plot - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Qt::Horizontal - - - - 40 - 20 - - - - - - - - - - - - - - - - - - - - - - Qt::Horizontal - - - - 39 - 15 - - - - - - - - - - - Show - - - - - - - - - - - - - - Wr Period: - - - - - - - Save - - - - - - - - - - - - - - - - - - - - - - - - - - - - ftp - - - - - - - - - - - - - - RTI Noise - - - - - - - - - - - - - - - - - - - - - Time Range: - - - - - - - - - - - Output - - - - - - Type: - - - - - - - - .pdata - - - - - - - - Path: - - - - - - - - - - Qt::Vertical - - - - 20 - 40 - - - - - - - - ... - - - - - - - - - - Blocks per File: - - - - - - - Profile per Block: - - - - - - - - - - - - - - - SpectraHeis - - - - - - QFrame::StyledPanel - - - QFrame::Raised - - - - - - Clear - - - - - - - Ok - - - - - - - - - - 0 - - - - Operation - - - - - - Qt::Horizontal - - - - 40 - 20 - - - - - - - - - Time Interval - - - - - - - - Incoherent Intergration - - - - - - - - - - Qt::Vertical - - - - 20 - 40 - - - - - - - - - Graphics - - - - - - Prefix - - - - - - - ... - - - - - - - - - - - - - - ymin - ymax - - - - - - - Tmin - Tmax: - - - - - - - Time Range: - - - - - - - Wr Period - - - - - - - Qt::Horizontal - - - - 134 - 20 - - - - - - - - - - - - - - - - - - - - - Channel List: - - - - - - - xmin - xmax - - - - - - - - - - Save - - - - - - - ftp - - - - - - - Spectra Plot - - - - - - - - - - - - - - - - - Path - - - - - - - Show - - - - - - - - - - - - - - Qt::Horizontal - - - - 40 - 20 - - - - - - - - - - - - - - - - - Qt::Horizontal - - - - 106 - 20 - - - - - - - - - - - - - - - - - - - - - - RTI PLot - - - - - - - Qt::Horizontal - - - - 40 - 20 - - - - - - - - - - - - Output - - - - - - Path: - - - - - - - Blocks per File: - - - - - - - Type: - - - - - - - Qt::Vertical - - - - 20 - 40 - - - - - - - - ... - - - - - - - - - - - .fits - - - - - - - - Metada - - - - - - - - - - - - - ... - - - - - - - - - - - - Correlation - - - - - - 0 - - - - Operation - - - - - Graphics - - - - - - - - - - - 0 - 0 - - - - 0 - - - - Console - - - - - - - - - - - - 0 - - - - Project Property - - - - - - - - - - - - - - - toolBar - - - TopToolBarArea - - - false - - - - - - - - - - - - - - - - - - - - 0 - 0 - 1065 - 25 - - - - - Project - - - - - - - - - Run - - - - - - - Options - - - - - - Help - - - - - - - - - - - - ../Documents/figure/open.png../Documents/figure/open.png - - - Open - - - - - - ../Documents/figure/create_class_obj.png../Documents/figure/create_class_obj.png - - - Create - - - - - - ../Documents/figure/save.gif../Documents/figure/save.gif - - - Save - - - - - Close - - - - - - ../Documents/figure/startServer.png../Documents/figure/startServer.png - - - Start - - - - - Pause - - - - - About - - - - - - ../Documents/figure/open.png../Documents/figure/open.png - - - openToolbar - - - Open - - - - - - ../Documents/figure/create_class_obj.png../Documents/figure/create_class_obj.png - - - createToolbar - - - Create - - - - - - ../Documents/figure/save.gif../Documents/figure/save.gif - - - saveToolbar - - - Save - - - - - - ../Documents/figure/startServer.png../Documents/figure/startServer.png - - - starToolbar - - - Start - - - - - - ../Documents/figure/stopServer.png../Documents/figure/stopServer.png - - - stopToolbar - - - Stop - - - - - - ../workspace/branch_gui_may/gui/figure/pause.png../workspace/branch_gui_may/gui/figure/pause.png - - - pauseToolbar - - - Pause - - - - - - ../addpu.jpg../addpu.jpg - - - addPU - - - - - FTP - - - - - - diff --git a/schainpy/gui/viewer/viewer_ui/MainWindow_21_02_13_v49.ui b/schainpy/gui/viewer/viewer_ui/MainWindow_21_02_13_v49.ui deleted file mode 100644 index 31cff87..0000000 --- a/schainpy/gui/viewer/viewer_ui/MainWindow_21_02_13_v49.ui +++ /dev/null @@ -1,2206 +0,0 @@ - - - MainWindow - - - - 0 - 0 - 1065 - 711 - - - - MainWindow - - - - - - - Qt::Horizontal - - - - - Qt::Vertical - - - - - 0 - 278 - - - - - 16777215 - 16777215 - - - - 0 - - - - Project - - - - - - QFrame::StyledPanel - - - QFrame::Raised - - - - - - Project Name : - - - - - - - - - - DataType : - - - - - - - - Voltage - - - - - Spectra - - - - - Fits - - - - - - - - - - - DataPath : - - - - - - - ... - - - - - - - - - - Read Mode: - - - - - - - - Off Line - - - - - On Line - - - - - - - - Delay: - - - - - - - - - - Walk : - - - - - - - - Search 0 - - - - - Search 1 - - - - - - - - Load - - - - - - - Set: - - - - - - - - - - - - - QFrame::StyledPanel - - - QFrame::Raised - - - - - - Star Date: - - - - - - - - - - End Date: - - - - - - - - - - Start Time: - - - - - - - - - - End Time: - - - - - - - - - - Description: - - - - - - - - - - - - - QFrame::StyledPanel - - - QFrame::Raised - - - - - - Ok - - - - - - - Clear - - - - - - - - - - - Voltage - - - - - - QFrame::StyledPanel - - - QFrame::Raised - - - - - - Ok - - - - - - - Clear - - - - - - - - - - 0 - - - - Operation - - - - - - - - - - Value - - - - - Index - - - - - - - - - Value - - - - - Index - - - - - - - - Profile Selector - - - - - - - - Profile List - - - - - Profile Range List - - - - - - - - Decoder - - - - - - - Coherent Integration - - - - - - - - - - - - - Qt::Horizontal - - - - 40 - 20 - - - - - - - - Qt::Horizontal - - - - 40 - 20 - - - - - - - - Qt::Horizontal - - - - 40 - 20 - - - - - - - - - - - Code: - - - - - - - Select Channels - - - - - - - Select Heights - - - - - - - Filter - - - - - - - - - - Radar Frequency - - - - - - - Mode: - - - - - - - Qt::Horizontal - - - - 40 - 20 - - - - - - - - - - - - Barker 3 - - - - - Barker 4 - - - - - Barker 5 - - - - - Barker 7 - - - - - Barker 11 - - - - - Barker 13 - - - - - Barker 3 + Comp. - - - - - Barker 4 + Comp. - - - - - Barker 5 + Comp. - - - - - Barker 7 + Comp. - - - - - Barker 11+ Comp. - - - - - Barker 13+ Comp. - - - - - None - - - - - - - - - Time - - - - - Freq 1 - - - - - - - - - Graphics - - - - - - Qt::Vertical - - - - 20 - 40 - - - - - - - - - - - - - - ... - - - - - - - - - - Scope - - - - - - - Qt::Vertical - - - - 20 - 40 - - - - - - - - Channel List - - - - - - - Show - - - - - - - Freq/Vel - - - - - - - - - - - - - - Path : - - - - - - - Prefix: - - - - - - - Height range - - - - - - - Qt::Horizontal - - - - 40 - 12 - - - - - - - - Qt::Horizontal - - - - 18 - 12 - - - - - - - - - - - - - - Save - - - - - - - - - - - - - - - Output - - - - - - Type: - - - - - - - Path: - - - - - - - - - - ... - - - - - - - - .rawdata - - - - - - - - Qt::Vertical - - - - 20 - 40 - - - - - - - - - - - Blocks per File : - - - - - - - Profiles per Block: - - - - - - - - - - - - - - - Spectra - - - - - - QFrame::StyledPanel - - - QFrame::Raised - - - - - - Ok - - - - - - - Clear - - - - - - - - - - 0 - - - - Operation - - - - - - Select Cross Spectra - - - - - - - - Value - - - - - Index - - - - - - - - - - - - Value - - - - - Index - - - - - - - - - - - - - - Remove DC - - - - - - - Select Heights - - - - - - - Select Channel - - - - - - - - - - - - - x-y pairs - - - - - - - nFFTPoints - - - - - - - Incoherent Integration - - - - - - - - Time Interval - - - - - Profiles - - - - - - - - Qt::Horizontal - - - - 40 - 20 - - - - - - - - Radar Frequency - - - - - - - Qt::Horizontal - - - - 40 - 20 - - - - - - - - Qt::Horizontal - - - - 40 - 20 - - - - - - - - - - - Qt::Horizontal - - - - 40 - 20 - - - - - - - - Profiles - - - - - - - - - - Remove Interference - - - - - - - Qt::Horizontal - - - - 40 - 20 - - - - - - - - IppFactor - - - - - - - Get Noise - - - - - - - - - - - Mode 1 - - - - - Mode 2 - - - - - - - - - - - - Graphics - - - - - - Coherence Map - - - - - - - Qt::Vertical - - - - 20 - 40 - - - - - - - - Tmin, Tmax: - - - - - - - - - - - - - - - - - - - - - - - - - - - Prefix : - - - - - - - - - - Qt::Horizontal - - - - 28 - 15 - - - - - - - - Qt::Vertical - - - - 20 - 40 - - - - - - - - RTI Plot - - - - - - - Height range: - - - - - - - dBs range: - - - - - - - magnitud [0-1] - - - - - - - - - - - - - Qt::Horizontal - - - - 49 - 15 - - - - - - - - Path : - - - - - - - - - - - - - Power Profile - - - - - - - Freq/Vel: - - - - - - - - - - Cross Spectra Plot - - - - - - - ... - - - - - - - Channel List: - - - - - - - Spectra Plot - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Qt::Horizontal - - - - 40 - 20 - - - - - - - - - - - - - - - - - - - - - - Qt::Horizontal - - - - 39 - 15 - - - - - - - - - - - Show - - - - - - - - - - - - - - Wr Period: - - - - - - - Save - - - - - - - - - - - - - - - - - - - - - - - - - - - - ftp - - - - - - - - - - - - - - RTI Noise - - - - - - - - - - - - - - - - - - - - - Time Range: - - - - - - - - - - - Output - - - - - - Type: - - - - - - - - .pdata - - - - - - - - Path: - - - - - - - - - - Qt::Vertical - - - - 20 - 40 - - - - - - - - ... - - - - - - - - - - Blocks per File: - - - - - - - Profile per Block: - - - - - - - - - - - - - - - SpectraHeis - - - - - - QFrame::StyledPanel - - - QFrame::Raised - - - - - - Clear - - - - - - - Ok - - - - - - - - - - 0 - - - - Operation - - - - - - Qt::Horizontal - - - - 40 - 20 - - - - - - - - - Time Interval - - - - - - - - Incoherent Intergration - - - - - - - - - - Qt::Vertical - - - - 20 - 40 - - - - - - - - - Graphics - - - - - - Prefix - - - - - - - ... - - - - - - - - - - - - - - ymin - ymax - - - - - - - Tmin - Tmax: - - - - - - - Time Range: - - - - - - - Wr Period - - - - - - - Qt::Horizontal - - - - 134 - 20 - - - - - - - - - - - - - - - - - - - - - Channel List: - - - - - - - xmin - xmax - - - - - - - - - - Save - - - - - - - ftp - - - - - - - Spectra Plot - - - - - - - - - - - - - - - - - Path - - - - - - - Show - - - - - - - - - - - - - - Qt::Horizontal - - - - 40 - 20 - - - - - - - - - - - - - - - - - Qt::Horizontal - - - - 106 - 20 - - - - - - - - - - - - - - - - - - - - - - RTI PLot - - - - - - - Qt::Horizontal - - - - 40 - 20 - - - - - - - - - - - - Output - - - - - - Path: - - - - - - - Blocks per File: - - - - - - - Type: - - - - - - - Qt::Vertical - - - - 20 - 40 - - - - - - - - ... - - - - - - - - - - - .fits - - - - - - - - Metada - - - - - - - - - - - - - ... - - - - - - - - - - - - Correlation - - - - - - 0 - - - - Operation - - - - - Graphics - - - - - - - - - - - 0 - 0 - - - - 0 - - - - Console - - - - - - - - - - - - 0 - - - - Project Property - - - - - - - - - - - - - - - toolBar - - - TopToolBarArea - - - false - - - - - - - - - - - - - - - - - - - - 0 - 0 - 1065 - 25 - - - - - Project - - - - - - - - - Run - - - - - - - Options - - - - - - Help - - - - - - - - - - - - ../Documents/figure/open.png../Documents/figure/open.png - - - Open - - - - - - ../Documents/figure/create_class_obj.png../Documents/figure/create_class_obj.png - - - Create - - - - - - ../Documents/figure/save.gif../Documents/figure/save.gif - - - Save - - - - - Close - - - - - - ../Documents/figure/startServer.png../Documents/figure/startServer.png - - - Start - - - - - Pause - - - - - About - - - - - - ../Documents/figure/open.png../Documents/figure/open.png - - - openToolbar - - - Open - - - - - - ../Documents/figure/create_class_obj.png../Documents/figure/create_class_obj.png - - - createToolbar - - - Create - - - - - - ../Documents/figure/save.gif../Documents/figure/save.gif - - - saveToolbar - - - Save - - - - - - ../Documents/figure/startServer.png../Documents/figure/startServer.png - - - starToolbar - - - Start - - - - - - ../Documents/figure/stopServer.png../Documents/figure/stopServer.png - - - stopToolbar - - - Stop - - - - - - ../workspace/branch_gui_may/gui/figure/pause.png../workspace/branch_gui_may/gui/figure/pause.png - - - pauseToolbar - - - Pause - - - - - - ../addpu.jpg../addpu.jpg - - - addPU - - - - - FTP - - - - - - diff --git a/schainpy/gui/viewer/viewer_ui/ftpConfig4.ui b/schainpy/gui/viewer/viewer_ui/ftpConfig4.ui deleted file mode 100644 index ec05dbf..0000000 --- a/schainpy/gui/viewer/viewer_ui/ftpConfig4.ui +++ /dev/null @@ -1,300 +0,0 @@ - - - MainWindow - - - - 0 - 0 - 374 - 399 - - - - - 374 - 399 - - - - - 374 - 399 - - - - MainWindow - - - - - - 9 - 38 - 47 - 17 - - - - Server: - - - - - - 9 - 133 - 77 - 17 - - - - User Name: - - - - - - 9 - 166 - 68 - 17 - - - - Password: - - - - - - 9 - 9 - 101 - 17 - - - - - 75 - true - - - - Server Details - - - - - - 9 - 104 - 87 - 17 - - - - - 75 - true - - - - User Details - - - - - - 9 - 71 - 47 - 17 - - - - Folder: - - - - - - 130 - 40 - 231 - 27 - - - - - - - 130 - 70 - 231 - 27 - - - - - - - 130 - 130 - 231 - 27 - - - - - - - 130 - 160 - 231 - 27 - - - - - - - 130 - 360 - 111 - 27 - - - - Cancel - - - - - - 250 - 360 - 111 - 27 - - - - Ok - - - - - - 10 - 200 - 66 - 17 - - - - - 75 - true - - - - Others - - - - - - 10 - 230 - 81 - 17 - - - - Ftp_wei: - - - - - - 10 - 260 - 81 - 17 - - - - Exp_code: - - - - - - 10 - 290 - 81 - 17 - - - - Sub_exp: - - - - - - 10 - 320 - 81 - 17 - - - - Plot_pos: - - - - - - 130 - 230 - 231 - 27 - - - - - - - 130 - 260 - 231 - 27 - - - - - - - 130 - 290 - 231 - 27 - - - - - - - 130 - 320 - 231 - 27 - - - - - - - - diff --git a/schainpy/gui/viewer/viewer_ui/initwindowv2.ui b/schainpy/gui/viewer/viewer_ui/initwindowv2.ui deleted file mode 100644 index d254434..0000000 --- a/schainpy/gui/viewer/viewer_ui/initwindowv2.ui +++ /dev/null @@ -1,112 +0,0 @@ - - - Dialog - - - - 0 - 0 - 652 - 496 - - - - Dialog - - - - - - - - - - - Cambria - 22 - 50 - false - - - - Signal Chain - Ver. 1.0 - - - - - - - Qt::Horizontal - - - - - - - - - - ../../Downloads/IMAGENES/w.jpg - - - true - - - - - - - - - - - - - Qt::Horizontal - - - - 40 - 20 - - - - - - - - Exit - - - - - - - Continue - - - - - - - Qt::Horizontal - - - - 40 - 20 - - - - - - - - - - - - - - - diff --git a/schainpy/gui/viewer/viewer_ui/unitProcess4.ui b/schainpy/gui/viewer/viewer_ui/unitProcess4.ui deleted file mode 100644 index d077917..0000000 --- a/schainpy/gui/viewer/viewer_ui/unitProcess4.ui +++ /dev/null @@ -1,170 +0,0 @@ - - - MainWindow - - - - 0 - 0 - 312 - 195 - - - - MainWindow - - - - - - - - 12 - 50 - false - - - - Input: - - - - - - - - 50 - false - - - - Cancel - - - - - - - - 50 - false - - - - Ok - - - - - - - - 12 - 50 - false - - - - Type: - - - - - - - - 13 - 50 - false - - - - Processing Unit - - - - - - - Qt::Horizontal - - - - 40 - 20 - - - - - - - - - 12 - - - - - - - - - 12 - - - - - Voltage - - - - - Spectra - - - - - Correlation - - - - - - - - Qt::Horizontal - - - - 40 - 20 - - - - - - - - Qt::Vertical - - - - 20 - 40 - - - - - - - - Qt::Horizontal - - - - - - - - - diff --git a/schainpy/gui/viewer/viewer_ui/workspacev5.ui b/schainpy/gui/viewer/viewer_ui/workspacev5.ui deleted file mode 100644 index 5aad07a..0000000 --- a/schainpy/gui/viewer/viewer_ui/workspacev5.ui +++ /dev/null @@ -1,247 +0,0 @@ - - - Dialog - - - true - - - - 0 - 0 - 730 - 295 - - - - - 730 - 295 - - - - - 730 - 295 - - - - Dialog - - - - - 0 - 0 - 731 - 71 - - - - true - - - <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0//EN" "http://www.w3.org/TR/REC-html40/strict.dtd"> -<html><head><meta name="qrichtext" content="1" /><style type="text/css"> -p, li { white-space: pre-wrap; } -</style></head><body style=" font-family:'Ubuntu'; font-size:11pt; font-weight:400; font-style:normal;"> -<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px;"><span style=" font-family:'Cantarell'; font-size:12pt; font-weight:600;">Select a workspace</span></p> -<p align="justify" style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px;"><span style=" font-family:'Cantarell';">Signal Chain stores your projects in a folder called a workspace.</span></p> -<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px;"><span style=" font-family:'Cantarell';">Choose a workspace folder to use for this session.</span></p></body></html> - - - - - - 10 - 90 - 91 - 16 - - - - - 11 - - - - Workspace : - - - - - - 110 - 80 - 501 - 31 - - - - - - - - - 255 - 255 - 255 - - - - - - - - - 255 - 255 - 255 - - - - - - - - - 255 - 255 - 255 - - - - - - - - - - - 620 - 80 - 101 - 31 - - - - - 11 - - - - Browse... - - - - - - 10 - 120 - 711 - 121 - - - - - - - Qt::Vertical - - - - 20 - 40 - - - - - - - - Qt::Vertical - - - - 20 - 40 - - - - - - - - Qt::Vertical - - - - 20 - 40 - - - - - - - - Qt::Vertical - - - - 20 - 40 - - - - - - - - Qt::Vertical - - - - 20 - 40 - - - - - - - - - - 490 - 250 - 111 - 31 - - - - Cancel - - - - - - 610 - 250 - 111 - 31 - - - - - 11 - - - - OK - - - - - - diff --git a/schainpy/gui/viewer/windows/__init__.py b/schainpy/gui/viewer/windows/__init__.py deleted file mode 100644 index af97a97..0000000 --- a/schainpy/gui/viewer/windows/__init__.py +++ /dev/null @@ -1,5 +0,0 @@ -from ui_project import * -from ui_voltage import * -from ui_spectra import * -from ui_spectra_heis import * -from ui_correlation import * \ No newline at end of file diff --git a/schainpy/gui/viewer/windows/ui_correlation.py b/schainpy/gui/viewer/windows/ui_correlation.py deleted file mode 100644 index 0c80785..0000000 --- a/schainpy/gui/viewer/windows/ui_correlation.py +++ /dev/null @@ -1,45 +0,0 @@ - -from PyQt4 import QtCore, QtGui - -try: - _fromUtf8 = QtCore.QString.fromUtf8 -except AttributeError: - def _fromUtf8(s): - return s - -try: - _encoding = QtGui.QApplication.UnicodeUTF8 - def _translate(context, text, disambig): - return QtGui.QApplication.translate(context, text, disambig, _encoding) -except AttributeError: - def _translate(context, text, disambig): - return QtGui.QApplication.translate(context, text, disambig) - -class Ui_CorrelationTab(object): - - def setupUi(self): - - self.tabCorrelation = QtGui.QWidget() - self.tabCorrelation.setObjectName(_fromUtf8("tabCorrelation")) - self.gridLayout_13 = QtGui.QGridLayout(self.tabCorrelation) - self.gridLayout_13.setObjectName(_fromUtf8("gridLayout_13")) - self.tabWidgetCorrelation = QtGui.QTabWidget(self.tabCorrelation) - self.tabWidgetCorrelation.setObjectName(_fromUtf8("tabWidgetCorrelation")) - self.tabopCorrelation = QtGui.QWidget() - self.tabopCorrelation.setObjectName(_fromUtf8("tabopCorrelation")) - self.tabWidgetCorrelation.addTab(self.tabopCorrelation, _fromUtf8("")) - self.tabopCorrelation1 = QtGui.QWidget() - self.tabopCorrelation1.setObjectName(_fromUtf8("tabopCorrelation1")) - self.tabWidgetCorrelation.addTab(self.tabopCorrelation1, _fromUtf8("")) - self.gridLayout_13.addWidget(self.tabWidgetCorrelation, 0, 0, 1, 1) - - self.tabWidgetProject.addTab(self.tabCorrelation, _fromUtf8("")) - - self.tabWidgetCorrelation.setCurrentIndex(0) - - def retranslateUi(self): - - self.tabWidgetCorrelation.setTabText(self.tabWidgetCorrelation.indexOf(self.tabopCorrelation), _translate("MainWindow", "Operation", None)) - self.tabWidgetCorrelation.setTabText(self.tabWidgetCorrelation.indexOf(self.tabopCorrelation1), _translate("MainWindow", "Graphics", None)) - self.tabWidgetProject.setTabText(self.tabWidgetProject.indexOf(self.tabCorrelation), _translate("MainWindow", "Correlation", None)) - \ No newline at end of file diff --git a/schainpy/gui/viewer/windows/ui_project.py b/schainpy/gui/viewer/windows/ui_project.py deleted file mode 100644 index 7b22c32..0000000 --- a/schainpy/gui/viewer/windows/ui_project.py +++ /dev/null @@ -1,257 +0,0 @@ - -from PyQt4 import QtCore, QtGui - -try: - _fromUtf8 = QtCore.QString.fromUtf8 -except AttributeError: - def _fromUtf8(s): - return s - -try: - _encoding = QtGui.QApplication.UnicodeUTF8 - def _translate(context, text, disambig): - return QtGui.QApplication.translate(context, text, disambig, _encoding) -except AttributeError: - def _translate(context, text, disambig): - return QtGui.QApplication.translate(context, text, disambig) - -class Ui_ProjectTab(object): - - def setupUi(self): - - self.tabProject = QtGui.QWidget() - self.tabProject.setObjectName(_fromUtf8("tabProject")) - self.gridLayout_15 = QtGui.QGridLayout(self.tabProject) - self.gridLayout_15.setObjectName(_fromUtf8("gridLayout_15")) - - self.frame = QtGui.QFrame(self.tabProject) - self.frame.setFrameShape(QtGui.QFrame.StyledPanel) - self.frame.setFrameShadow(QtGui.QFrame.Raised) - self.frame.setObjectName(_fromUtf8("frame")) - - self.gridLayout_2 = QtGui.QGridLayout(self.frame) - self.gridLayout_2.setObjectName(_fromUtf8("gridLayout_2")) - - self.label = QtGui.QLabel(self.frame) - self.label.setObjectName(_fromUtf8("label")) - self.gridLayout_2.addWidget(self.label, 0, 0, 1, 1) - - self.proName = QtGui.QLineEdit(self.frame) - self.proName.setObjectName(_fromUtf8("proName")) - self.gridLayout_2.addWidget(self.proName, 0, 1, 1, 8) - - self.labDatatype = QtGui.QLabel(self.frame) - self.labDatatype.setObjectName(_fromUtf8("labDatatype")) - self.gridLayout_2.addWidget(self.labDatatype, 1, 0, 1, 1) - - self.proComDataType = QtGui.QComboBox(self.frame) - self.proComDataType.setObjectName(_fromUtf8("proComDataType")) - self.proComDataType.addItem(_fromUtf8("")) - self.proComDataType.addItem(_fromUtf8("")) - self.proComDataType.addItem(_fromUtf8("")) - self.proComDataType.addItem(_fromUtf8("")) - self.gridLayout_2.addWidget(self.proComDataType, 1, 1, 1, 6) - - self.proDataType = QtGui.QLineEdit(self.frame) - self.proDataType.setObjectName(_fromUtf8("proDataType")) - self.gridLayout_2.addWidget(self.proDataType, 1, 7, 1, 2) - - self.labDatapath = QtGui.QLabel(self.frame) - self.labDatapath.setObjectName(_fromUtf8("labDatapath")) - self.gridLayout_2.addWidget(self.labDatapath, 2, 0, 1, 1) - - self.proToolPath = QtGui.QToolButton(self.frame) - self.proToolPath.setObjectName(_fromUtf8("proToolPath")) - self.gridLayout_2.addWidget(self.proToolPath, 2, 1, 1, 1) - - self.proDataPath = QtGui.QLineEdit(self.frame) - self.proDataPath.setObjectName(_fromUtf8("proDataPath")) - self.gridLayout_2.addWidget(self.proDataPath, 2, 2, 1, 7) - - self.labelWalk = QtGui.QLabel(self.frame) - self.labelWalk.setObjectName(_fromUtf8("labelWalk")) - self.gridLayout_2.addWidget(self.labelWalk, 3, 0, 1, 1) - - self.proComWalk = QtGui.QComboBox(self.frame) - self.proComWalk.setObjectName(_fromUtf8("proComWalk")) - self.proComWalk.addItem(_fromUtf8("")) - self.proComWalk.addItem(_fromUtf8("")) - self.gridLayout_2.addWidget(self.proComWalk, 3, 1, 1, 4) - - self.labExpLabel = QtGui.QLabel(self.frame) - self.labExpLabel.setObjectName(_fromUtf8("labExpLabel")) - self.gridLayout_2.addWidget(self.labExpLabel, 3, 5, 1, 1) - - self.proExpLabel = QtGui.QLineEdit(self.frame) - self.proExpLabel.setObjectName(_fromUtf8("proExpLabel")) - self.gridLayout_2.addWidget(self.proExpLabel, 3, 6, 1, 1) - - self.labReadMode = QtGui.QLabel(self.frame) - self.labReadMode.setObjectName(_fromUtf8("labReadMode")) - self.gridLayout_2.addWidget(self.labReadMode, 4, 0, 1, 1) - - self.proComReadMode = QtGui.QComboBox(self.frame) - self.proComReadMode.setObjectName(_fromUtf8("proComReadMode")) - self.proComReadMode.addItem(_fromUtf8("")) - self.proComReadMode.addItem(_fromUtf8("")) - self.gridLayout_2.addWidget(self.proComReadMode, 4, 1, 1, 4) - - self.labDelay = QtGui.QLabel(self.frame) - self.labDelay.setObjectName(_fromUtf8("labDelay")) - self.gridLayout_2.addWidget(self.labDelay, 4, 5, 1, 1) - - self.proDelay = QtGui.QLineEdit(self.frame) - self.proDelay.setObjectName(_fromUtf8("proDelay")) - self.gridLayout_2.addWidget(self.proDelay, 4, 6, 1, 1) - - self.labelSet = QtGui.QLabel(self.frame) - self.labelSet.setObjectName(_fromUtf8("labelSet")) - self.gridLayout_2.addWidget(self.labelSet, 4, 7, 1, 1) - - self.proSet = QtGui.QLineEdit(self.frame) - self.proSet.setObjectName(_fromUtf8("proSet")) - self.gridLayout_2.addWidget(self.proSet, 4, 8, 1, 1) - - - self.proLoadButton = QtGui.QPushButton(self.frame) - self.proLoadButton.setObjectName(_fromUtf8("proLoadButton")) - self.gridLayout_2.addWidget(self.proLoadButton, 5, 0, 1, 9) - - self.frame_data = QtGui.QFrame(self.tabProject) - self.frame_data.setFrameShape(QtGui.QFrame.StyledPanel) - self.frame_data.setFrameShadow(QtGui.QFrame.Raised) - self.frame_data.setObjectName(_fromUtf8("frame_data")) - - self.gridLayout_data = QtGui.QGridLayout(self.frame_data) - self.gridLayout_data.setObjectName(_fromUtf8("gridLayout_data")) - - self.labelIPPKm = QtGui.QLabel(self.frame_data) - self.labelIPPKm.setObjectName(_fromUtf8("labelIPPKm")) - self.gridLayout_data.addWidget(self.labelIPPKm, 6, 0, 1, 1) - - self.proIPPKm = QtGui.QLineEdit(self.frame_data) - self.proIPPKm.setObjectName(_fromUtf8("proIPPKm")) - self.gridLayout_data.addWidget(self.proIPPKm, 6, 1, 1, 6) - - self.labnTxs = QtGui.QLabel(self.frame_data) - self.labnTxs.setObjectName(_fromUtf8("labnTxs")) - self.gridLayout_data.addWidget(self.labnTxs, 6, 0, 1, 1) - - self.pronTxs = QtGui.QLineEdit(self.frame_data) - self.pronTxs.setObjectName(_fromUtf8("pronTxs")) - self.gridLayout_data.addWidget(self.pronTxs, 6, 1, 1, 6) - - self.labByBlock = QtGui.QLabel(self.frame_data) - self.labByBlock.setObjectName(_fromUtf8("labByBlock")) - self.gridLayout_data.addWidget(self.labByBlock, 6, 7, 1, 1) - - self.proComByBlock = QtGui.QComboBox(self.frame_data) - self.proComByBlock.setObjectName(_fromUtf8("proComByBlock")) - self.proComByBlock.addItem(_fromUtf8("")) - self.proComByBlock.addItem(_fromUtf8("")) - self.gridLayout_data.addWidget(self.proComByBlock, 6, 8, 1, 1) - - - self.frame_2 = QtGui.QFrame(self.tabProject) - self.frame_2.setFrameShape(QtGui.QFrame.StyledPanel) - self.frame_2.setFrameShadow(QtGui.QFrame.Raised) - self.frame_2.setObjectName(_fromUtf8("frame_2")) - - self.gridLayout_10 = QtGui.QGridLayout(self.frame_2) - self.gridLayout_10.setObjectName(_fromUtf8("gridLayout_10")) - - self.label_27 = QtGui.QLabel(self.frame_2) - self.label_27.setObjectName(_fromUtf8("label_27")) - self.gridLayout_10.addWidget(self.label_27, 0, 0, 1, 1) - self.proComStartDate = QtGui.QComboBox(self.frame_2) - self.proComStartDate.setObjectName(_fromUtf8("proComStartDate")) - self.gridLayout_10.addWidget(self.proComStartDate, 0, 1, 1, 1) - self.label_28 = QtGui.QLabel(self.frame_2) - self.label_28.setObjectName(_fromUtf8("label_28")) - self.gridLayout_10.addWidget(self.label_28, 1, 0, 1, 1) - self.proComEndDate = QtGui.QComboBox(self.frame_2) - self.proComEndDate.setObjectName(_fromUtf8("proComEndDate")) - self.gridLayout_10.addWidget(self.proComEndDate, 1, 1, 1, 1) - self.label_2 = QtGui.QLabel(self.frame_2) - self.label_2.setObjectName(_fromUtf8("label_2")) - self.gridLayout_10.addWidget(self.label_2, 2, 0, 1, 1) - self.proStartTime = QtGui.QTimeEdit(self.frame_2) - self.proStartTime.setObjectName(_fromUtf8("proStartTime")) - self.gridLayout_10.addWidget(self.proStartTime, 2, 1, 1, 1) - self.label_3 = QtGui.QLabel(self.frame_2) - self.label_3.setObjectName(_fromUtf8("label_3")) - self.gridLayout_10.addWidget(self.label_3, 3, 0, 1, 1) - self.proEndTime = QtGui.QTimeEdit(self.frame_2) - self.proEndTime.setObjectName(_fromUtf8("proEndTime")) - self.gridLayout_10.addWidget(self.proEndTime, 3, 1, 1, 1) - self.label_30 = QtGui.QLabel(self.frame_2) - self.label_30.setObjectName(_fromUtf8("label_30")) - self.gridLayout_10.addWidget(self.label_30, 4, 0, 1, 1) - self.proDescription = QtGui.QTextEdit(self.frame_2) - self.proDescription.setObjectName(_fromUtf8("proDescription")) - self.gridLayout_10.addWidget(self.proDescription, 4, 1, 1, 1) - - self.frame_3 = QtGui.QFrame(self.tabProject) - self.frame_3.setFrameShape(QtGui.QFrame.StyledPanel) - self.frame_3.setFrameShadow(QtGui.QFrame.Raised) - self.frame_3.setObjectName(_fromUtf8("frame_3")) - self.gridLayout_14 = QtGui.QGridLayout(self.frame_3) - self.gridLayout_14.setObjectName(_fromUtf8("gridLayout_14")) - self.proOk = QtGui.QPushButton(self.frame_3) - self.proOk.setObjectName(_fromUtf8("proOk")) - self.gridLayout_14.addWidget(self.proOk, 0, 0, 1, 1) - self.proClear = QtGui.QPushButton(self.frame_3) - self.proClear.setObjectName(_fromUtf8("proClear")) - self.gridLayout_14.addWidget(self.proClear, 0, 1, 1, 1) - - self.gridLayout_15.addWidget(self.frame, 0, 0, 8, 1) - self.gridLayout_15.addWidget(self.frame_data, 8, 0, 2, 1) - self.gridLayout_15.addWidget(self.frame_2, 10, 0, 7, 1) - self.gridLayout_15.addWidget(self.frame_3, 17, 0, 2, 1) - - self.tabWidgetProject.addTab(self.tabProject, _fromUtf8("")) - - def retranslateUi(self): - - self.label.setText(_translate("MainWindow", "Project Name :", None)) - self.labDatatype.setText(_translate("MainWindow", "Data type :", None)) - self.proComDataType.setItemText(0, _translate("MainWindow", "Voltage", None)) - self.proComDataType.setItemText(1, _translate("MainWindow", "Spectra", None)) - self.proComDataType.setItemText(2, _translate("MainWindow", "Fits", None)) - self.proComDataType.setItemText(3, _translate("MainWindow", "USRP", None)) - self.labDatapath.setText(_translate("MainWindow", "Data path :", None)) - self.proToolPath.setText(_translate("MainWindow", "...", None)) - self.labReadMode.setText(_translate("MainWindow", "Read mode:", None)) - self.proComReadMode.setItemText(0, _translate("MainWindow", "Offline", None)) - self.proComReadMode.setItemText(1, _translate("MainWindow", "Online", None)) - self.labDelay.setText(_translate("MainWindow", "Delay:", None)) - self.labExpLabel.setText(_translate("MainWindow", "Exp. Label:", None)) - self.labelWalk.setText(_translate("MainWindow", "Search data :", None)) - self.proComWalk.setItemText(0, _translate("MainWindow", "On files", None)) - self.proComWalk.setItemText(1, _translate("MainWindow", "On sub-folders", None)) - self.proComByBlock.setItemText(0, _translate("MainWindow", "By profile", None)) - self.proComByBlock.setItemText(1, _translate("MainWindow", "By block", None)) - self.labByBlock.setText(_translate("MainWindow", "Get data:", None)) - - - self.proLoadButton.setText(_translate("MainWindow", "Load", None)) - self.labelSet.setText(_translate("MainWindow", "File set:", None)) - self.labelIPPKm.setText(_translate("MainWindow", "IPP (km):", None)) - self.labnTxs.setText(_translate("MainWindow", "Number of Txs:", None)) - - self.label_27.setText(_translate("MainWindow", "Star Date:", None)) - self.label_28.setText(_translate("MainWindow", "End Date:", None)) - self.label_2.setText(_translate("MainWindow", "Start Time:", None)) - self.label_3.setText(_translate("MainWindow", "End Time:", None)) - self.label_30.setText(_translate("MainWindow", "Description:", None)) - self.proOk.setText(_translate("MainWindow", "Ok", None)) - self.proClear.setText(_translate("MainWindow", "Clear", None)) - - self.tabWidgetProject.setTabText(self.tabWidgetProject.indexOf(self.tabProject), _translate("MainWindow", "Project", None)) - - self.__setToolTip() - - def __setToolTip(self): - - self.proComWalk.setToolTip('On Files:Search file in format .r or pdata On Folders:Search file in a directory DYYYYDOY') - \ No newline at end of file diff --git a/schainpy/gui/viewer/windows/ui_spectra.py b/schainpy/gui/viewer/windows/ui_spectra.py deleted file mode 100644 index c1d09ab..0000000 --- a/schainpy/gui/viewer/windows/ui_spectra.py +++ /dev/null @@ -1,528 +0,0 @@ -from PyQt4 import QtCore, QtGui - -try: - _fromUtf8 = QtCore.QString.fromUtf8 -except AttributeError: - def _fromUtf8(s): - return s - -try: - _encoding = QtGui.QApplication.UnicodeUTF8 - def _translate(context, text, disambig): - return QtGui.QApplication.translate(context, text, disambig, _encoding) -except AttributeError: - def _translate(context, text, disambig): - return QtGui.QApplication.translate(context, text, disambig) - -class Ui_SpectraTab(object): - - def setupUi(self): - - self.tabSpectra = QtGui.QWidget() - self.tabSpectra.setObjectName(_fromUtf8("tabSpectra")) - self.gridLayout_7 = QtGui.QGridLayout(self.tabSpectra) - self.gridLayout_7.setObjectName(_fromUtf8("gridLayout_7")) - self.frame_5 = QtGui.QFrame(self.tabSpectra) - self.frame_5.setFrameShape(QtGui.QFrame.StyledPanel) - self.frame_5.setFrameShadow(QtGui.QFrame.Raised) - self.frame_5.setObjectName(_fromUtf8("frame_5")) - self.gridLayout_18 = QtGui.QGridLayout(self.frame_5) - self.gridLayout_18.setObjectName(_fromUtf8("gridLayout_18")) - - self.specOpOk = QtGui.QPushButton(self.frame_5) - self.specOpOk.setObjectName(_fromUtf8("specOpOk")) - self.gridLayout_18.addWidget(self.specOpOk, 0, 0, 1, 1) - - self.specGraphClear = QtGui.QPushButton(self.frame_5) - self.specGraphClear.setObjectName(_fromUtf8("specGraphClear")) - self.gridLayout_18.addWidget(self.specGraphClear, 0, 1, 1, 1) - - self.gridLayout_7.addWidget(self.frame_5, 1, 1, 1, 1) - - self.tabWidgetSpectra = QtGui.QTabWidget(self.tabSpectra) - self.tabWidgetSpectra.setObjectName(_fromUtf8("tabWidgetSpectra")) - - self.tabopSpectra = QtGui.QWidget() - self.tabopSpectra.setObjectName(_fromUtf8("tabopSpectra")) - - self.gridLayout_5 = QtGui.QGridLayout(self.tabopSpectra) - self.gridLayout_5.setObjectName(_fromUtf8("gridLayout_5")) - - - - self.specOpCebRadarfrequency = QtGui.QCheckBox(self.tabopSpectra) - self.specOpCebRadarfrequency.setObjectName(_fromUtf8("specOpCebRadarfrequency")) - self.gridLayout_5.addWidget(self.specOpCebRadarfrequency, 0, 0, 1, 1) - - self.specOpRadarfrequency = QtGui.QLineEdit(self.tabopSpectra) - self.specOpRadarfrequency.setObjectName(_fromUtf8("specOpRadarfrequency")) - self.gridLayout_5.addWidget(self.specOpRadarfrequency, 0, 1, 1, 4) - - - self.specLabProfiles = QtGui.QLabel(self.tabopSpectra) - self.specLabProfiles.setObjectName(_fromUtf8("specLabProfiles")) - self.gridLayout_5.addWidget(self.specLabProfiles, 1, 0, 1, 1) - - self.specOpProfiles = QtGui.QLineEdit(self.tabopSpectra) - self.specOpProfiles.setObjectName(_fromUtf8("specOpProfiles")) - self.gridLayout_5.addWidget(self.specOpProfiles, 1, 1, 1, 4) - - - self.specLabnFFTPoints = QtGui.QLabel(self.tabopSpectra) - self.specLabnFFTPoints.setObjectName(_fromUtf8("specLabnFFTPoints")) - self.gridLayout_5.addWidget(self.specLabnFFTPoints, 2, 0, 1, 1) - - self.specOpnFFTpoints = QtGui.QLineEdit(self.tabopSpectra) - self.specOpnFFTpoints.setObjectName(_fromUtf8("specOpnFFTpoints")) - self.gridLayout_5.addWidget(self.specOpnFFTpoints, 2, 1, 1, 4) - - - self.specLabippFactor = QtGui.QLabel(self.tabopSpectra) - self.specLabippFactor.setObjectName(_fromUtf8("specLabippFactor")) - self.gridLayout_5.addWidget(self.specLabippFactor, 3, 0, 1, 1) - - self.specOpippFactor = QtGui.QLineEdit(self.tabopSpectra) - self.specOpippFactor.setObjectName(_fromUtf8("specOpippFactor")) - self.gridLayout_5.addWidget(self.specOpippFactor, 3, 1, 1, 4) - - - self.specOpCebChannel = QtGui.QCheckBox(self.tabopSpectra) - self.specOpCebChannel.setObjectName(_fromUtf8("specOpCebChannel")) - self.gridLayout_5.addWidget(self.specOpCebChannel, 4, 0, 1, 1) - - self.specOpComChannel = QtGui.QComboBox(self.tabopSpectra) - self.specOpComChannel.setObjectName(_fromUtf8("specOpComChannel")) - self.specOpComChannel.addItem(_fromUtf8("")) - self.specOpComChannel.addItem(_fromUtf8("")) - self.gridLayout_5.addWidget(self.specOpComChannel, 4, 1, 1, 2) - - self.specOpChannel = QtGui.QLineEdit(self.tabopSpectra) - self.specOpChannel.setObjectName(_fromUtf8("specOpChannel")) - self.gridLayout_5.addWidget(self.specOpChannel, 4, 3, 1, 2) - - - self.specOpCebCrossSpectra = QtGui.QCheckBox(self.tabopSpectra) - self.specOpCebCrossSpectra.setObjectName(_fromUtf8("specOpCebCrossSpectra")) - self.gridLayout_5.addWidget(self.specOpCebCrossSpectra, 5, 0, 1, 1) - - self.specOpComCrossSpectra = QtGui.QComboBox(self.tabopSpectra) - self.specOpComCrossSpectra.setObjectName(_fromUtf8("specOpComCrossSpectra")) - self.specOpComCrossSpectra.addItem(_fromUtf8("")) - self.gridLayout_5.addWidget(self.specOpComCrossSpectra, 5, 1, 1, 2) - - self.specOppairsList = QtGui.QLineEdit(self.tabopSpectra) - self.specOppairsList.setObjectName(_fromUtf8("specOppairsList")) - self.gridLayout_5.addWidget(self.specOppairsList, 5, 3, 1, 2) - - - self.specOpCebHeights = QtGui.QCheckBox(self.tabopSpectra) - self.specOpCebHeights.setObjectName(_fromUtf8("specOpCebHeights")) - self.gridLayout_5.addWidget(self.specOpCebHeights, 6, 0, 1, 1) - - self.specOpComHeights = QtGui.QComboBox(self.tabopSpectra) - self.specOpComHeights.setObjectName(_fromUtf8("specOpComHeights")) - self.specOpComHeights.addItem(_fromUtf8("")) - self.specOpComHeights.addItem(_fromUtf8("")) - self.gridLayout_5.addWidget(self.specOpComHeights, 6, 1, 1, 2) - - self.specOpHeights = QtGui.QLineEdit(self.tabopSpectra) - self.specOpHeights.setObjectName(_fromUtf8("specOpHeights")) - self.gridLayout_5.addWidget(self.specOpHeights, 6, 3, 1, 2) - - - self.specOpCebIncoherent = QtGui.QCheckBox(self.tabopSpectra) - self.specOpCebIncoherent.setObjectName(_fromUtf8("specOpCebIncoherent")) - self.gridLayout_5.addWidget(self.specOpCebIncoherent, 7, 0, 1, 1) - - self.specOpCobIncInt = QtGui.QComboBox(self.tabopSpectra) - self.specOpCobIncInt.setObjectName(_fromUtf8("specOpCobIncInt")) - self.specOpCobIncInt.addItem(_fromUtf8("")) - self.specOpCobIncInt.addItem(_fromUtf8("")) - self.gridLayout_5.addWidget(self.specOpCobIncInt, 7, 1, 1, 2) - - self.specOpIncoherent = QtGui.QLineEdit(self.tabopSpectra) - self.specOpIncoherent.setObjectName(_fromUtf8("specOpIncoherent")) - self.gridLayout_5.addWidget(self.specOpIncoherent, 7, 3, 1, 2) - - - self.specOpCebRemoveDC = QtGui.QCheckBox(self.tabopSpectra) - self.specOpCebRemoveDC.setObjectName(_fromUtf8("specOpCebRemoveDC")) - self.gridLayout_5.addWidget(self.specOpCebRemoveDC, 8, 0, 1, 1) - - self.specOpComRemoveDC = QtGui.QComboBox(self.tabopSpectra) - self.specOpComRemoveDC.setObjectName(_fromUtf8("specOpComRemoveDC")) - self.specOpComRemoveDC.addItem(_fromUtf8("")) - self.specOpComRemoveDC.addItem(_fromUtf8("")) - self.gridLayout_5.addWidget(self.specOpComRemoveDC, 8, 1, 1, 2) - - - self.specOpCebRemoveInt = QtGui.QCheckBox(self.tabopSpectra) - self.specOpCebRemoveInt.setObjectName(_fromUtf8("specOpCebRemoveInt")) - self.gridLayout_5.addWidget(self.specOpCebRemoveInt, 9, 0, 1, 1) - - - self.specOpCebgetNoise = QtGui.QCheckBox(self.tabopSpectra) - self.specOpCebgetNoise.setObjectName(_fromUtf8("specOpCebgetNoise")) - self.gridLayout_5.addWidget(self.specOpCebgetNoise, 10, 0, 1, 1) - - self.specOpgetNoise = QtGui.QLineEdit(self.tabopSpectra) - self.specOpgetNoise.setObjectName(_fromUtf8("specOpgetNoise")) - self.gridLayout_5.addWidget(self.specOpgetNoise, 10, 1, 1, 4) - -# spacerItem9 = QtGui.QSpacerItem(40, 20, QtGui.QSizePolicy.Expanding, QtGui.QSizePolicy.Minimum) -# self.gridLayout_5.addItem(spacerItem9, 12, 3, 1, 1) -# -# spacerItem10 = QtGui.QSpacerItem(40, 20, QtGui.QSizePolicy.Expanding, QtGui.QSizePolicy.Minimum) -# self.gridLayout_5.addItem(spacerItem10, 9, 3, 1, 1) -# -# spacerItem11 = QtGui.QSpacerItem(40, 20, QtGui.QSizePolicy.Expanding, QtGui.QSizePolicy.Minimum) -# self.gridLayout_5.addItem(spacerItem11, 7, 3, 1, 1) -# -# spacerItem12 = QtGui.QSpacerItem(40, 20, QtGui.QSizePolicy.Expanding, QtGui.QSizePolicy.Minimum) -# self.gridLayout_5.addItem(spacerItem12, 4, 3, 1, 1) -# -# spacerItem13 = QtGui.QSpacerItem(40, 20, QtGui.QSizePolicy.Expanding, QtGui.QSizePolicy.Minimum) -# self.gridLayout_5.addItem(spacerItem13, 15, 3, 1, 1) - - - self.tabWidgetSpectra.addTab(self.tabopSpectra, _fromUtf8("")) - - ################################################################ - ################################################################ - - self.tabgraphSpectra = QtGui.QWidget() - self.tabgraphSpectra.setObjectName(_fromUtf8("tabgraphSpectra")) - self.gridLayout_9 = QtGui.QGridLayout(self.tabgraphSpectra) - self.gridLayout_9.setObjectName(_fromUtf8("gridLayout_9")) - -# spacerItem14 = QtGui.QSpacerItem(20, 40, QtGui.QSizePolicy.Minimum, QtGui.QSizePolicy.Expanding) -# self.gridLayout_9.addItem(spacerItem14, 14, 2, 1, 1) - xi = 0 - - self.label_24 = QtGui.QLabel(self.tabgraphSpectra) - self.label_24.setObjectName(_fromUtf8("label_24")) - self.gridLayout_9.addWidget(self.label_24, xi, 0, 1, 1) - - self.specGraphPath = QtGui.QLineEdit(self.tabgraphSpectra) - self.specGraphPath.setObjectName(_fromUtf8("specGraphPath")) - self.gridLayout_9.addWidget(self.specGraphPath, xi, 1, 1, 6) - - self.specGraphToolPath = QtGui.QToolButton(self.tabgraphSpectra) - self.specGraphToolPath.setObjectName(_fromUtf8("specGraphToolPath")) - self.gridLayout_9.addWidget(self.specGraphToolPath, xi, 7, 1, 1) - - self.label_25 = QtGui.QLabel(self.tabgraphSpectra) - self.label_25.setObjectName(_fromUtf8("label_25")) - self.gridLayout_9.addWidget(self.label_25, xi+1, 0, 1, 1) - self.specGraphPrefix = QtGui.QLineEdit(self.tabgraphSpectra) - self.specGraphPrefix.setObjectName(_fromUtf8("specGraphPrefix")) - self.gridLayout_9.addWidget(self.specGraphPrefix, xi+1, 1, 1, 7) - - xi = 2 - - self.label_40 = QtGui.QLabel(self.tabgraphSpectra) - self.label_40.setObjectName(_fromUtf8("label_40")) - self.gridLayout_9.addWidget(self.label_40, xi+1, 0, 1, 1) - self.label_41 = QtGui.QLabel(self.tabgraphSpectra) - self.label_41.setObjectName(_fromUtf8("label_41")) - self.gridLayout_9.addWidget(self.label_41, xi+2, 0, 1, 1) - self.label_42 = QtGui.QLabel(self.tabgraphSpectra) - self.label_42.setObjectName(_fromUtf8("label_42")) - self.gridLayout_9.addWidget(self.label_42, xi+3, 0, 1, 1) - self.label_44 = QtGui.QLabel(self.tabgraphSpectra) - self.label_44.setObjectName(_fromUtf8("label_44")) - self.gridLayout_9.addWidget(self.label_44, xi+4, 0, 1, 1) - self.label_46 = QtGui.QLabel(self.tabgraphSpectra) - self.label_46.setObjectName(_fromUtf8("label_46")) - self.gridLayout_9.addWidget(self.label_46, xi+5, 0, 1, 1) - self.label_45 = QtGui.QLabel(self.tabgraphSpectra) - self.label_45.setObjectName(_fromUtf8("label_45")) - self.gridLayout_9.addWidget(self.label_45, xi+6, 0, 1, 1) - - self.label_43 = QtGui.QLabel(self.tabgraphSpectra) - self.label_43.setObjectName(_fromUtf8("label_43")) - self.gridLayout_9.addWidget(self.label_43, xi, 3, 1, 1) - self.specGraphCebSpectraplot = QtGui.QCheckBox(self.tabgraphSpectra) - self.specGraphCebSpectraplot.setText(_fromUtf8("")) - self.specGraphCebSpectraplot.setObjectName(_fromUtf8("specGraphCebSpectraplot")) - self.gridLayout_9.addWidget(self.specGraphCebSpectraplot, xi+1, 3, 1, 1) - self.specGraphCebCrossSpectraplot = QtGui.QCheckBox(self.tabgraphSpectra) - self.specGraphCebCrossSpectraplot.setText(_fromUtf8("")) - self.specGraphCebCrossSpectraplot.setObjectName(_fromUtf8("specGraphCebCrossSpectraplot")) - self.gridLayout_9.addWidget(self.specGraphCebCrossSpectraplot, xi+2, 3, 1, 1) - self.specGraphCebRTIplot = QtGui.QCheckBox(self.tabgraphSpectra) - self.specGraphCebRTIplot.setText(_fromUtf8("")) - self.specGraphCebRTIplot.setObjectName(_fromUtf8("specGraphCebRTIplot")) - self.gridLayout_9.addWidget(self.specGraphCebRTIplot, xi+3, 3, 1, 1) - self.specGraphCebCoherencmap = QtGui.QCheckBox(self.tabgraphSpectra) - self.specGraphCebCoherencmap.setText(_fromUtf8("")) - self.specGraphCebCoherencmap.setObjectName(_fromUtf8("specGraphCebCoherencmap")) - self.gridLayout_9.addWidget(self.specGraphCebCoherencmap, xi+4, 3, 1, 1) - self.specGraphPowerprofile = QtGui.QCheckBox(self.tabgraphSpectra) - self.specGraphPowerprofile.setText(_fromUtf8("")) - self.specGraphPowerprofile.setObjectName(_fromUtf8("specGraphPowerprofile")) - self.gridLayout_9.addWidget(self.specGraphPowerprofile, xi+5, 3, 1, 1) - self.specGraphCebRTInoise = QtGui.QCheckBox(self.tabgraphSpectra) - self.specGraphCebRTInoise.setText(_fromUtf8("")) - self.specGraphCebRTInoise.setObjectName(_fromUtf8("specGraphCebRTInoise")) - self.gridLayout_9.addWidget(self.specGraphCebRTInoise, xi+6, 3, 1, 1) - -# spacerItem18 = QtGui.QSpacerItem(40, 20, QtGui.QSizePolicy.Expanding, QtGui.QSizePolicy.Minimum) -# self.gridLayout_9.addItem(spacerItem18, 4, 3, 1, 1) - - self.label_47 = QtGui.QLabel(self.tabgraphSpectra) - self.label_47.setObjectName(_fromUtf8("label_47")) - self.gridLayout_9.addWidget(self.label_47, xi, 5, 1, 1) - self.specGraphSaveSpectra = QtGui.QCheckBox(self.tabgraphSpectra) - self.specGraphSaveSpectra.setText(_fromUtf8("")) - self.specGraphSaveSpectra.setObjectName(_fromUtf8("specGraphSaveSpectra")) - self.gridLayout_9.addWidget(self.specGraphSaveSpectra, xi+1, 5, 1, 1) - self.specGraphSaveCross = QtGui.QCheckBox(self.tabgraphSpectra) - self.specGraphSaveCross.setText(_fromUtf8("")) - self.specGraphSaveCross.setObjectName(_fromUtf8("specGraphSaveCross")) - self.gridLayout_9.addWidget(self.specGraphSaveCross, xi+2, 5, 1, 1) - self.specGraphSaveRTIplot = QtGui.QCheckBox(self.tabgraphSpectra) - self.specGraphSaveRTIplot.setText(_fromUtf8("")) - self.specGraphSaveRTIplot.setObjectName(_fromUtf8("specGraphSaveRTIplot")) - self.gridLayout_9.addWidget(self.specGraphSaveRTIplot, xi+3, 5, 1, 1) - self.specGraphSaveCoherencemap = QtGui.QCheckBox(self.tabgraphSpectra) - self.specGraphSaveCoherencemap.setText(_fromUtf8("")) - self.specGraphSaveCoherencemap.setObjectName(_fromUtf8("specGraphSaveCoherencemap")) - self.gridLayout_9.addWidget(self.specGraphSaveCoherencemap, xi+4, 5, 1, 1) - self.specGraphSavePowerprofile = QtGui.QCheckBox(self.tabgraphSpectra) - self.specGraphSavePowerprofile.setText(_fromUtf8("")) - self.specGraphSavePowerprofile.setObjectName(_fromUtf8("specGraphSavePowerprofile")) - self.gridLayout_9.addWidget(self.specGraphSavePowerprofile, xi+5, 5, 1, 1) - self.specGraphSaveRTInoise = QtGui.QCheckBox(self.tabgraphSpectra) - self.specGraphSaveRTInoise.setText(_fromUtf8("")) - self.specGraphSaveRTInoise.setObjectName(_fromUtf8("specGraphSaveRTInoise")) - self.gridLayout_9.addWidget(self.specGraphSaveRTInoise, xi+6, 5, 1, 1) - - self.label_19 = QtGui.QLabel(self.tabgraphSpectra) - self.label_19.setObjectName(_fromUtf8("label_19")) - self.gridLayout_9.addWidget(self.label_19, xi, 7, 1, 1) - self.specGraphftpSpectra = QtGui.QCheckBox(self.tabgraphSpectra) - self.specGraphftpSpectra.setText(_fromUtf8("")) - self.specGraphftpSpectra.setObjectName(_fromUtf8("specGraphftpSpectra")) - self.gridLayout_9.addWidget(self.specGraphftpSpectra, xi+1, 7, 1, 1) - self.specGraphftpCross = QtGui.QCheckBox(self.tabgraphSpectra) - self.specGraphftpCross.setText(_fromUtf8("")) - self.specGraphftpCross.setObjectName(_fromUtf8("specGraphftpCross")) - self.gridLayout_9.addWidget(self.specGraphftpCross, xi+2, 7, 1, 1) - self.specGraphftpRTIplot = QtGui.QCheckBox(self.tabgraphSpectra) - self.specGraphftpRTIplot.setText(_fromUtf8("")) - self.specGraphftpRTIplot.setObjectName(_fromUtf8("specGraphftpRTIplot")) - self.gridLayout_9.addWidget(self.specGraphftpRTIplot, xi+3, 7, 1, 1) - self.specGraphftpCoherencemap = QtGui.QCheckBox(self.tabgraphSpectra) - self.specGraphftpCoherencemap.setText(_fromUtf8("")) - self.specGraphftpCoherencemap.setObjectName(_fromUtf8("specGraphftpCoherencemap")) - self.gridLayout_9.addWidget(self.specGraphftpCoherencemap, xi+4, 7, 1, 1) - self.specGraphftpPowerprofile = QtGui.QCheckBox(self.tabgraphSpectra) - self.specGraphftpPowerprofile.setText(_fromUtf8("")) - self.specGraphftpPowerprofile.setObjectName(_fromUtf8("specGraphftpPowerprofile")) - self.gridLayout_9.addWidget(self.specGraphftpPowerprofile, xi+5, 7, 1, 1) - self.specGraphftpRTInoise = QtGui.QCheckBox(self.tabgraphSpectra) - self.specGraphftpRTInoise.setText(_fromUtf8("")) - self.specGraphftpRTInoise.setObjectName(_fromUtf8("specGraphftpRTInoise")) - self.gridLayout_9.addWidget(self.specGraphftpRTInoise, xi+6, 7, 1, 1) - -# spacerItem19 = QtGui.QSpacerItem(39, 15, QtGui.QSizePolicy.Expanding, QtGui.QSizePolicy.Minimum) -# self.gridLayout_9.addItem(spacerItem19, 27, 4, 1, 1) - - xi = 11 - - self.label_22 = QtGui.QLabel(self.tabgraphSpectra) - self.label_22.setObjectName(_fromUtf8("label_22")) - self.gridLayout_9.addWidget(self.label_22, xi, 0, 1, 1) - self.specGgraphFreq = QtGui.QLineEdit(self.tabgraphSpectra) - self.specGgraphFreq.setObjectName(_fromUtf8("specGgraphFreq")) - self.gridLayout_9.addWidget(self.specGgraphFreq, xi, 2, 1, 2) - - self.label_16 = QtGui.QLabel(self.tabgraphSpectra) - self.label_16.setObjectName(_fromUtf8("label_16")) - self.gridLayout_9.addWidget(self.label_16, xi+1, 0, 1, 1) - self.specGgraphHeight = QtGui.QLineEdit(self.tabgraphSpectra) - self.specGgraphHeight.setObjectName(_fromUtf8("specGgraphHeight")) - self.gridLayout_9.addWidget(self.specGgraphHeight, xi+1, 2, 1, 2) - - self.label_17 = QtGui.QLabel(self.tabgraphSpectra) - self.label_17.setObjectName(_fromUtf8("label_17")) - self.gridLayout_9.addWidget(self.label_17, xi+2, 0, 1, 1) - self.specGgraphDbsrange = QtGui.QLineEdit(self.tabgraphSpectra) - self.specGgraphDbsrange.setObjectName(_fromUtf8("specGgraphDbsrange")) - self.gridLayout_9.addWidget(self.specGgraphDbsrange, xi+2, 2, 1, 2) - - self.specGraphTminTmaxLabel = QtGui.QLabel(self.tabgraphSpectra) - self.specGraphTminTmaxLabel.setObjectName(_fromUtf8("specGraphTminTmaxLabel")) - self.gridLayout_9.addWidget(self.specGraphTminTmaxLabel, xi+3, 0, 1, 2) - self.specGgraphTminTmax = QtGui.QLineEdit(self.tabgraphSpectra) - self.specGgraphTminTmax.setObjectName(_fromUtf8("specGgraphTminTmax")) - self.gridLayout_9.addWidget(self.specGgraphTminTmax, xi+3, 2, 1, 2) - - self.label_48 = QtGui.QLabel(self.tabgraphSpectra) - self.label_48.setObjectName(_fromUtf8("label_48")) - self.gridLayout_9.addWidget(self.label_48, xi+4, 0, 1, 2) - self.specGgraphTimeRange = QtGui.QLineEdit(self.tabgraphSpectra) - self.specGgraphTimeRange.setObjectName(_fromUtf8("specGgraphTimeRange")) - self.gridLayout_9.addWidget(self.specGgraphTimeRange, xi+4, 2, 1, 2) - - self.specGraphMagLabel = QtGui.QLabel(self.tabgraphSpectra) - self.specGraphMagLabel.setObjectName(_fromUtf8("specGraphMagLabel")) - self.gridLayout_9.addWidget(self.specGraphMagLabel, xi, 4, 1, 2) - self.specGgraphmagnitud = QtGui.QLineEdit(self.tabgraphSpectra) - self.specGgraphmagnitud.setObjectName(_fromUtf8("specGgraphmagnitud")) - self.gridLayout_9.addWidget(self.specGgraphmagnitud, xi, 6, 1, 2) - - self.specGraphPhaseLabel = QtGui.QLabel(self.tabgraphSpectra) - self.specGraphPhaseLabel.setObjectName(_fromUtf8("specGraphPhaseLabel")) - self.gridLayout_9.addWidget(self.specGraphPhaseLabel, xi+1, 4, 1, 2) - self.specGgraphPhase = QtGui.QLineEdit(self.tabgraphSpectra) - self.specGgraphPhase.setObjectName(_fromUtf8("specGgraphPhase")) - self.gridLayout_9.addWidget(self.specGgraphPhase, xi+1, 6, 1, 2) - - self.label_6 = QtGui.QLabel(self.tabgraphSpectra) - self.label_6.setObjectName(_fromUtf8("label_6")) - self.gridLayout_9.addWidget(self.label_6, xi+2, 4, 1, 1) - self.specGgraphChannelList = QtGui.QLineEdit(self.tabgraphSpectra) - self.specGgraphChannelList.setObjectName(_fromUtf8("specGgraphChannelList")) - self.gridLayout_9.addWidget(self.specGgraphChannelList, xi+2, 6, 1, 2) - - self.label_29 = QtGui.QLabel(self.tabgraphSpectra) - self.label_29.setObjectName(_fromUtf8("label_29")) - self.gridLayout_9.addWidget(self.label_29, xi+3, 4, 1, 2) - self.specGgraphftpratio = QtGui.QLineEdit(self.tabgraphSpectra) - self.specGgraphftpratio.setObjectName(_fromUtf8("specGgraphftpratio")) - self.gridLayout_9.addWidget(self.specGgraphftpratio, xi+3, 6, 1, 2) - -# spacerItem15 = QtGui.QSpacerItem(28, 15, QtGui.QSizePolicy.Expanding, QtGui.QSizePolicy.Minimum) -# self.gridLayout_9.addItem(spacerItem15, 27, 6, 1, 2) -# spacerItem16 = QtGui.QSpacerItem(20, 40, QtGui.QSizePolicy.Minimum, QtGui.QSizePolicy.Expanding) -# self.gridLayout_9.addItem(spacerItem16, 3, 5, 1, 1) -# spacerItem17 = QtGui.QSpacerItem(49, 15, QtGui.QSizePolicy.Expanding, QtGui.QSizePolicy.Minimum) -# self.gridLayout_9.addItem(spacerItem17, 27, 0, 1, 1) - - - - self.tabWidgetSpectra.addTab(self.tabgraphSpectra, _fromUtf8("")) - self.taboutputSpectra = QtGui.QWidget() - self.taboutputSpectra.setObjectName(_fromUtf8("taboutputSpectra")) - self.gridLayout_11 = QtGui.QGridLayout(self.taboutputSpectra) - self.gridLayout_11.setObjectName(_fromUtf8("gridLayout_11")) - self.label_39 = QtGui.QLabel(self.taboutputSpectra) - self.label_39.setObjectName(_fromUtf8("label_39")) - self.gridLayout_11.addWidget(self.label_39, 0, 0, 1, 1) - self.specOutputComData = QtGui.QComboBox(self.taboutputSpectra) - self.specOutputComData.setObjectName(_fromUtf8("specOutputComData")) - self.specOutputComData.addItem(_fromUtf8("")) - self.gridLayout_11.addWidget(self.specOutputComData, 0, 2, 1, 2) - self.label_34 = QtGui.QLabel(self.taboutputSpectra) - self.label_34.setObjectName(_fromUtf8("label_34")) - self.gridLayout_11.addWidget(self.label_34, 1, 0, 1, 1) - self.specOutputPath = QtGui.QLineEdit(self.taboutputSpectra) - self.specOutputPath.setObjectName(_fromUtf8("specOutputPath")) - self.gridLayout_11.addWidget(self.specOutputPath, 1, 2, 1, 1) - spacerItem20 = QtGui.QSpacerItem(20, 40, QtGui.QSizePolicy.Minimum, QtGui.QSizePolicy.Expanding) - self.gridLayout_11.addItem(spacerItem20, 4, 2, 1, 1) - self.specOutputToolPath = QtGui.QToolButton(self.taboutputSpectra) - self.specOutputToolPath.setObjectName(_fromUtf8("specOutputToolPath")) - self.gridLayout_11.addWidget(self.specOutputToolPath, 1, 3, 1, 1) - self.specOutputblocksperfile = QtGui.QLineEdit(self.taboutputSpectra) - self.specOutputblocksperfile.setObjectName(_fromUtf8("specOutputblocksperfile")) - self.gridLayout_11.addWidget(self.specOutputblocksperfile, 2, 2, 1, 1) - self.label_9 = QtGui.QLabel(self.taboutputSpectra) - self.label_9.setObjectName(_fromUtf8("label_9")) - self.gridLayout_11.addWidget(self.label_9, 2, 0, 1, 2) - - self.tabWidgetSpectra.addTab(self.taboutputSpectra, _fromUtf8("")) - self.gridLayout_7.addWidget(self.tabWidgetSpectra, 0, 1, 1, 1) - - self.tabWidgetProject.addTab(self.tabSpectra, _fromUtf8("")) - - self.tabWidgetSpectra.setCurrentIndex(0) - - def retranslateUi(self): - - self.specOpOk.setText(_translate("MainWindow", "Ok", None)) - self.specGraphClear.setText(_translate("MainWindow", "Clear", None)) - self.specOpCebCrossSpectra.setText(_translate("MainWindow", "Select Cross Spectra:", None)) - self.specOpComChannel.setItemText(0, _translate("MainWindow", "Value", None)) - self.specOpComChannel.setItemText(1, _translate("MainWindow", "Index", None)) - self.specOpComHeights.setItemText(0, _translate("MainWindow", "Value", None)) - self.specOpComHeights.setItemText(1, _translate("MainWindow", "Index", None)) - self.specOpCebRemoveDC.setText(_translate("MainWindow", "Remove DC:", None)) - self.specOpCebHeights.setText(_translate("MainWindow", "Select Heights:", None)) - self.specOpCebChannel.setText(_translate("MainWindow", "Select Channel:", None)) - - self.specOpComCrossSpectra.setItemText(0, _translate("MainWindow", "x-y pairs", None)) - - self.specLabnFFTPoints.setText(_translate("MainWindow", "Number of FFT points:", None)) - self.specOpCebIncoherent.setText(_translate("MainWindow", "Incoherent Integration:", None)) - self.specOpCobIncInt.setItemText(0, _translate("MainWindow", "Time Interval", None)) - self.specOpCobIncInt.setItemText(1, _translate("MainWindow", "Number of Profiles", None)) - self.specOpCebRadarfrequency.setText(_translate("MainWindow", "Radar frequency (MHz):", None)) - self.specLabProfiles.setText(_translate("MainWindow", "Number of Profiles", None)) - self.specOpCebRemoveInt.setText(_translate("MainWindow", "Remove Interference:", None)) - self.specLabippFactor.setText(_translate("MainWindow", "Ipp Factor:", None)) - self.specOpCebgetNoise.setText(_translate("MainWindow", "Set Noise area:", None)) - self.specOpComRemoveDC.setItemText(0, _translate("MainWindow", "Mode 1", None)) - self.specOpComRemoveDC.setItemText(1, _translate("MainWindow", "Mode 2", None)) - self.tabWidgetSpectra.setTabText(self.tabWidgetSpectra.indexOf(self.tabopSpectra), _translate("MainWindow", "Operation", None)) - - self.label_44.setText(_translate("MainWindow", "Coherence Map:", None)) - self.specGraphTminTmaxLabel.setText(_translate("MainWindow", "Time range (hours):", None)) - self.label_25.setText(_translate("MainWindow", "Prefix:", None)) - self.label_42.setText(_translate("MainWindow", "RTI Plot:", None)) - self.label_16.setText(_translate("MainWindow", "Height range:", None)) - self.label_17.setText(_translate("MainWindow", "dB range:", None)) - self.specGraphMagLabel.setText(_translate("MainWindow", "Coh. Magnitud:", None)) - self.label_24.setText(_translate("MainWindow", "Path:", None)) - self.label_46.setText(_translate("MainWindow", "Power Profile:", None)) - self.label_22.setText(_translate("MainWindow", "Freq/Vel range:", None)) - self.label_41.setText(_translate("MainWindow", "Cross Spectra Plot:", None)) - self.specGraphToolPath.setText(_translate("MainWindow", "...", None)) - self.label_6.setText(_translate("MainWindow", "Channel List:", None)) - self.label_40.setText(_translate("MainWindow", "Spectra Plot:", None)) - self.label_43.setText(_translate("MainWindow", "Show:", None)) - self.label_29.setText(_translate("MainWindow", "Writing Period:", None)) - self.label_47.setText(_translate("MainWindow", "Save:", None)) - self.label_19.setText(_translate("MainWindow", "Ftp:", None)) - self.label_45.setText(_translate("MainWindow", "Noise:", None)) - self.label_48.setText(_translate("MainWindow", "Time window (seconds):", None)) - self.specGraphPhaseLabel.setText(_translate("MainWindow", "Coh. Phase:", None)) -# self.label_48.hide() -# self.specGgraphTimeRange.hide() - self.tabWidgetSpectra.setTabText(self.tabWidgetSpectra.indexOf(self.tabgraphSpectra), _translate("MainWindow", "Graphics", None)) - - self.label_39.setText(_translate("MainWindow", "Type:", None)) - self.specOutputComData.setItemText(0, _translate("MainWindow", ".pdata", None)) - self.label_34.setText(_translate("MainWindow", "Path:", None)) - self.specOutputToolPath.setText(_translate("MainWindow", "...", None)) - self.label_9.setText(_translate("MainWindow", "Blocks per File: ", None)) - - self.tabWidgetSpectra.setTabText(self.tabWidgetSpectra.indexOf(self.taboutputSpectra), _translate("MainWindow", "Output", None)) - - self.tabWidgetProject.setTabText(self.tabWidgetProject.indexOf(self.tabSpectra), _translate("MainWindow", "Spectra", None)) - - self.__setToolTip() - - def __setToolTip(self): - - self.specOpnFFTpoints.setToolTip('Number of FFT points used in FFT function. Example: 128') - self.specOpProfiles.setToolTip('Number of data points used in FFT function. Example: 128') - self.specOpippFactor.setToolTip('This factor is multiplied to IPP value to get velocity and frequency range. Example: 4') - self.specOpIncoherent.setToolTip('Example: 10') - self.specOpgetNoise.setToolTip('Example:20,180,30,120 (minHei,maxHei,minVel,maxVel)') - - self.specOpChannel.setToolTip('Example: 0,1,2,3') - self.specOpHeights.setToolTip('Example: 90,180') - self.specOppairsList.setToolTip('Example: (0,1),(2,3)') - # tool tip gui specGraph - - self.specGgraphChannelList.setToolTip('Example: 0,3,4') - self.specGgraphFreq.setToolTip('Example: -20,20') - self.specGgraphHeight.setToolTip('Example: 100,400') - self.specGgraphDbsrange.setToolTip('Example: 30,170') - - self.specGraphPrefix.setToolTip('Example: EXPERIMENT_NAME') - - \ No newline at end of file diff --git a/schainpy/gui/viewer/windows/ui_spectra_heis.py b/schainpy/gui/viewer/windows/ui_spectra_heis.py deleted file mode 100644 index 7fcbcd3..0000000 --- a/schainpy/gui/viewer/windows/ui_spectra_heis.py +++ /dev/null @@ -1,252 +0,0 @@ -from PyQt4 import QtCore, QtGui - -try: - _fromUtf8 = QtCore.QString.fromUtf8 -except AttributeError: - def _fromUtf8(s): - return s - -try: - _encoding = QtGui.QApplication.UnicodeUTF8 - def _translate(context, text, disambig): - return QtGui.QApplication.translate(context, text, disambig, _encoding) -except AttributeError: - def _translate(context, text, disambig): - return QtGui.QApplication.translate(context, text, disambig) - -class Ui_SpectraHeisTab(object): - - def setupUi(self): - - self.tabSpectraHeis = QtGui.QWidget() - self.tabSpectraHeis.setObjectName(_fromUtf8("tabSpectraHeis")) - self.gridLayout_23 = QtGui.QGridLayout(self.tabSpectraHeis) - self.gridLayout_23.setObjectName(_fromUtf8("gridLayout_23")) - self.frame_6 = QtGui.QFrame(self.tabSpectraHeis) - self.frame_6.setFrameShape(QtGui.QFrame.StyledPanel) - self.frame_6.setFrameShadow(QtGui.QFrame.Raised) - self.frame_6.setObjectName(_fromUtf8("frame_6")) - self.gridLayout_22 = QtGui.QGridLayout(self.frame_6) - self.gridLayout_22.setObjectName(_fromUtf8("gridLayout_22")) - self.specHeisGraphClear = QtGui.QPushButton(self.frame_6) - self.specHeisGraphClear.setObjectName(_fromUtf8("specHeisGraphClear")) - self.gridLayout_22.addWidget(self.specHeisGraphClear, 0, 1, 1, 1) - self.specHeisOpOk = QtGui.QPushButton(self.frame_6) - self.specHeisOpOk.setObjectName(_fromUtf8("specHeisOpOk")) - self.gridLayout_22.addWidget(self.specHeisOpOk, 0, 0, 1, 1) - self.gridLayout_23.addWidget(self.frame_6, 1, 0, 1, 1) - self.tabWidgetSpectraHeis = QtGui.QTabWidget(self.tabSpectraHeis) - self.tabWidgetSpectraHeis.setObjectName(_fromUtf8("tabWidgetSpectraHeis")) - self.tabopSpectraHeis = QtGui.QWidget() - self.tabopSpectraHeis.setObjectName(_fromUtf8("tabopSpectraHeis")) - self.gridLayout_21 = QtGui.QGridLayout(self.tabopSpectraHeis) - self.gridLayout_21.setObjectName(_fromUtf8("gridLayout_21")) - spacerItem21 = QtGui.QSpacerItem(40, 20, QtGui.QSizePolicy.Expanding, QtGui.QSizePolicy.Minimum) - self.gridLayout_21.addItem(spacerItem21, 0, 1, 1, 1) - self.specHeisOpCobIncInt = QtGui.QComboBox(self.tabopSpectraHeis) - self.specHeisOpCobIncInt.setObjectName(_fromUtf8("specHeisOpCobIncInt")) - self.specHeisOpCobIncInt.addItem(_fromUtf8("")) - self.gridLayout_21.addWidget(self.specHeisOpCobIncInt, 1, 0, 1, 1) - self.specHeisOpCebIncoherent = QtGui.QCheckBox(self.tabopSpectraHeis) - self.specHeisOpCebIncoherent.setObjectName(_fromUtf8("specHeisOpCebIncoherent")) - self.gridLayout_21.addWidget(self.specHeisOpCebIncoherent, 0, 0, 1, 1) - self.specHeisOpIncoherent = QtGui.QLineEdit(self.tabopSpectraHeis) - self.specHeisOpIncoherent.setObjectName(_fromUtf8("specHeisOpIncoherent")) - self.gridLayout_21.addWidget(self.specHeisOpIncoherent, 1, 1, 1, 1) - spacerItem22 = QtGui.QSpacerItem(20, 40, QtGui.QSizePolicy.Minimum, QtGui.QSizePolicy.Expanding) - self.gridLayout_21.addItem(spacerItem22, 2, 0, 1, 1) - self.tabWidgetSpectraHeis.addTab(self.tabopSpectraHeis, _fromUtf8("")) - self.tabgraphSpectraHeis = QtGui.QWidget() - self.tabgraphSpectraHeis.setObjectName(_fromUtf8("tabgraphSpectraHeis")) - self.gridLayout_20 = QtGui.QGridLayout(self.tabgraphSpectraHeis) - self.gridLayout_20.setObjectName(_fromUtf8("gridLayout_20")) - self.label_54 = QtGui.QLabel(self.tabgraphSpectraHeis) - self.label_54.setObjectName(_fromUtf8("label_54")) - self.gridLayout_20.addWidget(self.label_54, 1, 0, 1, 1) - self.specHeisGraphToolPath = QtGui.QToolButton(self.tabgraphSpectraHeis) - self.specHeisGraphToolPath.setObjectName(_fromUtf8("specHeisGraphToolPath")) - self.gridLayout_20.addWidget(self.specHeisGraphToolPath, 0, 6, 1, 1) - self.specHeisGraphCebRTIplot = QtGui.QCheckBox(self.tabgraphSpectraHeis) - self.specHeisGraphCebRTIplot.setText(_fromUtf8("")) - self.specHeisGraphCebRTIplot.setObjectName(_fromUtf8("specHeisGraphCebRTIplot")) - self.gridLayout_20.addWidget(self.specHeisGraphCebRTIplot, 4, 2, 1, 1) - self.label_62 = QtGui.QLabel(self.tabgraphSpectraHeis) - self.label_62.setObjectName(_fromUtf8("label_62")) - self.gridLayout_20.addWidget(self.label_62, 7, 0, 1, 1) - self.label_63 = QtGui.QLabel(self.tabgraphSpectraHeis) - self.label_63.setObjectName(_fromUtf8("label_63")) - self.gridLayout_20.addWidget(self.label_63, 8, 0, 1, 1) - self.label_64 = QtGui.QLabel(self.tabgraphSpectraHeis) - self.label_64.setObjectName(_fromUtf8("label_64")) - self.gridLayout_20.addWidget(self.label_64, 9, 0, 1, 1) - self.label_65 = QtGui.QLabel(self.tabgraphSpectraHeis) - self.label_65.setObjectName(_fromUtf8("label_65")) - self.gridLayout_20.addWidget(self.label_65, 10, 0, 1, 1) - spacerItem23 = QtGui.QSpacerItem(134, 20, QtGui.QSizePolicy.Expanding, QtGui.QSizePolicy.Minimum) - self.gridLayout_20.addItem(spacerItem23, 11, 0, 1, 2) - self.specHeisGgraphftpratio = QtGui.QLineEdit(self.tabgraphSpectraHeis) - self.specHeisGgraphftpratio.setObjectName(_fromUtf8("specHeisGgraphftpratio")) - self.gridLayout_20.addWidget(self.specHeisGgraphftpratio, 10, 1, 1, 6) - self.specHeisGraphftpRTIplot = QtGui.QCheckBox(self.tabgraphSpectraHeis) - self.specHeisGraphftpRTIplot.setText(_fromUtf8("")) - self.specHeisGraphftpRTIplot.setObjectName(_fromUtf8("specHeisGraphftpRTIplot")) - self.gridLayout_20.addWidget(self.specHeisGraphftpRTIplot, 4, 6, 1, 1) - self.specHeisGgraphTminTmax = QtGui.QLineEdit(self.tabgraphSpectraHeis) - self.specHeisGgraphTminTmax.setObjectName(_fromUtf8("specHeisGgraphTminTmax")) - self.gridLayout_20.addWidget(self.specHeisGgraphTminTmax, 8, 1, 1, 6) - self.label_60 = QtGui.QLabel(self.tabgraphSpectraHeis) - self.label_60.setObjectName(_fromUtf8("label_60")) - self.gridLayout_20.addWidget(self.label_60, 5, 0, 1, 1) - self.label_61 = QtGui.QLabel(self.tabgraphSpectraHeis) - self.label_61.setObjectName(_fromUtf8("label_61")) - self.gridLayout_20.addWidget(self.label_61, 6, 0, 1, 1) - self.specHeisGraphPrefix = QtGui.QLineEdit(self.tabgraphSpectraHeis) - self.specHeisGraphPrefix.setObjectName(_fromUtf8("specHeisGraphPrefix")) - self.gridLayout_20.addWidget(self.specHeisGraphPrefix, 1, 1, 1, 6) - self.label_56 = QtGui.QLabel(self.tabgraphSpectraHeis) - self.label_56.setObjectName(_fromUtf8("label_56")) - self.gridLayout_20.addWidget(self.label_56, 2, 4, 1, 1) - self.label_57 = QtGui.QLabel(self.tabgraphSpectraHeis) - self.label_57.setObjectName(_fromUtf8("label_57")) - self.gridLayout_20.addWidget(self.label_57, 2, 6, 1, 1) - self.label_58 = QtGui.QLabel(self.tabgraphSpectraHeis) - self.label_58.setObjectName(_fromUtf8("label_58")) - self.gridLayout_20.addWidget(self.label_58, 3, 0, 1, 1) - self.specHeisGraphCebSpectraplot = QtGui.QCheckBox(self.tabgraphSpectraHeis) - self.specHeisGraphCebSpectraplot.setText(_fromUtf8("")) - self.specHeisGraphCebSpectraplot.setObjectName(_fromUtf8("specHeisGraphCebSpectraplot")) - self.gridLayout_20.addWidget(self.specHeisGraphCebSpectraplot, 3, 2, 1, 1) - self.specHeisGgraphYminYmax = QtGui.QLineEdit(self.tabgraphSpectraHeis) - self.specHeisGgraphYminYmax.setObjectName(_fromUtf8("specHeisGgraphYminYmax")) - self.gridLayout_20.addWidget(self.specHeisGgraphYminYmax, 7, 1, 1, 6) - self.label_53 = QtGui.QLabel(self.tabgraphSpectraHeis) - self.label_53.setObjectName(_fromUtf8("label_53")) - self.gridLayout_20.addWidget(self.label_53, 0, 0, 1, 1) - self.label_55 = QtGui.QLabel(self.tabgraphSpectraHeis) - self.label_55.setObjectName(_fromUtf8("label_55")) - self.gridLayout_20.addWidget(self.label_55, 2, 2, 1, 1) - self.specHeisGraphSaveRTIplot = QtGui.QCheckBox(self.tabgraphSpectraHeis) - self.specHeisGraphSaveRTIplot.setText(_fromUtf8("")) - self.specHeisGraphSaveRTIplot.setObjectName(_fromUtf8("specHeisGraphSaveRTIplot")) - self.gridLayout_20.addWidget(self.specHeisGraphSaveRTIplot, 4, 4, 1, 1) - spacerItem24 = QtGui.QSpacerItem(40, 20, QtGui.QSizePolicy.Expanding, QtGui.QSizePolicy.Minimum) - self.gridLayout_20.addItem(spacerItem24, 2, 3, 1, 1) - self.specHeisGgraphXminXmax = QtGui.QLineEdit(self.tabgraphSpectraHeis) - self.specHeisGgraphXminXmax.setObjectName(_fromUtf8("specHeisGgraphXminXmax")) - self.gridLayout_20.addWidget(self.specHeisGgraphXminXmax, 6, 1, 1, 6) - self.specHeisGgraphChannelList = QtGui.QLineEdit(self.tabgraphSpectraHeis) - self.specHeisGgraphChannelList.setObjectName(_fromUtf8("specHeisGgraphChannelList")) - self.gridLayout_20.addWidget(self.specHeisGgraphChannelList, 5, 1, 1, 6) - self.specHeisGgraphTimeRange = QtGui.QLineEdit(self.tabgraphSpectraHeis) - self.specHeisGgraphTimeRange.setObjectName(_fromUtf8("specHeisGgraphTimeRange")) - self.gridLayout_20.addWidget(self.specHeisGgraphTimeRange, 9, 1, 1, 6) - spacerItem25 = QtGui.QSpacerItem(106, 20, QtGui.QSizePolicy.Expanding, QtGui.QSizePolicy.Minimum) - self.gridLayout_20.addItem(spacerItem25, 11, 3, 1, 3) - self.specHeisGraphSaveSpectra = QtGui.QCheckBox(self.tabgraphSpectraHeis) - self.specHeisGraphSaveSpectra.setText(_fromUtf8("")) - self.specHeisGraphSaveSpectra.setObjectName(_fromUtf8("specHeisGraphSaveSpectra")) - self.gridLayout_20.addWidget(self.specHeisGraphSaveSpectra, 3, 4, 1, 1) - self.specHeisGraphftpSpectra = QtGui.QCheckBox(self.tabgraphSpectraHeis) - self.specHeisGraphftpSpectra.setText(_fromUtf8("")) - self.specHeisGraphftpSpectra.setObjectName(_fromUtf8("specHeisGraphftpSpectra")) - self.gridLayout_20.addWidget(self.specHeisGraphftpSpectra, 3, 6, 1, 1) - self.label_59 = QtGui.QLabel(self.tabgraphSpectraHeis) - self.label_59.setObjectName(_fromUtf8("label_59")) - self.gridLayout_20.addWidget(self.label_59, 4, 0, 1, 1) - spacerItem26 = QtGui.QSpacerItem(40, 20, QtGui.QSizePolicy.Expanding, QtGui.QSizePolicy.Minimum) - self.gridLayout_20.addItem(spacerItem26, 2, 5, 1, 1) - self.specHeisGraphPath = QtGui.QLineEdit(self.tabgraphSpectraHeis) - self.specHeisGraphPath.setObjectName(_fromUtf8("specHeisGraphPath")) - self.gridLayout_20.addWidget(self.specHeisGraphPath, 0, 1, 1, 5) - self.tabWidgetSpectraHeis.addTab(self.tabgraphSpectraHeis, _fromUtf8("")) - self.taboutputSpectraHeis = QtGui.QWidget() - self.taboutputSpectraHeis.setObjectName(_fromUtf8("taboutputSpectraHeis")) - self.gridLayout_19 = QtGui.QGridLayout(self.taboutputSpectraHeis) - self.gridLayout_19.setObjectName(_fromUtf8("gridLayout_19")) - self.label_67 = QtGui.QLabel(self.taboutputSpectraHeis) - self.label_67.setObjectName(_fromUtf8("label_67")) - self.gridLayout_19.addWidget(self.label_67, 1, 0, 1, 1) - self.label_68 = QtGui.QLabel(self.taboutputSpectraHeis) - self.label_68.setObjectName(_fromUtf8("label_68")) - self.gridLayout_19.addWidget(self.label_68, 2, 0, 1, 2) - self.label_66 = QtGui.QLabel(self.taboutputSpectraHeis) - self.label_66.setObjectName(_fromUtf8("label_66")) - self.gridLayout_19.addWidget(self.label_66, 0, 0, 1, 1) - spacerItem27 = QtGui.QSpacerItem(20, 40, QtGui.QSizePolicy.Minimum, QtGui.QSizePolicy.Expanding) - self.gridLayout_19.addItem(spacerItem27, 4, 0, 1, 1) - self.specHeisOutputToolPath = QtGui.QToolButton(self.taboutputSpectraHeis) - self.specHeisOutputToolPath.setObjectName(_fromUtf8("specHeisOutputToolPath")) - self.gridLayout_19.addWidget(self.specHeisOutputToolPath, 1, 4, 1, 1) - self.specHeisOutputPath = QtGui.QLineEdit(self.taboutputSpectraHeis) - self.specHeisOutputPath.setObjectName(_fromUtf8("specHeisOutputPath")) - self.gridLayout_19.addWidget(self.specHeisOutputPath, 1, 3, 1, 1) - self.specHeisOutputComdata = QtGui.QComboBox(self.taboutputSpectraHeis) - self.specHeisOutputComdata.setObjectName(_fromUtf8("specHeisOutputComdata")) - self.specHeisOutputComdata.addItem(_fromUtf8("")) - self.gridLayout_19.addWidget(self.specHeisOutputComdata, 0, 3, 1, 2) - self.label_69 = QtGui.QLabel(self.taboutputSpectraHeis) - self.label_69.setObjectName(_fromUtf8("label_69")) - self.gridLayout_19.addWidget(self.label_69, 3, 0, 1, 2) - self.specHeisOutputblocksperfile = QtGui.QLineEdit(self.taboutputSpectraHeis) - self.specHeisOutputblocksperfile.setObjectName(_fromUtf8("specHeisOutputblocksperfile")) - self.gridLayout_19.addWidget(self.specHeisOutputblocksperfile, 2, 3, 1, 1) - self.specHeisOutputMetada = QtGui.QLineEdit(self.taboutputSpectraHeis) - self.specHeisOutputMetada.setObjectName(_fromUtf8("specHeisOutputMetada")) - self.gridLayout_19.addWidget(self.specHeisOutputMetada, 3, 3, 1, 1) - self.specHeisOutputMetadaToolPath = QtGui.QToolButton(self.taboutputSpectraHeis) - self.specHeisOutputMetadaToolPath.setObjectName(_fromUtf8("specHeisOutputMetadaToolPath")) - self.gridLayout_19.addWidget(self.specHeisOutputMetadaToolPath, 3, 4, 1, 1) - self.tabWidgetSpectraHeis.addTab(self.taboutputSpectraHeis, _fromUtf8("")) - self.gridLayout_23.addWidget(self.tabWidgetSpectraHeis, 0, 0, 1, 1) - - self.tabWidgetProject.addTab(self.tabSpectraHeis, _fromUtf8("")) - - self.tabWidgetSpectraHeis.setCurrentIndex(0) - - def retranslateUi(self): - - self.specHeisGraphClear.setText(_translate("MainWindow", "Clear", None)) - self.specHeisOpOk.setText(_translate("MainWindow", "Ok", None)) - self.specHeisOpCobIncInt.setItemText(0, _translate("MainWindow", "Time Interval", None)) - self.specHeisOpCebIncoherent.setText(_translate("MainWindow", "Incoherent Intergration", None)) - - self.tabWidgetSpectraHeis.setTabText(self.tabWidgetSpectraHeis.indexOf(self.tabopSpectraHeis), _translate("MainWindow", "Operation", None)) - self.label_54.setText(_translate("MainWindow", "Prefix", None)) - self.specHeisGraphToolPath.setText(_translate("MainWindow", "...", None)) - self.label_62.setText(_translate("MainWindow", "Intensity range (dB)", None)) - self.label_63.setText(_translate("MainWindow", "Time range (hours):", None)) - self.label_64.setText(_translate("MainWindow", "Time interval:", None)) - self.label_65.setText(_translate("MainWindow", "Wr Period", None)) - self.label_60.setText(_translate("MainWindow", "Channel List:", None)) - self.label_61.setText(_translate("MainWindow", "Frequency range", None)) - self.label_56.setText(_translate("MainWindow", "Save", None)) - self.label_57.setText(_translate("MainWindow", "ftp", None)) - self.label_58.setText(_translate("MainWindow", "Spectra Plot", None)) - self.label_53.setText(_translate("MainWindow", "Path", None)) - self.label_55.setText(_translate("MainWindow", "Show", None)) - self.label_59.setText(_translate("MainWindow", "RTI Plot", None)) - - self.tabWidgetSpectraHeis.setTabText(self.tabWidgetSpectraHeis.indexOf(self.tabgraphSpectraHeis), _translate("MainWindow", "Graphics", None)) - self.label_67.setText(_translate("MainWindow", "Path:", None)) - self.label_68.setText(_translate("MainWindow", "Blocks per file:", None)) - self.label_66.setText(_translate("MainWindow", "Type:", None)) - - self.tabWidgetSpectraHeis.setTabText(self.tabWidgetSpectraHeis.indexOf(self.taboutputSpectraHeis), _translate("MainWindow", "Output", None)) - self.specHeisOutputToolPath.setText(_translate("MainWindow", "...", None)) - self.specHeisOutputComdata.setItemText(0, _translate("MainWindow", ".fits", None)) - self.label_69.setText(_translate("MainWindow", "Metadata file:", None)) - self.specHeisOutputMetadaToolPath.setText(_translate("MainWindow", "...", None)) - - self.tabWidgetProject.setTabText(self.tabWidgetProject.indexOf(self.tabSpectraHeis), _translate("MainWindow", "SpectraHeis", None)) - - self.__setToolTip() - - def __setToolTip(self): - - self.specHeisOpIncoherent.setToolTip('Example: 10') - - self.specHeisGgraphChannelList.setToolTip('Example: 0,2,3') - self.specHeisGgraphXminXmax.setToolTip('Example (Hz): -1000, 1000') - self.specHeisGgraphYminYmax.setToolTip('Example (dB): 5, 35') - self.specHeisGgraphTminTmax.setToolTip('Example (hours): 0, 24') - self.specHeisGgraphTimeRange.setToolTip('Example (hours): 8') \ No newline at end of file diff --git a/schainpy/gui/viewer/windows/ui_voltage.py b/schainpy/gui/viewer/windows/ui_voltage.py deleted file mode 100644 index 4536b0f..0000000 --- a/schainpy/gui/viewer/windows/ui_voltage.py +++ /dev/null @@ -1,414 +0,0 @@ - -from PyQt4 import QtCore, QtGui - -try: - _fromUtf8 = QtCore.QString.fromUtf8 -except AttributeError: - def _fromUtf8(s): - return s - -try: - _encoding = QtGui.QApplication.UnicodeUTF8 - def _translate(context, text, disambig): - return QtGui.QApplication.translate(context, text, disambig, _encoding) -except AttributeError: - def _translate(context, text, disambig): - return QtGui.QApplication.translate(context, text, disambig) - -class Ui_VoltageTab(object): - - def setupUi(self): - - self.tabVoltage = QtGui.QWidget() - self.tabVoltage.setObjectName(_fromUtf8("tabVoltage")) - - self.gridLayout_3 = QtGui.QGridLayout(self.tabVoltage) - self.gridLayout_3.setObjectName(_fromUtf8("gridLayout_3")) - - self.frame_4 = QtGui.QFrame(self.tabVoltage) - self.frame_4.setFrameShape(QtGui.QFrame.StyledPanel) - self.frame_4.setFrameShadow(QtGui.QFrame.Raised) - self.frame_4.setObjectName(_fromUtf8("frame_4")) - - self.gridLayout_17 = QtGui.QGridLayout(self.frame_4) - self.gridLayout_17.setObjectName(_fromUtf8("gridLayout_17")) - self.volOpOk = QtGui.QPushButton(self.frame_4) - self.volOpOk.setObjectName(_fromUtf8("volOpOk")) - self.gridLayout_17.addWidget(self.volOpOk, 0, 0, 1, 1) - self.volGraphClear = QtGui.QPushButton(self.frame_4) - self.volGraphClear.setObjectName(_fromUtf8("volGraphClear")) - self.gridLayout_17.addWidget(self.volGraphClear, 0, 1, 1, 1) - self.gridLayout_3.addWidget(self.frame_4, 1, 1, 1, 1) - - - self.tabWidgetVoltage = QtGui.QTabWidget(self.tabVoltage) - self.tabWidgetVoltage.setObjectName(_fromUtf8("tabWidgetVoltage")) - self.tabopVoltage = QtGui.QWidget() - self.tabopVoltage.setObjectName(_fromUtf8("tabopVoltage")) - self.gridLayout = QtGui.QGridLayout(self.tabopVoltage) - self.gridLayout.setObjectName(_fromUtf8("gridLayout")) - - self.volOpCebRadarfrequency = QtGui.QCheckBox(self.tabopVoltage) - self.volOpCebRadarfrequency.setObjectName(_fromUtf8("volOpCebRadarfrequency")) - self.gridLayout.addWidget(self.volOpCebRadarfrequency, 0, 0, 1, 1) - - self.volOpRadarfrequency = QtGui.QLineEdit(self.tabopVoltage) - self.volOpRadarfrequency.setObjectName(_fromUtf8("volOpRadarfrequency")) - self.gridLayout.addWidget(self.volOpRadarfrequency, 0, 1, 1, 4) - - self.volOpCebChannels = QtGui.QCheckBox(self.tabopVoltage) - self.volOpCebChannels.setObjectName(_fromUtf8("volOpCebChannels")) - self.gridLayout.addWidget(self.volOpCebChannels, 1, 0, 1, 1) - - self.volOpComChannels = QtGui.QComboBox(self.tabopVoltage) - self.volOpComChannels.setObjectName(_fromUtf8("volOpComChannels")) - self.volOpComChannels.addItem(_fromUtf8("")) - self.volOpComChannels.addItem(_fromUtf8("")) - self.gridLayout.addWidget(self.volOpComChannels, 1, 1, 1, 2) - - self.volOpChannel = QtGui.QLineEdit(self.tabopVoltage) - self.volOpChannel.setObjectName(_fromUtf8("volOpChannel")) - self.gridLayout.addWidget(self.volOpChannel, 1, 3, 1, 2) - - - self.volOpCebHeights = QtGui.QCheckBox(self.tabopVoltage) - self.volOpCebHeights.setObjectName(_fromUtf8("volOpCebHeights")) - self.gridLayout.addWidget(self.volOpCebHeights, 3, 0, 1, 1) - - self.volOpComHeights = QtGui.QComboBox(self.tabopVoltage) - self.volOpComHeights.setObjectName(_fromUtf8("volOpComHeights")) - self.volOpComHeights.addItem(_fromUtf8("")) - self.volOpComHeights.addItem(_fromUtf8("")) - self.gridLayout.addWidget(self.volOpComHeights, 3, 1, 1, 2) - - self.volOpHeights = QtGui.QLineEdit(self.tabopVoltage) - self.volOpHeights.setObjectName(_fromUtf8("volOpHeights")) - self.gridLayout.addWidget(self.volOpHeights, 3, 3, 1, 2) - - self.volOpCebSplitter = QtGui.QCheckBox(self.tabopVoltage) - self.volOpCebSplitter.setObjectName(_fromUtf8("volOpCebSplitter")) - self.gridLayout.addWidget(self.volOpCebSplitter, 5, 0, 1, 1) - - self.volOpSplitter = QtGui.QLineEdit(self.tabopVoltage) - self.volOpSplitter.setObjectName(_fromUtf8("volOpSplitter")) - self.gridLayout.addWidget(self.volOpSplitter, 5, 1, 1, 4) - - self.volOpCebProfile = QtGui.QCheckBox(self.tabopVoltage) - self.volOpCebProfile.setObjectName(_fromUtf8("volOpCebProfile")) - self.gridLayout.addWidget(self.volOpCebProfile, 6, 0, 1, 1) - - self.volOpComProfile = QtGui.QComboBox(self.tabopVoltage) - self.volOpComProfile.setObjectName(_fromUtf8("volOpComProfile")) - self.volOpComProfile.addItem(_fromUtf8("")) - self.volOpComProfile.addItem(_fromUtf8("")) - self.volOpComProfile.addItem(_fromUtf8("")) - self.gridLayout.addWidget(self.volOpComProfile, 6, 1, 1, 2) - - self.volOpProfile = QtGui.QLineEdit(self.tabopVoltage) - self.volOpProfile.setObjectName(_fromUtf8("volOpProfile")) - self.gridLayout.addWidget(self.volOpProfile, 6, 3, 1, 2) - - self.volOpCebCombiner = QtGui.QCheckBox(self.tabopVoltage) - self.volOpCebCombiner.setObjectName(_fromUtf8("volOpCebCombiner")) - self.gridLayout.addWidget(self.volOpCebCombiner, 7, 0, 1, 1) - - self.volOpCombiner = QtGui.QLineEdit(self.tabopVoltage) - self.volOpCombiner.setObjectName(_fromUtf8("volOpCombiner")) - self.gridLayout.addWidget(self.volOpCombiner, 7, 1, 1, 4) - - self.volOpCebFilter = QtGui.QCheckBox(self.tabopVoltage) - self.volOpCebFilter.setObjectName(_fromUtf8("volOpCebFilter")) - self.gridLayout.addWidget(self.volOpCebFilter, 8, 0, 1, 1) - - self.volOpFilter = QtGui.QLineEdit(self.tabopVoltage) - self.volOpFilter.setObjectName(_fromUtf8("volOpFilter")) - self.gridLayout.addWidget(self.volOpFilter, 8, 1, 1, 4) - -# spacerItem = QtGui.QSpacerItem(40, 20, QtGui.QSizePolicy.Expanding, QtGui.QSizePolicy.Minimum) -# self.gridLayout.addItem(spacerItem, 6, 4, 1, 1) -# spacerItem1 = QtGui.QSpacerItem(40, 20, QtGui.QSizePolicy.Expanding, QtGui.QSizePolicy.Minimum) -# self.gridLayout.addItem(spacerItem1, 8, 4, 1, 1) -# spacerItem2 = QtGui.QSpacerItem(40, 20, QtGui.QSizePolicy.Expanding, QtGui.QSizePolicy.Minimum) -# self.gridLayout.addItem(spacerItem2, 3, 4, 1, 1) - - - -# spacerItem3 = QtGui.QSpacerItem(40, 20, QtGui.QSizePolicy.Expanding, QtGui.QSizePolicy.Minimum) -# self.gridLayout.addItem(spacerItem3, 1, 4, 1, 1) - - - self.volOpCebDecodification = QtGui.QCheckBox(self.tabopVoltage) - self.volOpCebDecodification.setObjectName(_fromUtf8("volOpCebDecodification")) - self.gridLayout.addWidget(self.volOpCebDecodification, 9, 0, 1, 1) - - self.volLabCodeMode = QtGui.QLabel(self.tabopVoltage) - self.volLabCodeMode.setObjectName(_fromUtf8("volLabCodeMode")) - self.gridLayout.addWidget(self.volLabCodeMode, 9, 1, 1, 1) - - self.volOpComMode = QtGui.QComboBox(self.tabopVoltage) - self.volOpComMode.setObjectName(_fromUtf8("volOpComMode")) - self.volOpComMode.addItem(_fromUtf8("")) - self.volOpComMode.addItem(_fromUtf8("")) - self.gridLayout.addWidget(self.volOpComMode, 9, 2, 1, 3) - - self.volLabCodeType = QtGui.QLabel(self.tabopVoltage) - self.volLabCodeType.setObjectName(_fromUtf8("volLabCodeType")) - self.gridLayout.addWidget(self.volLabCodeType, 10, 1, 1, 1) - - self.volOpComCode = QtGui.QComboBox(self.tabopVoltage) - self.volOpComCode.setObjectName(_fromUtf8("volOpComCode")) - self.volOpComCode.addItem(_fromUtf8("")) - self.volOpComCode.addItem(_fromUtf8("")) - self.volOpComCode.addItem(_fromUtf8("")) - self.volOpComCode.addItem(_fromUtf8("")) - self.volOpComCode.addItem(_fromUtf8("")) - self.volOpComCode.addItem(_fromUtf8("")) - self.volOpComCode.addItem(_fromUtf8("")) - self.volOpComCode.addItem(_fromUtf8("")) - self.volOpComCode.addItem(_fromUtf8("")) - self.volOpComCode.addItem(_fromUtf8("")) - self.volOpComCode.addItem(_fromUtf8("")) - self.volOpComCode.addItem(_fromUtf8("")) - self.volOpComCode.addItem(_fromUtf8("")) - self.volOpComCode.addItem(_fromUtf8("")) - self.gridLayout.addWidget(self.volOpComCode, 10, 2, 1, 3) - - self.volLabCode = QtGui.QLabel(self.tabopVoltage) - self.volLabCode.setObjectName(_fromUtf8("volLabCode")) - self.gridLayout.addWidget(self.volLabCode, 11, 1, 1, 1) - - self.volOpCode = QtGui.QLineEdit(self.tabopVoltage) - self.volOpCode.setObjectName(_fromUtf8("volOpCode")) - self.gridLayout.addWidget(self.volOpCode, 11, 2, 1, 3) - - self.volOpCebFlip = QtGui.QCheckBox(self.tabopVoltage) - self.volOpCebFlip.setObjectName(_fromUtf8("volOpCebFlip")) - self.gridLayout.addWidget(self.volOpCebFlip, 12, 0, 1, 1) - - self.volOpFlip = QtGui.QLineEdit(self.tabopVoltage) - self.volOpFlip.setObjectName(_fromUtf8("volOpFlip")) - self.gridLayout.addWidget(self.volOpFlip, 13, 1, 1, 4) - - self.volOpCebCohInt = QtGui.QCheckBox(self.tabopVoltage) - self.volOpCebCohInt.setObjectName(_fromUtf8("volOpCebCohInt")) - self.gridLayout.addWidget(self.volOpCebCohInt, 13, 0, 1, 1) - - self.volOpCohInt = QtGui.QLineEdit(self.tabopVoltage) - self.volOpCohInt.setObjectName(_fromUtf8("volOpCohInt")) - self.gridLayout.addWidget(self.volOpCohInt, 13, 1, 1, 4) - - self.volOpCebAdjustHei = QtGui.QCheckBox(self.tabopVoltage) - self.volOpCebAdjustHei.setObjectName(_fromUtf8("volOpCebAdjustHei")) - self.gridLayout.addWidget(self.volOpCebAdjustHei, 14, 0, 1, 1) - - self.volOpAdjustHei = QtGui.QLineEdit(self.tabopVoltage) - self.volOpAdjustHei.setObjectName(_fromUtf8("volOpAdjustHei")) - self.gridLayout.addWidget(self.volOpAdjustHei, 14, 1, 1, 4) - - self.tabWidgetVoltage.addTab(self.tabopVoltage, _fromUtf8("")) - - self.tabgraphVoltage = QtGui.QWidget() - self.tabgraphVoltage.setObjectName(_fromUtf8("tabgraphVoltage")) - self.gridLayout_6 = QtGui.QGridLayout(self.tabgraphVoltage) - self.gridLayout_6.setObjectName(_fromUtf8("gridLayout_6")) - spacerItem4 = QtGui.QSpacerItem(20, 40, QtGui.QSizePolicy.Minimum, QtGui.QSizePolicy.Expanding) - self.gridLayout_6.addItem(spacerItem4, 12, 3, 1, 1) - self.volGraphIntensityRange = QtGui.QLineEdit(self.tabgraphVoltage) - self.volGraphIntensityRange.setObjectName(_fromUtf8("volGraphIntensityRange")) - self.gridLayout_6.addWidget(self.volGraphIntensityRange, 9, 1, 1, 6) - self.volGraphPrefix = QtGui.QLineEdit(self.tabgraphVoltage) - self.volGraphPrefix.setObjectName(_fromUtf8("volGraphPrefix")) - self.gridLayout_6.addWidget(self.volGraphPrefix, 2, 1, 1, 6) - self.volGraphToolPath = QtGui.QToolButton(self.tabgraphVoltage) - self.volGraphToolPath.setObjectName(_fromUtf8("volGraphToolPath")) - self.gridLayout_6.addWidget(self.volGraphToolPath, 1, 5, 1, 2) - self.volGraphPath = QtGui.QLineEdit(self.tabgraphVoltage) - self.volGraphPath.setObjectName(_fromUtf8("volGraphPath")) - self.gridLayout_6.addWidget(self.volGraphPath, 1, 1, 1, 4) - self.label_14 = QtGui.QLabel(self.tabgraphVoltage) - self.label_14.setObjectName(_fromUtf8("label_14")) - self.gridLayout_6.addWidget(self.label_14, 6, 0, 1, 1) - spacerItem5 = QtGui.QSpacerItem(20, 40, QtGui.QSizePolicy.Minimum, QtGui.QSizePolicy.Expanding) - self.gridLayout_6.addItem(spacerItem5, 3, 3, 1, 1) - self.label_8 = QtGui.QLabel(self.tabgraphVoltage) - self.label_8.setObjectName(_fromUtf8("label_8")) - self.gridLayout_6.addWidget(self.label_8, 8, 0, 1, 1) - self.label_49 = QtGui.QLabel(self.tabgraphVoltage) - self.label_49.setObjectName(_fromUtf8("label_49")) - self.gridLayout_6.addWidget(self.label_49, 4, 3, 1, 1) - self.label_51 = QtGui.QLabel(self.tabgraphVoltage) - self.label_51.setObjectName(_fromUtf8("label_51")) - self.gridLayout_6.addWidget(self.label_51, 9, 0, 1, 1) - self.volGraphCebshow = QtGui.QCheckBox(self.tabgraphVoltage) - self.volGraphCebshow.setText(_fromUtf8("")) - self.volGraphCebshow.setObjectName(_fromUtf8("volGraphCebshow")) - self.gridLayout_6.addWidget(self.volGraphCebshow, 6, 3, 1, 1) - self.label_12 = QtGui.QLabel(self.tabgraphVoltage) - self.label_12.setObjectName(_fromUtf8("label_12")) - self.gridLayout_6.addWidget(self.label_12, 1, 0, 1, 1) - self.label_13 = QtGui.QLabel(self.tabgraphVoltage) - self.label_13.setObjectName(_fromUtf8("label_13")) - self.gridLayout_6.addWidget(self.label_13, 2, 0, 1, 1) - self.label_52 = QtGui.QLabel(self.tabgraphVoltage) - self.label_52.setObjectName(_fromUtf8("label_52")) - - self.gridLayout_6.addWidget(self.label_52, 11, 0, 1, 1) - spacerItem6 = QtGui.QSpacerItem(40, 12, QtGui.QSizePolicy.Expanding, QtGui.QSizePolicy.Minimum) - self.gridLayout_6.addItem(spacerItem6, 14, 5, 1, 2) - spacerItem7 = QtGui.QSpacerItem(18, 12, QtGui.QSizePolicy.Expanding, QtGui.QSizePolicy.Minimum) - self.gridLayout_6.addItem(spacerItem7, 14, 3, 1, 1) - - self.volLabScopeType = QtGui.QLabel(self.tabgraphVoltage) - self.volLabScopeType.setObjectName(_fromUtf8("volLabScopeType")) - self.gridLayout_6.addWidget(self.volLabScopeType, 7, 0, 1, 1) - - self.volComScopeType = QtGui.QComboBox(self.tabgraphVoltage) - self.volComScopeType.setObjectName(_fromUtf8("volComScopeType")) - self.volComScopeType.addItem(_fromUtf8("")) - self.volComScopeType.addItem(_fromUtf8("")) - self.gridLayout_6.addWidget(self.volComScopeType, 7, 1, 1, 6) - - self.volGraphChannelList = QtGui.QLineEdit(self.tabgraphVoltage) - self.volGraphChannelList.setObjectName(_fromUtf8("volGraphChannelList")) - self.gridLayout_6.addWidget(self.volGraphChannelList, 8, 1, 1, 6) - - - self.volGraphHeightrange = QtGui.QLineEdit(self.tabgraphVoltage) - self.volGraphHeightrange.setObjectName(_fromUtf8("volGraphHeightrange")) - self.gridLayout_6.addWidget(self.volGraphHeightrange, 11, 1, 1, 6) - self.label_50 = QtGui.QLabel(self.tabgraphVoltage) - self.label_50.setObjectName(_fromUtf8("label_50")) - self.gridLayout_6.addWidget(self.label_50, 4, 4, 1, 1) - self.volGraphCebSave = QtGui.QCheckBox(self.tabgraphVoltage) - self.volGraphCebSave.setText(_fromUtf8("")) - self.volGraphCebSave.setObjectName(_fromUtf8("volGraphCebSave")) - self.gridLayout_6.addWidget(self.volGraphCebSave, 6, 4, 1, 1) - self.tabWidgetVoltage.addTab(self.tabgraphVoltage, _fromUtf8("")) - - self.taboutputVoltage = QtGui.QWidget() - self.taboutputVoltage.setObjectName(_fromUtf8("taboutputVoltage")) - self.gridLayout_12 = QtGui.QGridLayout(self.taboutputVoltage) - self.gridLayout_12.setObjectName(_fromUtf8("gridLayout_12")) - self.label_36 = QtGui.QLabel(self.taboutputVoltage) - self.label_36.setObjectName(_fromUtf8("label_36")) - self.gridLayout_12.addWidget(self.label_36, 0, 0, 1, 1) - self.label_37 = QtGui.QLabel(self.taboutputVoltage) - self.label_37.setObjectName(_fromUtf8("label_37")) - self.gridLayout_12.addWidget(self.label_37, 1, 0, 1, 1) - self.volOutputPath = QtGui.QLineEdit(self.taboutputVoltage) - self.volOutputPath.setObjectName(_fromUtf8("volOutputPath")) - self.gridLayout_12.addWidget(self.volOutputPath, 1, 2, 1, 1) - self.volOutputToolPath = QtGui.QToolButton(self.taboutputVoltage) - self.volOutputToolPath.setObjectName(_fromUtf8("volOutputToolPath")) - self.gridLayout_12.addWidget(self.volOutputToolPath, 1, 3, 1, 1) - self.volOutputComData = QtGui.QComboBox(self.taboutputVoltage) - self.volOutputComData.setObjectName(_fromUtf8("volOutputComData")) - self.volOutputComData.addItem(_fromUtf8("")) - self.gridLayout_12.addWidget(self.volOutputComData, 0, 2, 1, 2) - spacerItem8 = QtGui.QSpacerItem(20, 40, QtGui.QSizePolicy.Minimum, QtGui.QSizePolicy.Expanding) - self.gridLayout_12.addItem(spacerItem8, 5, 2, 1, 1) - self.volOutputblocksperfile = QtGui.QLineEdit(self.taboutputVoltage) - self.volOutputblocksperfile.setObjectName(_fromUtf8("volOutputblocksperfile")) - self.gridLayout_12.addWidget(self.volOutputblocksperfile, 3, 2, 1, 1) - self.label_7 = QtGui.QLabel(self.taboutputVoltage) - self.label_7.setObjectName(_fromUtf8("label_7")) - self.gridLayout_12.addWidget(self.label_7, 3, 0, 1, 1) - self.label_35 = QtGui.QLabel(self.taboutputVoltage) - self.label_35.setObjectName(_fromUtf8("label_35")) - self.gridLayout_12.addWidget(self.label_35, 4, 0, 1, 1) - self.volOutputprofilesperblock = QtGui.QLineEdit(self.taboutputVoltage) - self.volOutputprofilesperblock.setObjectName(_fromUtf8("volOutputprofilesperblock")) - self.gridLayout_12.addWidget(self.volOutputprofilesperblock, 4, 2, 1, 1) - self.tabWidgetVoltage.addTab(self.taboutputVoltage, _fromUtf8("")) - self.gridLayout_3.addWidget(self.tabWidgetVoltage, 0, 1, 1, 1) - - self.tabWidgetProject.addTab(self.tabVoltage, _fromUtf8("")) - - self.tabWidgetVoltage.setCurrentIndex(0) - - def retranslateUi(self): - - self.volOpOk.setText(_translate("MainWindow", "Ok", None)) - self.volGraphClear.setText(_translate("MainWindow", "Clear", None)) - self.volOpComHeights.setItemText(0, _translate("MainWindow", "Value", None)) - self.volOpComHeights.setItemText(1, _translate("MainWindow", "Index", None)) - self.volOpComChannels.setItemText(0, _translate("MainWindow", "Value", None)) - self.volOpComChannels.setItemText(1, _translate("MainWindow", "Index", None)) - self.volOpCebProfile.setText(_translate("MainWindow", "Select Profiles", None)) - self.volOpComProfile.setItemText(0, _translate("MainWindow", "Profile List", None)) - self.volOpComProfile.setItemText(1, _translate("MainWindow", "Profile Range", None)) - self.volOpComProfile.setItemText(2, _translate("MainWindow", "List of Profile Ranges", None)) - self.volOpCebDecodification.setText(_translate("MainWindow", "Decoder:", None)) - self.volOpCebCohInt.setText(_translate("MainWindow", "Coherent Integration:", None)) - self.volOpCebFlip.setText(_translate("MainWindow", "Flip:", None)) - self.volLabCodeType.setText(_translate("MainWindow", "Code type:", None)) - self.volOpCebChannels.setText(_translate("MainWindow", "Select Channels:", None)) - self.volOpCebHeights.setText(_translate("MainWindow", "Select Heights:", None)) - self.volOpCebFilter.setText(_translate("MainWindow", "Range Window Filter:", None)) - self.volOpCebSplitter.setText(_translate("MainWindow", "Split Profiles: ", None)) - self.volOpCebCombiner.setText(_translate("MainWindow", "Combine Profiles: ", None)) - self.volOpCebRadarfrequency.setText(_translate("MainWindow", "Radar frequency (MHz):", None)) - self.volLabCodeMode.setText(_translate("MainWindow", "Mode:", None)) - self.volLabCode.setText(_translate("MainWindow", "Code:", None)) - self.volOpComCode.setItemText(0, _translate("MainWindow", "Read from header", None)) - self.volOpComCode.setItemText(1, _translate("MainWindow", "Barker 3", None)) - self.volOpComCode.setItemText(2, _translate("MainWindow", "Barker 4", None)) - self.volOpComCode.setItemText(3, _translate("MainWindow", "Barker 5", None)) - self.volOpComCode.setItemText(4, _translate("MainWindow", "Barker 7", None)) - self.volOpComCode.setItemText(5, _translate("MainWindow", "Barker 11", None)) - self.volOpComCode.setItemText(6, _translate("MainWindow", "Barker 13", None)) - self.volOpComCode.setItemText(7, _translate("MainWindow", "Barker 3 + Comp.", None)) - self.volOpComCode.setItemText(8, _translate("MainWindow", "Barker 4 + Comp.", None)) - self.volOpComCode.setItemText(9, _translate("MainWindow", "Barker 5 + Comp.", None)) - self.volOpComCode.setItemText(10, _translate("MainWindow", "Barker 7 + Comp.", None)) - self.volOpComCode.setItemText(11, _translate("MainWindow", "Barker 11+ Comp.", None)) - self.volOpComCode.setItemText(12, _translate("MainWindow", "Barker 13+ Comp.", None)) - self.volOpComCode.setItemText(13, _translate("MainWindow", "User defined", None)) - self.volOpComMode.setItemText(0, _translate("MainWindow", "Time", None)) - self.volOpComMode.setItemText(1, _translate("MainWindow", "Frequency", None)) - self.volOpCebAdjustHei.setText(_translate("MainWindow", "Calibrate H0:", None)) - - self.tabWidgetVoltage.setTabText(self.tabWidgetVoltage.indexOf(self.tabopVoltage), _translate("MainWindow", "Operation", None)) - - self.volGraphToolPath.setText(_translate("MainWindow", "...", None)) - self.label_14.setText(_translate("MainWindow", "Scope:", None)) - self.label_8.setText(_translate("MainWindow", "Channel List:", None)) - self.label_49.setText(_translate("MainWindow", "Show:", None)) - self.label_51.setText(_translate("MainWindow", "Amplitude/Intensity:", None)) - self.label_12.setText(_translate("MainWindow", "Path :", None)) - self.label_13.setText(_translate("MainWindow", "Figure name:", None)) - self.label_52.setText(_translate("MainWindow", "Height range:", None)) - self.label_50.setText(_translate("MainWindow", "Save:", None)) - - self.volLabScopeType.setText(_translate("MainWindow", "Scope type:", None)) - self.volComScopeType.setItemText(0, _translate("MainWindow", "I&Q", None)) - self.volComScopeType.setItemText(1, _translate("MainWindow", "Power", None)) - - self.tabWidgetVoltage.setTabText(self.tabWidgetVoltage.indexOf(self.tabgraphVoltage), _translate("MainWindow", "Graphics", None)) - - self.label_36.setText(_translate("MainWindow", "Type:", None)) - self.label_37.setText(_translate("MainWindow", "Path:", None)) - self.volOutputToolPath.setText(_translate("MainWindow", "...", None)) - self.volOutputComData.setItemText(0, _translate("MainWindow", ".rawdata", None)) - self.label_7.setText(_translate("MainWindow", "Blocks per File: ", None)) - self.label_35.setText(_translate("MainWindow", "Profiles per Block: ", None)) - self.tabWidgetVoltage.setTabText(self.tabWidgetVoltage.indexOf(self.taboutputVoltage), _translate("MainWindow", "Output", None)) - - self.tabWidgetProject.setTabText(self.tabWidgetProject.indexOf(self.tabVoltage), _translate("MainWindow", "Voltage", None)) - - self.__setToolTip() - - def __setToolTip(self): - - self.volOpChannel.setToolTip('Example: 1,2,3,4,5') - self.volOpHeights.setToolTip('Example: 90,180') - self.volOpFilter.setToolTip('Example: 2') - self.volOpProfile.setToolTip('Example:0,127') - self.volOpCohInt.setToolTip('Example: 128') - self.volOpFlip.setToolTip('ChannelList where flip will be applied. Example: 0,2,3') - self.volOpOk.setToolTip('If you have finished, please Ok ') - # tool tip gui volGraph - self.volGraphIntensityRange.setToolTip('Height range. Example: 50,100') - self.volGraphHeightrange.setToolTip('Amplitude. Example: 0,10000') - - \ No newline at end of file diff --git a/schainpy/model/__init__.py b/schainpy/model/__init__.py index ae3ce46..0422618 100644 --- a/schainpy/model/__init__.py +++ b/schainpy/model/__init__.py @@ -1,12 +1,6 @@ -#from schainpy.model.data.jrodata import * -# from schainpy.model.io.jrodataIO import * -# from schainpy.model.proc.jroprocessing import * -# from schainpy.model.graphics.jroplot import * -# from schainpy.model.utils.jroutils import * -# from schainpy.serializer import * -from graphics import * -from data import * -from io import * -from proc import * -from utils import * +from .graphics import * +from .data import * +from .io import * +from .proc import * +from .utils import * diff --git a/schainpy/model/data/BLTRheaderIO.py b/schainpy/model/data/BLTRheaderIO.py index 69e0d25..534f109 100644 --- a/schainpy/model/data/BLTRheaderIO.py +++ b/schainpy/model/data/BLTRheaderIO.py @@ -7,7 +7,7 @@ import sys import numpy import copy import datetime -from __builtin__ import None + SPEED_OF_LIGHT = 299792458 SPEED_OF_LIGHT = 3e8 @@ -78,7 +78,7 @@ class Header(object): message += self.__class__.__name__.upper() + "\n" message += "#"*50 + "\n" - keyList = self.__dict__.keys() + keyList = list(self.__dict__.keys()) keyList.sort() for key in keyList: @@ -90,7 +90,7 @@ class Header(object): if attr: message += "%s = %s" %("size", attr) + "\n" - print message + print(message) class FileHeader(Header): @@ -134,9 +134,9 @@ class FileHeader(Header): ''' - except Exception, e: - print "FileHeader: " - print eBasicHeader + except Exception as e: + print("FileHeader: ") + print(eBasicHeader) return 0 self.FileMgcNumber= byte(header['FileMgcNumber'][0]) @@ -279,8 +279,8 @@ class RecordHeader(Header): try: header = numpy.fromfile(fp,RECORD_STRUCTURE,1) - except Exception, e: - print "System Header: " + e + except Exception as e: + print("System Header: " + e) return 0 self.RecMgcNumber = header['RecMgcNumber'][0] #0x23030001 diff --git a/schainpy/model/data/__init__.py b/schainpy/model/data/__init__.py index 6052841..c9dbc75 100644 --- a/schainpy/model/data/__init__.py +++ b/schainpy/model/data/__init__.py @@ -1,3 +1,3 @@ -from jrodata import * -from jroheaderIO import * -from jroamisr import * \ No newline at end of file +from .jrodata import * +from .jroheaderIO import * +from .jroamisr import * \ No newline at end of file diff --git a/schainpy/model/data/jroamisr.py b/schainpy/model/data/jroamisr.py index dc87704..954502b 100644 --- a/schainpy/model/data/jroamisr.py +++ b/schainpy/model/data/jroamisr.py @@ -68,7 +68,7 @@ class AMISR: if inputObj is None: return copy.deepcopy(self) - for key in inputObj.__dict__.keys(): + for key in list(inputObj.__dict__.keys()): self.__dict__[key] = inputObj.__dict__[key] def getNHeights(self): diff --git a/schainpy/model/data/jrodata.py b/schainpy/model/data/jrodata.py index 96ad913..258e7e1 100644 --- a/schainpy/model/data/jrodata.py +++ b/schainpy/model/data/jrodata.py @@ -7,9 +7,12 @@ $Id: JROData.py 173 2012-11-20 15:06:21Z murco $ import copy import numpy import datetime +import json -from jroheaderIO import SystemHeader, RadarControllerHeader -from schainpy import cSchain +import schainpy.admin +from schainpy.utils import log +from .jroheaderIO import SystemHeader, RadarControllerHeader +from schainpy.model.data import _noise def getNumpyDtype(dataTypeCode): @@ -27,7 +30,7 @@ def getNumpyDtype(dataTypeCode): elif dataTypeCode == 5: numpyDtype = numpy.dtype([('real', ' nums_min: -# rtest = float(j)/(j-1) + 1.0/navg -# if ((sumq*j) > (rtest*sump**2)): -# j = j - 1 -# sump = sump - sortdata[j] -# sumq = sumq - sortdata[j]**2 -# cont = 0 -# -# j += 1 -# -# lnoise = sump /j -# -# return lnoise + ''' + lenOfData = len(sortdata) + nums_min = lenOfData*0.2 + + if nums_min <= 5: + + nums_min = 5 + + sump = 0. + sumq = 0. + + j = 0 + cont = 1 - return cSchain.hildebrand_sekhon(sortdata, navg) + while((cont == 1)and(j < lenOfData)): + + sump += sortdata[j] + sumq += sortdata[j]**2 + + if j > nums_min: + rtest = float(j)/(j-1) + 1.0/navg + if ((sumq*j) > (rtest*sump**2)): + j = j - 1 + sump = sump - sortdata[j] + sumq = sumq - sortdata[j]**2 + cont = 0 + + j += 1 + + lnoise = sump / j + ''' + return _noise.hildebrand_sekhon(sortdata, navg) class Beam: @@ -122,7 +121,7 @@ class GenericData(object): if inputObj == None: return copy.deepcopy(self) - for key in inputObj.__dict__.keys(): + for key in list(inputObj.__dict__.keys()): attribute = inputObj.__dict__[key] @@ -146,6 +145,10 @@ class GenericData(object): return self.flagNoData + def isReady(self): + + return not self.flagNoData + class JROData(GenericData): @@ -153,83 +156,52 @@ class JROData(GenericData): # m_ProcessingHeader = ProcessingHeader() systemHeaderObj = SystemHeader() - radarControllerHeaderObj = RadarControllerHeader() - # data = None - type = None - datatype = None # dtype but in string - # dtype = None - # nChannels = None - # nHeights = None - nProfiles = None - heightList = None - channelList = None - flagDiscontinuousBlock = False - useLocalTime = False - utctime = None - timeZone = None - dstFlag = None - errorCount = None - blocksize = None - # nCode = None -# # nBaud = None -# # code = None - flagDecodeData = False # asumo q la data no esta decodificada - flagDeflipData = False # asumo q la data no esta sin flip - flagShiftFFT = False - # ippSeconds = None - # timeInterval = None - nCohInt = None - # noise = None - windowOfFilter = 1 - # Speed of ligth C = 3e8 - frequency = 49.92e6 - realtime = False - beacon_heiIndexList = None - last_block = None - blocknow = None - azimuth = None - zenith = None - beam = Beam() - profileIndex = None + error = None + data = None + nmodes = None + + def __str__(self): + + return '{} - {}'.format(self.type, self.getDatatime()) def getNoise(self): @@ -241,7 +213,7 @@ class JROData(GenericData): def getChannelIndexList(self): - return range(self.nChannels) + return list(range(self.nChannels)) def getNHeights(self): @@ -388,6 +360,10 @@ class Voltage(JROData): # data es un numpy array de 2 dmensiones (canales, alturas) data = None + dataPP_POW = None + dataPP_DOP = None + dataPP_WIDTH = None + dataPP_SNR = None def __init__(self): ''' @@ -395,53 +371,30 @@ class Voltage(JROData): ''' self.useLocalTime = True - self.radarControllerHeaderObj = RadarControllerHeader() - self.systemHeaderObj = SystemHeader() - self.type = "Voltage" - self.data = None - # self.dtype = None - # self.nChannels = 0 - # self.nHeights = 0 - self.nProfiles = None - self.heightList = None - self.channelList = None - # self.channelIndexList = None - self.flagNoData = True - self.flagDiscontinuousBlock = False - self.utctime = None - - self.timeZone = None - + self.timeZone = 0 self.dstFlag = None - self.errorCount = None - self.nCohInt = None - self.blocksize = None - + self.flagCohInt = False self.flagDecodeData = False # asumo q la data no esta decodificada - self.flagDeflipData = False # asumo q la data no esta sin flip - self.flagShiftFFT = False - self.flagDataAsBlock = False # Asumo que la data es leida perfil a perfil - self.profileIndex = 0 def getNoisebyHildebrand(self, channel=None): @@ -505,32 +458,20 @@ class Spectra(JROData): # data spc es un numpy array de 2 dmensiones (canales, perfiles, alturas) data_spc = None - # data cspc es un numpy array de 2 dmensiones (canales, pares, alturas) data_cspc = None - # data dc es un numpy array de 2 dmensiones (canales, alturas) data_dc = None - # data power data_pwr = None - nFFTPoints = None - # nPairs = None - pairsList = None - nIncohInt = None - wavelength = None # Necesario para cacular el rango de velocidad desde la frecuencia - nCohInt = None # se requiere para determinar el valor de timeInterval - ippFactor = None - profileIndex = 0 - plotting = "spectra" def __init__(self): @@ -539,59 +480,33 @@ class Spectra(JROData): ''' self.useLocalTime = True - self.radarControllerHeaderObj = RadarControllerHeader() - self.systemHeaderObj = SystemHeader() - self.type = "Spectra" - + self.timeZone = 0 # self.data = None - # self.dtype = None - # self.nChannels = 0 - # self.nHeights = 0 - self.nProfiles = None - self.heightList = None - self.channelList = None - # self.channelIndexList = None - self.pairsList = None - self.flagNoData = True - self.flagDiscontinuousBlock = False - self.utctime = None - self.nCohInt = None - self.nIncohInt = None - self.blocksize = None - self.nFFTPoints = None - self.wavelength = None - self.flagDecodeData = False # asumo q la data no esta decodificada - self.flagDeflipData = False # asumo q la data no esta sin flip - self.flagShiftFFT = False - self.ippFactor = 1 - #self.noise = None - self.beacon_heiIndexList = [] - self.noise_estimation = None def getNoisebyHildebrand(self, xmin_index=None, xmax_index=None, ymin_index=None, ymax_index=None): @@ -624,37 +539,33 @@ class Spectra(JROData): def getFreqRangeTimeResponse(self, extrapoints=0): deltafreq = self.getFmaxTimeResponse() / (self.nFFTPoints * self.ippFactor) - freqrange = deltafreq * \ - (numpy.arange(self.nFFTPoints + extrapoints) - - self.nFFTPoints / 2.) - deltafreq / 2 + freqrange = deltafreq * (numpy.arange(self.nFFTPoints + extrapoints) - self.nFFTPoints / 2.) - deltafreq / 2 return freqrange def getAcfRange(self, extrapoints=0): deltafreq = 10. / (self.getFmax() / (self.nFFTPoints * self.ippFactor)) - freqrange = deltafreq * \ - (numpy.arange(self.nFFTPoints + extrapoints) - - self.nFFTPoints / 2.) - deltafreq / 2 + freqrange = deltafreq * (numpy.arange(self.nFFTPoints + extrapoints) -self.nFFTPoints / 2.) - deltafreq / 2 return freqrange def getFreqRange(self, extrapoints=0): deltafreq = self.getFmax() / (self.nFFTPoints * self.ippFactor) - freqrange = deltafreq * \ - (numpy.arange(self.nFFTPoints + extrapoints) - - self.nFFTPoints / 2.) - deltafreq / 2 + freqrange = deltafreq * (numpy.arange(self.nFFTPoints + extrapoints) -self.nFFTPoints / 2.) - deltafreq / 2 return freqrange def getVelRange(self, extrapoints=0): deltav = self.getVmax() / (self.nFFTPoints * self.ippFactor) - velrange = deltav * (numpy.arange(self.nFFTPoints + - extrapoints) - self.nFFTPoints / 2.) # - deltav/2 + velrange = deltav * (numpy.arange(self.nFFTPoints + extrapoints) - self.nFFTPoints / 2.) - return velrange + if self.nmodes: + return velrange/self.nmodes + else: + return velrange def getNPairs(self): @@ -662,7 +573,7 @@ class Spectra(JROData): def getPairsIndexList(self): - return range(self.nPairs) + return list(range(self.nPairs)) def getNormFactor(self): @@ -671,8 +582,7 @@ class Spectra(JROData): if self.flagDecodeData: pwcode = numpy.sum(self.code[0]**2) #normFactor = min(self.nFFTPoints,self.nProfiles)*self.nIncohInt*self.nCohInt*pwcode*self.windowOfFilter - normFactor = self.nProfiles * self.nIncohInt * \ - self.nCohInt * pwcode * self.windowOfFilter + normFactor = self.nProfiles * self.nIncohInt * self.nCohInt * pwcode * self.windowOfFilter return normFactor @@ -693,8 +603,10 @@ class Spectra(JROData): def getTimeInterval(self): timeInterval = self.ippSeconds * self.nCohInt * self.nIncohInt * self.nProfiles * self.ippFactor - - return timeInterval + if self.nmodes: + return self.nmodes*timeInterval + else: + return timeInterval def getPower(self): @@ -714,13 +626,12 @@ class Spectra(JROData): pairsIndexList = [] for pair in pairsList: if pair not in self.pairsList: - raise ValueError, "Pair %s is not in dataOut.pairsList" % ( - pair) + raise ValueError("Pair %s is not in dataOut.pairsList" % ( + pair)) pairsIndexList.append(self.pairsList.index(pair)) for i in range(len(pairsIndexList)): pair = self.pairsList[pairsIndexList[i]] - ccf = numpy.average( - self.data_cspc[pairsIndexList[i], :, :], axis=0) + ccf = numpy.average(self.data_cspc[pairsIndexList[i], :, :], axis=0) powa = numpy.average(self.data_spc[pair[0], :, :], axis=0) powb = numpy.average(self.data_spc[pair[1], :, :], axis=0) avgcoherenceComplex = ccf / numpy.sqrt(powa * powb) @@ -736,7 +647,7 @@ class Spectra(JROData): def setValue(self, value): - print "This property should not be initialized" + print("This property should not be initialized") return @@ -755,19 +666,12 @@ class Spectra(JROData): class SpectraHeis(Spectra): data_spc = None - data_cspc = None - data_dc = None - nFFTPoints = None - # nPairs = None - pairsList = None - nCohInt = None - nIncohInt = None def __init__(self): @@ -830,36 +734,20 @@ class SpectraHeis(Spectra): class Fits(JROData): heightList = None - channelList = None - flagNoData = True - flagDiscontinuousBlock = False - useLocalTime = False - utctime = None - - timeZone = None - # ippSeconds = None - # timeInterval = None - nCohInt = None - nIncohInt = None - noise = None - windowOfFilter = 1 - # Speed of ligth C = 3e8 - frequency = 49.92e6 - realtime = False def __init__(self): @@ -887,7 +775,7 @@ class Fits(JROData): self.profileIndex = 0 # self.utctime = None -# self.timeZone = None + self.timeZone = 0 # self.ltctime = None # self.timeInterval = None # self.header = None @@ -941,7 +829,7 @@ class Fits(JROData): def getChannelIndexList(self): - return range(self.nChannels) + return list(range(self.nChannels)) def getNoise(self, type=1): @@ -964,6 +852,12 @@ class Fits(JROData): return timeInterval + def get_ippSeconds(self): + ''' + ''' + return self.ipp_sec + + datatime = property(getDatatime, "I'm the 'datatime' property") nHeights = property(getNHeights, "I'm the 'nHeights' property.") nChannels = property(getNChannels, "I'm the 'nChannel' property.") @@ -973,38 +867,24 @@ class Fits(JROData): ltctime = property(getltctime, "I'm the 'ltctime' property") timeInterval = property(getTimeInterval, "I'm the 'timeInterval' property") - + ippSeconds = property(get_ippSeconds, '') class Correlation(JROData): noise = None - SNR = None - #-------------------------------------------------- - mode = None - split = False - data_cf = None - lags = None - lagRange = None - pairsList = None - normFactor = None - #-------------------------------------------------- - # calculateVelocity = None - nLags = None - nPairs = None - nAvg = None def __init__(self): @@ -1033,7 +913,7 @@ class Correlation(JROData): self.utctime = None - self.timeZone = None + self.timeZone = 0 self.dstFlag = None @@ -1068,7 +948,8 @@ class Correlation(JROData): ind_vel = numpy.array([-2, -1, 1, 2]) + freq_dc if ind_vel[0] < 0: - ind_vel[range(0, 1)] = ind_vel[range(0, 1)] + self.num_prof + ind_vel[list(range(0, 1))] = ind_vel[list( + range(0, 1))] + self.num_prof if mode == 1: jspectra[:, freq_dc, :] = ( @@ -1080,7 +961,7 @@ class Correlation(JROData): xx = numpy.zeros([4, 4]) for fil in range(4): - xx[fil, :] = vel[fil]**numpy.asarray(range(4)) + xx[fil, :] = vel[fil]**numpy.asarray(list(range(4))) xx_inv = numpy.linalg.inv(xx) xx_aux = xx_inv[0, :] @@ -1154,55 +1035,30 @@ class Correlation(JROData): class Parameters(Spectra): experimentInfo = None # Information about the experiment - # Information from previous data - inputUnit = None # Type of data to be processed - operation = None # Type of operation to parametrize - # normFactor = None #Normalization Factor - groupList = None # List of Pairs, Groups, etc - # Parameters - data_param = None # Parameters obtained - data_pre = None # Data Pre Parametrization - data_SNR = None # Signal to Noise Ratio - # heightRange = None #Heights - abscissaList = None # Abscissa, can be velocities, lags or time - # noise = None #Noise Potency - utctimeInit = None # Initial UTC time - paramInterval = None # Time interval to calculate Parameters in seconds - useLocalTime = True - # Fitting - data_error = None # Error of the estimation - constants = None - library = None - # Output signal - outputInterval = None # Time interval to calculate output signal in seconds - data_output = None # Out signal - nAvg = None - noise_estimation = None - GauSPC = None # Fit gaussian SPC def __init__(self): @@ -1210,10 +1066,9 @@ class Parameters(Spectra): Constructor ''' self.radarControllerHeaderObj = RadarControllerHeader() - self.systemHeaderObj = SystemHeader() - self.type = "Parameters" + self.timeZone = 0 def getTimeRange1(self, interval): @@ -1239,7 +1094,7 @@ class Parameters(Spectra): def setValue(self, value): - print "This property should not be initialized" + print("This property should not be initialized") return @@ -1249,3 +1104,297 @@ class Parameters(Spectra): timeInterval = property(getTimeInterval) noise = property(getNoise, setValue, "I'm the 'Noise' property.") + + +class PlotterData(object): + ''' + Object to hold data to be plotted + ''' + + MAXNUMX = 200 + MAXNUMY = 200 + + def __init__(self, code, throttle_value, exp_code, localtime=True, buffering=True, snr=False): + + self.key = code + self.throttle = throttle_value + self.exp_code = exp_code + self.buffering = buffering + self.ready = False + self.flagNoData = False + self.localtime = localtime + self.data = {} + self.meta = {} + self.__heights = [] + + if 'snr' in code: + self.plottypes = ['snr'] + elif code == 'spc': + self.plottypes = ['spc', 'noise', 'rti'] + elif code == 'cspc': + self.plottypes = ['cspc', 'spc', 'noise', 'rti'] + elif code == 'rti': + self.plottypes = ['noise', 'rti'] + else: + self.plottypes = [code] + + if 'snr' not in self.plottypes and snr: + self.plottypes.append('snr') + + for plot in self.plottypes: + self.data[plot] = {} + + def __str__(self): + dum = ['{}{}'.format(key, self.shape(key)) for key in self.data] + return 'Data[{}][{}]'.format(';'.join(dum), len(self.times)) + + def __len__(self): + return len(self.data[self.key]) + + def __getitem__(self, key): + + if key not in self.data: + raise KeyError(log.error('Missing key: {}'.format(key))) + if 'spc' in key or not self.buffering: + ret = self.data[key][self.tm] + elif 'scope' in key: + ret = numpy.array(self.data[key][float(self.tm)]) + else: + ret = numpy.array([self.data[key][x] for x in self.times]) + if ret.ndim > 1: + ret = numpy.swapaxes(ret, 0, 1) + return ret + + def __contains__(self, key): + return key in self.data + + def setup(self): + ''' + Configure object + ''' + self.type = '' + self.ready = False + del self.data + self.data = {} + self.__heights = [] + self.__all_heights = set() + for plot in self.plottypes: + if 'snr' in plot: + plot = 'snr' + elif 'spc_moments' == plot: + plot = 'moments' + self.data[plot] = {} + + if 'spc' in self.data or 'rti' in self.data or 'cspc' in self.data or 'moments' in self.data: + self.data['noise'] = {} + self.data['rti'] = {} + if 'noise' not in self.plottypes: + self.plottypes.append('noise') + if 'rti' not in self.plottypes: + self.plottypes.append('rti') + + def shape(self, key): + ''' + Get the shape of the one-element data for the given key + ''' + + if len(self.data[key]): + if 'spc' in key or not self.buffering: + return self.data[key].shape + return self.data[key][self.times[0]].shape + return (0,) + + def update(self, dataOut, tm): + ''' + Update data object with new dataOut + ''' + + self.profileIndex = dataOut.profileIndex + self.tm = tm + self.type = dataOut.type + self.parameters = getattr(dataOut, 'parameters', []) + + if hasattr(dataOut, 'meta'): + self.meta.update(dataOut.meta) + + if hasattr(dataOut, 'pairsList'): + self.pairs = dataOut.pairsList + + self.interval = dataOut.getTimeInterval() + if True in ['spc' in ptype for ptype in self.plottypes]: + self.xrange = (dataOut.getFreqRange(1)/1000., + dataOut.getAcfRange(1), dataOut.getVelRange(1)) + self.__heights.append(dataOut.heightList) + self.__all_heights.update(dataOut.heightList) + + for plot in self.plottypes: + if plot in ('spc', 'spc_moments', 'spc_cut'): + z = dataOut.data_spc/dataOut.normFactor + buffer = 10*numpy.log10(z) + if plot == 'cspc': + buffer = (dataOut.data_spc, dataOut.data_cspc) + if plot == 'noise': + buffer = 10*numpy.log10(dataOut.getNoise()/dataOut.normFactor) + if plot in ('rti', 'spcprofile'): + buffer = dataOut.getPower() + if plot == 'snr_db': + buffer = dataOut.data_SNR + if plot == 'snr': + buffer = 10*numpy.log10(dataOut.data_SNR) + if plot == 'dop': + buffer = dataOut.data_DOP + if plot == 'pow': + buffer = 10*numpy.log10(dataOut.data_POW) + if plot == 'width': + buffer = dataOut.data_WIDTH + if plot == 'coh': + buffer = dataOut.getCoherence() + if plot == 'phase': + buffer = dataOut.getCoherence(phase=True) + if plot == 'output': + buffer = dataOut.data_output + if plot == 'param': + buffer = dataOut.data_param + if plot == 'scope': + buffer = dataOut.data + self.flagDataAsBlock = dataOut.flagDataAsBlock + self.nProfiles = dataOut.nProfiles + if plot == 'pp_power': + buffer = dataOut.dataPP_POWER + self.flagDataAsBlock = dataOut.flagDataAsBlock + self.nProfiles = dataOut.nProfiles + if plot == 'pp_signal': + buffer = dataOut.dataPP_POW + self.flagDataAsBlock = dataOut.flagDataAsBlock + self.nProfiles = dataOut.nProfiles + if plot == 'pp_velocity': + buffer = dataOut.dataPP_DOP + self.flagDataAsBlock = dataOut.flagDataAsBlock + self.nProfiles = dataOut.nProfiles + if plot == 'pp_specwidth': + buffer = dataOut.dataPP_WIDTH + self.flagDataAsBlock = dataOut.flagDataAsBlock + self.nProfiles = dataOut.nProfiles + + if plot == 'spc': + self.data['spc'][tm] = buffer + elif plot == 'cspc': + self.data['cspc'][tm] = buffer + elif plot == 'spc_moments': + self.data['spc'][tm] = buffer + self.data['moments'][tm] = dataOut.moments + else: + if self.buffering: + self.data[plot][tm] = buffer + else: + self.data[plot][tm] = buffer + + if dataOut.channelList is None: + self.channels = range(buffer.shape[0]) + else: + self.channels = dataOut.channelList + + if buffer is None: + self.flagNoData = True + raise schainpy.admin.SchainWarning('Attribute data_{} is empty'.format(self.key)) + + def normalize_heights(self): + ''' + Ensure same-dimension of the data for different heighList + ''' + + H = numpy.array(list(self.__all_heights)) + H.sort() + for key in self.data: + shape = self.shape(key)[:-1] + H.shape + for tm, obj in list(self.data[key].items()): + h = self.__heights[self.times.tolist().index(tm)] + if H.size == h.size: + continue + index = numpy.where(numpy.in1d(H, h))[0] + dummy = numpy.zeros(shape) + numpy.nan + if len(shape) == 2: + dummy[:, index] = obj + else: + dummy[index] = obj + self.data[key][tm] = dummy + + self.__heights = [H for tm in self.times] + + def jsonify(self, tm, plot_name, plot_type, decimate=False): + ''' + Convert data to json + ''' + + dy = int(self.heights.size/self.MAXNUMY) + 1 + if self.key in ('spc', 'cspc'): + dx = int(self.data[self.key][tm].shape[1]/self.MAXNUMX) + 1 + data = self.roundFloats( + self.data[self.key][tm][::, ::dx, ::dy].tolist()) + else: + if self.key is 'noise': + data = [[x] for x in self.roundFloats(self.data[self.key][tm].tolist())] + else: + data = self.roundFloats(self.data[self.key][tm][::, ::dy].tolist()) + + meta = {} + ret = { + 'plot': plot_name, + 'code': self.exp_code, + 'time': float(tm), + 'data': data, + } + meta['type'] = plot_type + meta['interval'] = float(self.interval) + meta['localtime'] = self.localtime + meta['yrange'] = self.roundFloats(self.heights[::dy].tolist()) + if 'spc' in self.data or 'cspc' in self.data: + meta['xrange'] = self.roundFloats(self.xrange[2][::dx].tolist()) + else: + meta['xrange'] = [] + + meta.update(self.meta) + ret['metadata'] = meta + return json.dumps(ret) + + @property + def times(self): + ''' + Return the list of times of the current data + ''' + + ret = numpy.array([*self.data[self.key]]) + if self: + ret.sort() + return ret + + @property + def min_time(self): + ''' + Return the minimun time value + ''' + + return self.times[0] + + @property + def max_time(self): + ''' + Return the maximun time value + ''' + + return self.times[-1] + + @property + def heights(self): + ''' + Return the list of heights of the current data + ''' + + return numpy.array(self.__heights[-1]) + + @staticmethod + def roundFloats(obj): + if isinstance(obj, list): + return list(map(PlotterData.roundFloats, obj)) + elif isinstance(obj, float): + return round(obj, 2) diff --git a/schainpy/model/data/jroheaderIO.py b/schainpy/model/data/jroheaderIO.py index 2f0b4f9..4d1eeca 100644 --- a/schainpy/model/data/jroheaderIO.py +++ b/schainpy/model/data/jroheaderIO.py @@ -8,6 +8,7 @@ import numpy import copy import datetime import inspect +from schainpy.utils import log SPEED_OF_LIGHT = 299792458 SPEED_OF_LIGHT = 3e8 @@ -110,7 +111,7 @@ class Header(object): message += self.__class__.__name__.upper() + "\n" message += "#" * 50 + "\n" - keyList = self.__dict__.keys() + keyList = list(self.__dict__.keys()) keyList.sort() for key in keyList: @@ -122,7 +123,7 @@ class Header(object): if attr: message += "%s = %s" % ("size", attr) + "\n" - print message + print(message) class BasicHeader(Header): @@ -161,9 +162,9 @@ class BasicHeader(Header): header = numpy.fromfile(fp, BASIC_STRUCTURE, 1) else: header = numpy.fromstring(fp, BASIC_STRUCTURE, 1) - except Exception, e: - print "BasicHeader: " - print e + except Exception as e: + print("BasicHeader: ") + print(e) return 0 self.size = int(header['nSize'][0]) @@ -229,7 +230,7 @@ class SystemHeader(Header): self.length = 0 try: startFp = fp.tell() - except Exception, e: + except Exception as e: startFp = None pass @@ -238,8 +239,8 @@ class SystemHeader(Header): header = numpy.fromfile(fp, SYSTEM_STRUCTURE, 1) else: header = numpy.fromstring(fp, SYSTEM_STRUCTURE, 1) - except Exception, e: - print "System Header: " + str(e) + except Exception as e: + print("System Header: " + str(e)) return 0 self.size = header['nSize'][0] @@ -344,7 +345,7 @@ class RadarControllerHeader(Header): self.length = 0 try: startFp = fp.tell() - except Exception, e: + except Exception as e: startFp = None pass @@ -354,8 +355,8 @@ class RadarControllerHeader(Header): else: header = numpy.fromstring(fp, RADAR_STRUCTURE, 1) self.length += header.nbytes - except Exception, e: - print "RadarControllerHeader: " + str(e) + except Exception as e: + print("RadarControllerHeader: " + str(e)) return 0 size = int(header['nSize'][0]) @@ -384,8 +385,8 @@ class RadarControllerHeader(Header): samplingWindow = numpy.fromstring( fp[self.length:], SAMPLING_STRUCTURE, self.nWindows) self.length += samplingWindow.nbytes - except Exception, e: - print "RadarControllerHeader: " + str(e) + except Exception as e: + print("RadarControllerHeader: " + str(e)) return 0 self.nHeights = int(numpy.sum(samplingWindow['nsa'])) self.firstHeight = samplingWindow['h0'] @@ -399,8 +400,8 @@ class RadarControllerHeader(Header): self.Taus = numpy.fromstring( fp[self.length:], ' endFp: @@ -557,7 +558,7 @@ class RadarControllerHeader(Header): def set_size(self, value): - raise IOError, "size is a property and it cannot be set, just read" + raise IOError("size is a property and it cannot be set, just read") return @@ -617,7 +618,7 @@ class ProcessingHeader(Header): self.length = 0 try: startFp = fp.tell() - except Exception, e: + except Exception as e: startFp = None pass @@ -627,8 +628,8 @@ class ProcessingHeader(Header): else: header = numpy.fromstring(fp, PROCESSING_STRUCTURE, 1) self.length += header.nbytes - except Exception, e: - print "ProcessingHeader: " + str(e) + except Exception as e: + print("ProcessingHeader: " + str(e)) return 0 size = int(header['nSize'][0]) @@ -650,8 +651,8 @@ class ProcessingHeader(Header): samplingWindow = numpy.fromstring( fp[self.length:], SAMPLING_STRUCTURE, self.nWindows) self.length += samplingWindow.nbytes - except Exception, e: - print "ProcessingHeader: " + str(e) + except Exception as e: + print("ProcessingHeader: " + str(e)) return 0 self.nHeights = int(numpy.sum(samplingWindow['nsa'])) @@ -667,8 +668,8 @@ class ProcessingHeader(Header): self.spectraComb = numpy.fromstring( fp[self.length:], 'u1', 2 * self.totalSpectra) self.length += self.spectraComb.nbytes - except Exception, e: - print "ProcessingHeader: " + str(e) + except Exception as e: + print("ProcessingHeader: " + str(e)) return 0 if ((self.processFlags & PROCFLAG.DEFINE_PROCESS_CODE) == PROCFLAG.DEFINE_PROCESS_CODE): @@ -783,7 +784,7 @@ class ProcessingHeader(Header): def set_size(self, value): - raise IOError, "size is a property and it cannot be set, just read" + raise IOError("size is a property and it cannot be set, just read") return @@ -902,4 +903,4 @@ def get_procflag_dtype(index): def get_dtype_width(index): - return DTYPE_WIDTH[index] + return DTYPE_WIDTH[index] \ No newline at end of file diff --git a/schainpy/model/graphics/__init__.py b/schainpy/model/graphics/__init__.py index 4b0542b..a0d1295 100644 --- a/schainpy/model/graphics/__init__.py +++ b/schainpy/model/graphics/__init__.py @@ -1,7 +1,5 @@ -from jroplot_voltage import * -from jroplot_spectra import * -from jroplot_heispectra import * -from jroplot_correlation import * -from jroplot_parameters import * -from jroplot_data import * -from jroplotter import * +from .jroplot_voltage import * +from .jroplot_spectra import * +from .jroplot_heispectra import * +from .jroplot_correlation import * +from .jroplot_parameters import * diff --git a/schainpy/model/graphics/figure.py b/schainpy/model/graphics/figure.py deleted file mode 100644 index 85fd5bd..0000000 --- a/schainpy/model/graphics/figure.py +++ /dev/null @@ -1,657 +0,0 @@ -import os -import numpy -import time, datetime -import mpldriver - -from schainpy.model.proc.jroproc_base import Operation - -def isTimeInHourRange(datatime, xmin, xmax): - - if xmin == None or xmax == None: - return 1 - hour = datatime.hour + datatime.minute/60.0 - - if xmin < (xmax % 24): - - if hour >= xmin and hour <= xmax: - return 1 - else: - return 0 - - else: - - if hour >= xmin or hour <= (xmax % 24): - return 1 - else: - return 0 - - return 0 - -def isRealtime(utcdatatime): - - utcnow = time.mktime(time.localtime()) - delta = abs(utcnow - utcdatatime) # abs - if delta >= 30.: - return False - return True - -class Figure(Operation): - - __driver = mpldriver - fig = None - - id = None - wintitle = None - width = None - height = None - nplots = None - timerange = None - - axesObjList = [] - - WIDTH = 300 - HEIGHT = 200 - PREFIX = 'fig' - - xmin = None - xmax = None - - counter_imagwr = 0 - - figfile = None - - created = False - parameters = {} - def __init__(self, **kwargs): - - Operation.__init__(self, **kwargs) - - def __del__(self): - - self.__driver.closeFigure() - - def getFilename(self, name, ext='.png'): - - path = '%s%03d' %(self.PREFIX, self.id) - filename = '%s_%s%s' %(self.PREFIX, name, ext) - return os.path.join(path, filename) - - def getAxesObjList(self): - - return self.axesObjList - - def getSubplots(self): - - raise NotImplementedError - - def getScreenDim(self, widthplot, heightplot): - - nrow, ncol = self.getSubplots() - - widthscreen = widthplot*ncol - heightscreen = heightplot*nrow - - return widthscreen, heightscreen - - def getTimeLim(self, x, xmin=None, xmax=None, timerange=None): - -# if self.xmin != None and self.xmax != None: -# if timerange == None: -# timerange = self.xmax - self.xmin -# xmin = self.xmin + timerange -# xmax = self.xmax + timerange -# -# return xmin, xmax - - if timerange == None and (xmin==None or xmax==None): - timerange = 14400 #seconds - - if timerange != None: - txmin = x[0] #- x[0] % min(timerange/10, 10*60) - else: - txmin = x[0] #- x[0] % 10*60 - - thisdatetime = datetime.datetime.utcfromtimestamp(txmin) - thisdate = datetime.datetime.combine(thisdatetime.date(), datetime.time(0,0,0)) - - if timerange != None: - xmin = (thisdatetime - thisdate).seconds/(60*60.) - xmax = xmin + timerange/(60*60.) - - d1970 = datetime.datetime(1970,1,1) - - mindt = thisdate + datetime.timedelta(hours=xmin) #- datetime.timedelta(seconds=time.timezone) - xmin_sec = (mindt - d1970).total_seconds() #time.mktime(mindt.timetuple()) - time.timezone - - maxdt = thisdate + datetime.timedelta(hours=xmax) #- datetime.timedelta(seconds=time.timezone) - xmax_sec = (maxdt - d1970).total_seconds() #time.mktime(maxdt.timetuple()) - time.timezone - - return xmin_sec, xmax_sec - - def init(self, id, nplots, wintitle): - - raise NotImplementedError, "This method has been replaced by createFigure" - - def createFigure(self, id, wintitle, widthplot=None, heightplot=None, show=True): - - """ - Crea la figura de acuerdo al driver y parametros seleccionados seleccionados. - Las dimensiones de la pantalla es calculada a partir de los atributos self.WIDTH - y self.HEIGHT y el numero de subplots (nrow, ncol) - - Input: - id : Los parametros necesarios son - wintitle : - - """ - - if widthplot == None: - widthplot = self.WIDTH - - if heightplot == None: - heightplot = self.HEIGHT - - self.id = id - - self.wintitle = wintitle - - self.widthscreen, self.heightscreen = self.getScreenDim(widthplot, heightplot) - -# if self.created: -# self.__driver.closeFigure(self.fig) - - if not self.created: - self.fig = self.__driver.createFigure(id=self.id, - wintitle=self.wintitle, - width=self.widthscreen, - height=self.heightscreen, - show=show) - else: - self.__driver.clearFigure(self.fig) - - self.axesObjList = [] - self.counter_imagwr = 0 - - self.created = True - - def setDriver(self, driver=mpldriver): - - self.__driver = driver - - def setTitle(self, title): - - self.__driver.setTitle(self.fig, title) - - def setWinTitle(self, title): - - self.__driver.setWinTitle(self.fig, title=title) - - def setTextFromAxes(self, text): - - raise NotImplementedError, "This method has been replaced with Axes.setText" - - def makeAxes(self, nrow, ncol, xpos, ypos, colspan, rowspan): - - raise NotImplementedError, "This method has been replaced with Axes.addAxes" - - def addAxes(self, *args): - """ - - Input: - *args : Los parametros necesarios son - nrow, ncol, xpos, ypos, colspan, rowspan - """ - - axesObj = Axes(self.fig, *args) - self.axesObjList.append(axesObj) - - def saveFigure(self, figpath, figfile, *args): - - filename = os.path.join(figpath, figfile) - - fullpath = os.path.split(filename)[0] - - if not os.path.exists(fullpath): - subpath = os.path.split(fullpath)[0] - - if not os.path.exists(subpath): - os.mkdir(subpath) - - os.mkdir(fullpath) - - self.__driver.saveFigure(self.fig, filename, *args) - - def save(self, figpath, figfile=None, save=True, ftp=False, wr_period=1, thisDatetime=None, update_figfile=True): - - self.counter_imagwr += 1 - if self.counter_imagwr < wr_period: - return - - self.counter_imagwr = 0 - - if save: - - if not figfile: - - if not thisDatetime: - raise ValueError, "Saving figure: figfile or thisDatetime should be defined" - return - - str_datetime = thisDatetime.strftime("%Y%m%d_%H%M%S") - figfile = self.getFilename(name = str_datetime) - - if self.figfile == None: - self.figfile = figfile - - if update_figfile: - self.figfile = figfile - - # store png plot to local folder - self.saveFigure(figpath, self.figfile) - - - if not ftp: - return - - if not thisDatetime: - return - - # store png plot to FTP server according to RT-Web format - ftp_filename = self.getNameToFtp(thisDatetime, self.FTP_WEI, self.EXP_CODE, self.SUB_EXP_CODE, self.PLOT_CODE, self.PLOT_POS) -# ftp_filename = os.path.join(figpath, name) - self.saveFigure(figpath, ftp_filename) - - def getNameToFtp(self, thisDatetime, FTP_WEI, EXP_CODE, SUB_EXP_CODE, PLOT_CODE, PLOT_POS): - YEAR_STR = '%4.4d'%thisDatetime.timetuple().tm_year - DOY_STR = '%3.3d'%thisDatetime.timetuple().tm_yday - FTP_WEI = '%2.2d'%FTP_WEI - EXP_CODE = '%3.3d'%EXP_CODE - SUB_EXP_CODE = '%2.2d'%SUB_EXP_CODE - PLOT_CODE = '%2.2d'%PLOT_CODE - PLOT_POS = '%2.2d'%PLOT_POS - name = YEAR_STR + DOY_STR + FTP_WEI + EXP_CODE + SUB_EXP_CODE + PLOT_CODE + PLOT_POS - return name - - def draw(self): - - self.__driver.draw(self.fig) - - def run(self): - - raise NotImplementedError - - def close(self, show=False): - - self.__driver.closeFigure(show=show, fig=self.fig) - - axesList = property(getAxesObjList) - - -class Axes: - - __driver = mpldriver - fig = None - ax = None - plot = None - __missing = 1E30 - __firsttime = None - - __showprofile = False - - xmin = None - xmax = None - ymin = None - ymax = None - zmin = None - zmax = None - - x_buffer = None - z_buffer = None - - decimationx = None - decimationy = None - - __MAXNUMX = 200 - __MAXNUMY = 400 - - __MAXNUMTIME = 500 - - def __init__(self, *args): - - """ - - Input: - *args : Los parametros necesarios son - fig, nrow, ncol, xpos, ypos, colspan, rowspan - """ - - ax = self.__driver.createAxes(*args) - self.fig = args[0] - self.ax = ax - self.plot = None - - self.__firsttime = True - self.idlineList = [] - - self.x_buffer = numpy.array([]) - self.z_buffer = numpy.array([]) - - def setText(self, text): - - self.__driver.setAxesText(self.ax, text) - - def setXAxisAsTime(self): - pass - - def pline(self, x, y, - xmin=None, xmax=None, - ymin=None, ymax=None, - xlabel='', ylabel='', - title='', - **kwargs): - - """ - - Input: - x : - y : - xmin : - xmax : - ymin : - ymax : - xlabel : - ylabel : - title : - **kwargs : Los parametros aceptados son - - ticksize - ytick_visible - """ - - if self.__firsttime: - - if xmin == None: xmin = numpy.nanmin(x) - if xmax == None: xmax = numpy.nanmax(x) - if ymin == None: ymin = numpy.nanmin(y) - if ymax == None: ymax = numpy.nanmax(y) - - self.plot = self.__driver.createPline(self.ax, x, y, - xmin, xmax, - ymin, ymax, - xlabel=xlabel, - ylabel=ylabel, - title=title, - **kwargs) - - self.idlineList.append(0) - self.__firsttime = False - return - - self.__driver.pline(self.plot, x, y, xlabel=xlabel, - ylabel=ylabel, - title=title) - -# self.__driver.pause() - - def addpline(self, x, y, idline, **kwargs): - lines = self.ax.lines - - if idline in self.idlineList: - self.__driver.set_linedata(self.ax, x, y, idline) - - if idline not in(self.idlineList): - self.__driver.addpline(self.ax, x, y, **kwargs) - self.idlineList.append(idline) - - return - - def pmultiline(self, x, y, - xmin=None, xmax=None, - ymin=None, ymax=None, - xlabel='', ylabel='', - title='', - **kwargs): - - if self.__firsttime: - - if xmin == None: xmin = numpy.nanmin(x) - if xmax == None: xmax = numpy.nanmax(x) - if ymin == None: ymin = numpy.nanmin(y) - if ymax == None: ymax = numpy.nanmax(y) - - self.plot = self.__driver.createPmultiline(self.ax, x, y, - xmin, xmax, - ymin, ymax, - xlabel=xlabel, - ylabel=ylabel, - title=title, - **kwargs) - self.__firsttime = False - return - - self.__driver.pmultiline(self.plot, x, y, xlabel=xlabel, - ylabel=ylabel, - title=title) - -# self.__driver.pause() - - def pmultilineyaxis(self, x, y, - xmin=None, xmax=None, - ymin=None, ymax=None, - xlabel='', ylabel='', - title='', - **kwargs): - - if self.__firsttime: - - if xmin == None: xmin = numpy.nanmin(x) - if xmax == None: xmax = numpy.nanmax(x) - if ymin == None: ymin = numpy.nanmin(y) - if ymax == None: ymax = numpy.nanmax(y) - - self.plot = self.__driver.createPmultilineYAxis(self.ax, x, y, - xmin, xmax, - ymin, ymax, - xlabel=xlabel, - ylabel=ylabel, - title=title, - **kwargs) - if self.xmin == None: self.xmin = xmin - if self.xmax == None: self.xmax = xmax - if self.ymin == None: self.ymin = ymin - if self.ymax == None: self.ymax = ymax - - self.__firsttime = False - return - - self.__driver.pmultilineyaxis(self.plot, x, y, xlabel=xlabel, - ylabel=ylabel, - title=title) - -# self.__driver.pause() - - def pcolor(self, x, y, z, - xmin=None, xmax=None, - ymin=None, ymax=None, - zmin=None, zmax=None, - xlabel='', ylabel='', - title='', colormap='jet', - **kwargs): - - """ - Input: - x : - y : - x : - xmin : - xmax : - ymin : - ymax : - zmin : - zmax : - xlabel : - ylabel : - title : - **kwargs : Los parametros aceptados son - ticksize=9, - cblabel='' - """ - - #Decimating data - xlen = len(x) - ylen = len(y) - - decimationx = int(xlen/self.__MAXNUMX) + 1 - decimationy = int(ylen/self.__MAXNUMY) + 1 - - - x_buffer = x#[::decimationx] - y_buffer = y#[::decimationy] - z_buffer = z#[::decimationx, ::decimationy] - #=================================================== - - if self.__firsttime: - - if xmin == None: xmin = numpy.nanmin(x) - if xmax == None: xmax = numpy.nanmax(x) - if ymin == None: ymin = numpy.nanmin(y) - if ymax == None: ymax = numpy.nanmax(y) - if zmin == None: zmin = numpy.nanmin(z) - if zmax == None: zmax = numpy.nanmax(z) - - - self.plot = self.__driver.createPcolor(self.ax, x_buffer, - y_buffer, - z_buffer, - xmin, xmax, - ymin, ymax, - zmin, zmax, - xlabel=xlabel, - ylabel=ylabel, - title=title, - colormap=colormap, - **kwargs) - - if self.xmin == None: self.xmin = xmin - if self.xmax == None: self.xmax = xmax - if self.ymin == None: self.ymin = ymin - if self.ymax == None: self.ymax = ymax - if self.zmin == None: self.zmin = zmin - if self.zmax == None: self.zmax = zmax - - self.__firsttime = False - return - - self.__driver.pcolor(self.plot, - z_buffer, - xlabel=xlabel, - ylabel=ylabel, - title=title) - -# self.__driver.pause() - - def pcolorbuffer(self, x, y, z, - xmin=None, xmax=None, - ymin=None, ymax=None, - zmin=None, zmax=None, - xlabel='', ylabel='', - title='', rti = True, colormap='jet', - maxNumX = None, maxNumY = None, - **kwargs): - - if maxNumX == None: - maxNumX = self.__MAXNUMTIME - - if maxNumY == None: - maxNumY = self.__MAXNUMY - - if self.__firsttime: - self.z_buffer = z - self.x_buffer = numpy.hstack((self.x_buffer, x)) - - if xmin == None: xmin = numpy.nanmin(x) - if xmax == None: xmax = numpy.nanmax(x) - if ymin == None: ymin = numpy.nanmin(y) - if ymax == None: ymax = numpy.nanmax(y) - if zmin == None: zmin = numpy.nanmin(z) - if zmax == None: zmax = numpy.nanmax(z) - - self.plot = self.__driver.createPcolor(self.ax, self.x_buffer, y, z, - xmin, xmax, - ymin, ymax, - zmin, zmax, - xlabel=xlabel, - ylabel=ylabel, - title=title, - colormap=colormap, - **kwargs) - - if self.xmin == None: self.xmin = xmin - if self.xmax == None: self.xmax = xmax - if self.ymin == None: self.ymin = ymin - if self.ymax == None: self.ymax = ymax - if self.zmin == None: self.zmin = zmin - if self.zmax == None: self.zmax = zmax - - self.__firsttime = False - return - - self.x_buffer = numpy.hstack((self.x_buffer[:-1], x[0], x[-1])) - self.z_buffer = numpy.hstack((self.z_buffer, z)) - z_buffer = self.z_buffer.reshape(-1,len(y)) - - #Decimating data - xlen = len(self.x_buffer) - ylen = len(y) - - decimationx = int(xlen/maxNumX) + 1 - decimationy = int(ylen/maxNumY) + 1 - - x_buffer = self.x_buffer#[::decimationx] - y_buffer = y#[::decimationy] - z_buffer = z_buffer#[::decimationx, ::decimationy] - #=================================================== - - x_buffer, y_buffer, z_buffer = self.__fillGaps(x_buffer, y_buffer, z_buffer) - - self.__driver.addpcolorbuffer(self.ax, x_buffer, y_buffer, z_buffer, self.zmin, self.zmax, - xlabel=xlabel, - ylabel=ylabel, - title=title, - colormap=colormap) - -# self.__driver.pause() - - def polar(self, x, y, - title='', xlabel='',ylabel='',**kwargs): - - if self.__firsttime: - self.plot = self.__driver.createPolar(self.ax, x, y, title = title, xlabel = xlabel, ylabel = ylabel) - self.__firsttime = False - self.x_buffer = x - self.y_buffer = y - return - - self.x_buffer = numpy.hstack((self.x_buffer,x)) - self.y_buffer = numpy.hstack((self.y_buffer,y)) - self.__driver.polar(self.plot, self.x_buffer, self.y_buffer, xlabel=xlabel, - ylabel=ylabel, - title=title) - -# self.__driver.pause() - - def __fillGaps(self, x_buffer, y_buffer, z_buffer): - - if x_buffer.shape[0] < 2: - return x_buffer, y_buffer, z_buffer - - deltas = x_buffer[1:] - x_buffer[0:-1] - x_median = numpy.median(deltas) - - index = numpy.where(deltas > 5*x_median) - - if len(index[0]) != 0: - z_buffer[index[0],::] = self.__missing - z_buffer = numpy.ma.masked_inside(z_buffer,0.99*self.__missing,1.01*self.__missing) - - return x_buffer, y_buffer, z_buffer diff --git a/schainpy/model/graphics/jroplot_base.py b/schainpy/model/graphics/jroplot_base.py new file mode 100644 index 0000000..f3582ce --- /dev/null +++ b/schainpy/model/graphics/jroplot_base.py @@ -0,0 +1,713 @@ + +import os +import sys +import zmq +import time +import numpy +import datetime +from queue import Queue +from functools import wraps +from threading import Thread +import matplotlib + +if 'BACKEND' in os.environ: + matplotlib.use(os.environ['BACKEND']) +elif 'linux' in sys.platform: + matplotlib.use("TkAgg") +elif 'darwin' in sys.platform: + matplotlib.use('WxAgg') +else: + from schainpy.utils import log + log.warning('Using default Backend="Agg"', 'INFO') + matplotlib.use('Agg') + +import matplotlib.pyplot as plt +from matplotlib.patches import Polygon +from mpl_toolkits.axes_grid1 import make_axes_locatable +from matplotlib.ticker import FuncFormatter, LinearLocator, MultipleLocator + +from schainpy.model.data.jrodata import PlotterData +from schainpy.model.proc.jroproc_base import ProcessingUnit, Operation, MPDecorator +from schainpy.utils import log + +jet_values = matplotlib.pyplot.get_cmap('jet', 100)(numpy.arange(100))[10:90] +blu_values = matplotlib.pyplot.get_cmap( + 'seismic_r', 20)(numpy.arange(20))[10:15] +ncmap = matplotlib.colors.LinearSegmentedColormap.from_list( + 'jro', numpy.vstack((blu_values, jet_values))) +matplotlib.pyplot.register_cmap(cmap=ncmap) + +CMAPS = [plt.get_cmap(s) for s in ('jro', 'jet', 'viridis', + 'plasma', 'inferno', 'Greys', 'seismic', 'bwr', 'coolwarm')] + +EARTH_RADIUS = 6.3710e3 + +def ll2xy(lat1, lon1, lat2, lon2): + + p = 0.017453292519943295 + a = 0.5 - numpy.cos((lat2 - lat1) * p)/2 + numpy.cos(lat1 * p) * \ + numpy.cos(lat2 * p) * (1 - numpy.cos((lon2 - lon1) * p)) / 2 + r = 12742 * numpy.arcsin(numpy.sqrt(a)) + theta = numpy.arctan2(numpy.sin((lon2-lon1)*p)*numpy.cos(lat2*p), numpy.cos(lat1*p) + * numpy.sin(lat2*p)-numpy.sin(lat1*p)*numpy.cos(lat2*p)*numpy.cos((lon2-lon1)*p)) + theta = -theta + numpy.pi/2 + return r*numpy.cos(theta), r*numpy.sin(theta) + + +def km2deg(km): + ''' + Convert distance in km to degrees + ''' + + return numpy.rad2deg(km/EARTH_RADIUS) + + +def figpause(interval): + backend = plt.rcParams['backend'] + if backend in matplotlib.rcsetup.interactive_bk: + figManager = matplotlib._pylab_helpers.Gcf.get_active() + if figManager is not None: + canvas = figManager.canvas + if canvas.figure.stale: + canvas.draw() + try: + canvas.start_event_loop(interval) + except: + pass + return + + +def popup(message): + ''' + ''' + + fig = plt.figure(figsize=(12, 8), facecolor='r') + text = '\n'.join([s.strip() for s in message.split(':')]) + fig.text(0.01, 0.5, text, ha='left', va='center', + size='20', weight='heavy', color='w') + fig.show() + figpause(1000) + + +class Throttle(object): + ''' + Decorator that prevents a function from being called more than once every + time period. + To create a function that cannot be called more than once a minute, but + will sleep until it can be called: + @Throttle(minutes=1) + def foo(): + pass + + for i in range(10): + foo() + print "This function has run %s times." % i + ''' + + def __init__(self, seconds=0, minutes=0, hours=0): + self.throttle_period = datetime.timedelta( + seconds=seconds, minutes=minutes, hours=hours + ) + + self.time_of_last_call = datetime.datetime.min + + def __call__(self, fn): + @wraps(fn) + def wrapper(*args, **kwargs): + coerce = kwargs.pop('coerce', None) + if coerce: + self.time_of_last_call = datetime.datetime.now() + return fn(*args, **kwargs) + else: + now = datetime.datetime.now() + time_since_last_call = now - self.time_of_last_call + time_left = self.throttle_period - time_since_last_call + + if time_left > datetime.timedelta(seconds=0): + return + + self.time_of_last_call = datetime.datetime.now() + return fn(*args, **kwargs) + + return wrapper + +def apply_throttle(value): + + @Throttle(seconds=value) + def fnThrottled(fn): + fn() + + return fnThrottled + + +@MPDecorator +class Plot(Operation): + ''' + Base class for Schain plotting operations + ''' + + CODE = 'Figure' + colormap = 'jet' + bgcolor = 'white' + buffering = True + __missing = 1E30 + + __attrs__ = ['show', 'save', 'ymin', 'ymax', 'zmin', 'zmax', 'title', + 'showprofile'] + + def __init__(self): + + Operation.__init__(self) + self.isConfig = False + self.isPlotConfig = False + self.save_counter = 1 + self.sender_time = 0 + self.data = None + self.firsttime = True + self.sender_queue = Queue(maxsize=60) + self.plots_adjust = {'left': 0.125, 'right': 0.9, 'bottom': 0.15, 'top': 0.9, 'wspace': 0.2, 'hspace': 0.2} + + def __fmtTime(self, x, pos): + ''' + ''' + + return '{}'.format(self.getDateTime(x).strftime('%H:%M')) + + def __setup(self, **kwargs): + ''' + Initialize variables + ''' + + self.figures = [] + self.axes = [] + self.cb_axes = [] + self.localtime = kwargs.pop('localtime', True) + self.show = kwargs.get('show', True) + self.save = kwargs.get('save', False) + self.save_period = kwargs.get('save_period', 1) + self.colormap = kwargs.get('colormap', self.colormap) + self.colormap_coh = kwargs.get('colormap_coh', 'jet') + self.colormap_phase = kwargs.get('colormap_phase', 'RdBu_r') + self.colormaps = kwargs.get('colormaps', None) + self.bgcolor = kwargs.get('bgcolor', self.bgcolor) + self.showprofile = kwargs.get('showprofile', False) + self.title = kwargs.get('wintitle', self.CODE.upper()) + self.cb_label = kwargs.get('cb_label', None) + self.cb_labels = kwargs.get('cb_labels', None) + self.labels = kwargs.get('labels', None) + self.xaxis = kwargs.get('xaxis', 'frequency') + self.zmin = kwargs.get('zmin', None) + self.zmax = kwargs.get('zmax', None) + self.zlimits = kwargs.get('zlimits', None) + self.xmin = kwargs.get('xmin', None) + self.xmax = kwargs.get('xmax', None) + self.xrange = kwargs.get('xrange', 12) + self.xscale = kwargs.get('xscale', None) + self.ymin = kwargs.get('ymin', None) + self.ymax = kwargs.get('ymax', None) + self.yscale = kwargs.get('yscale', None) + self.xlabel = kwargs.get('xlabel', None) + self.attr_time = kwargs.get('attr_time', 'utctime') + self.decimation = kwargs.get('decimation', None) + self.showSNR = kwargs.get('showSNR', False) + self.oneFigure = kwargs.get('oneFigure', True) + self.width = kwargs.get('width', None) + self.height = kwargs.get('height', None) + self.colorbar = kwargs.get('colorbar', True) + self.factors = kwargs.get('factors', [1, 1, 1, 1, 1, 1, 1, 1]) + self.channels = kwargs.get('channels', None) + self.titles = kwargs.get('titles', []) + self.polar = False + self.type = kwargs.get('type', 'iq') + self.grid = kwargs.get('grid', False) + self.pause = kwargs.get('pause', False) + self.save_code = kwargs.get('save_code', None) + self.throttle = kwargs.get('throttle', 0) + self.exp_code = kwargs.get('exp_code', None) + self.plot_server = kwargs.get('plot_server', False) + self.sender_period = kwargs.get('sender_period', 60) + self.tag = kwargs.get('tag', '') + self.height_index = kwargs.get('height_index', None) + self.__throttle_plot = apply_throttle(self.throttle) + self.data = PlotterData( + self.CODE, self.throttle, self.exp_code, self.localtime, self.buffering, snr=self.showSNR) + + if self.plot_server: + if not self.plot_server.startswith('tcp://'): + self.plot_server = 'tcp://{}'.format(self.plot_server) + log.success( + 'Sending to server: {}'.format(self.plot_server), + self.name + ) + if 'plot_name' in kwargs: + self.plot_name = kwargs['plot_name'] + + def __setup_plot(self): + ''' + Common setup for all figures, here figures and axes are created + ''' + + self.setup() + + self.time_label = 'LT' if self.localtime else 'UTC' + + if self.width is None: + self.width = 8 + + self.figures = [] + self.axes = [] + self.cb_axes = [] + self.pf_axes = [] + self.cmaps = [] + + size = '15%' if self.ncols == 1 else '30%' + pad = '4%' if self.ncols == 1 else '8%' + + if self.oneFigure: + if self.height is None: + self.height = 1.4 * self.nrows + 1 + fig = plt.figure(figsize=(self.width, self.height), + edgecolor='k', + facecolor='w') + self.figures.append(fig) + for n in range(self.nplots): + ax = fig.add_subplot(self.nrows, self.ncols, + n + 1, polar=self.polar) + ax.tick_params(labelsize=8) + ax.firsttime = True + ax.index = 0 + ax.press = None + self.axes.append(ax) + if self.showprofile: + cax = self.__add_axes(ax, size=size, pad=pad) + cax.tick_params(labelsize=8) + self.pf_axes.append(cax) + else: + if self.height is None: + self.height = 3 + for n in range(self.nplots): + fig = plt.figure(figsize=(self.width, self.height), + edgecolor='k', + facecolor='w') + ax = fig.add_subplot(1, 1, 1, polar=self.polar) + ax.tick_params(labelsize=8) + ax.firsttime = True + ax.index = 0 + ax.press = None + self.figures.append(fig) + self.axes.append(ax) + if self.showprofile: + cax = self.__add_axes(ax, size=size, pad=pad) + cax.tick_params(labelsize=8) + self.pf_axes.append(cax) + + for n in range(self.nrows): + if self.colormaps is not None: + cmap = plt.get_cmap(self.colormaps[n]) + else: + cmap = plt.get_cmap(self.colormap) + cmap.set_bad(self.bgcolor, 1.) + self.cmaps.append(cmap) + + def __add_axes(self, ax, size='30%', pad='8%'): + ''' + Add new axes to the given figure + ''' + divider = make_axes_locatable(ax) + nax = divider.new_horizontal(size=size, pad=pad) + ax.figure.add_axes(nax) + return nax + + def fill_gaps(self, x_buffer, y_buffer, z_buffer): + ''' + Create a masked array for missing data + ''' + if x_buffer.shape[0] < 2: + return x_buffer, y_buffer, z_buffer + + deltas = x_buffer[1:] - x_buffer[0:-1] + x_median = numpy.median(deltas) + + index = numpy.where(deltas > 5 * x_median) + + if len(index[0]) != 0: + z_buffer[::, index[0], ::] = self.__missing + z_buffer = numpy.ma.masked_inside(z_buffer, + 0.99 * self.__missing, + 1.01 * self.__missing) + + return x_buffer, y_buffer, z_buffer + + def decimate(self): + + # dx = int(len(self.x)/self.__MAXNUMX) + 1 + dy = int(len(self.y) / self.decimation) + 1 + + # x = self.x[::dx] + x = self.x + y = self.y[::dy] + z = self.z[::, ::, ::dy] + + return x, y, z + + def format(self): + ''' + Set min and max values, labels, ticks and titles + ''' + + if self.xmin is None: + xmin = self.data.min_time + else: + if self.xaxis is 'time': + dt = self.getDateTime(self.data.min_time) + xmin = (dt.replace(hour=int(self.xmin), minute=0, second=0) - + datetime.datetime(1970, 1, 1)).total_seconds() + if self.data.localtime: + xmin += time.timezone + else: + xmin = self.xmin + + if self.xmax is None: + xmax = xmin + self.xrange * 60 * 60 + else: + if self.xaxis is 'time': + dt = self.getDateTime(self.data.max_time) + xmax = self.xmax - 1 + xmax = (dt.replace(hour=int(xmax), minute=59, second=59) - + datetime.datetime(1970, 1, 1) + datetime.timedelta(seconds=1)).total_seconds() + if self.data.localtime: + xmax += time.timezone + else: + xmax = self.xmax + + ymin = self.ymin if self.ymin else numpy.nanmin(self.y) + ymax = self.ymax if self.ymax else numpy.nanmax(self.y) + + for n, ax in enumerate(self.axes): + if ax.firsttime: + + dig = int(numpy.log10(ymax)) + if dig == 0: + digD = len(str(ymax)) - 2 + ydec = ymax*(10**digD) + + dig = int(numpy.log10(ydec)) + ystep = ((ydec + (10**(dig)))//10**(dig))*(10**(dig)) + ystep = ystep/5 + ystep = ystep/(10**digD) + + else: + ystep = ((ymax + (10**(dig)))//10**(dig))*(10**(dig)) + ystep = ystep/5 + + if self.xaxis is not 'time': + + dig = int(numpy.log10(xmax)) + + if dig <= 0: + digD = len(str(xmax)) - 2 + xdec = xmax*(10**digD) + + dig = int(numpy.log10(xdec)) + xstep = ((xdec + (10**(dig)))//10**(dig))*(10**(dig)) + xstep = xstep*0.5 + xstep = xstep/(10**digD) + + else: + xstep = ((xmax + (10**(dig)))//10**(dig))*(10**(dig)) + xstep = xstep/5 + + ax.set_facecolor(self.bgcolor) + ax.yaxis.set_major_locator(MultipleLocator(ystep)) + if self.xscale: + ax.xaxis.set_major_formatter(FuncFormatter( + lambda x, pos: '{0:g}'.format(x*self.xscale))) + if self.xscale: + ax.yaxis.set_major_formatter(FuncFormatter( + lambda x, pos: '{0:g}'.format(x*self.yscale))) + if self.xaxis is 'time': + ax.xaxis.set_major_formatter(FuncFormatter(self.__fmtTime)) + ax.xaxis.set_major_locator(LinearLocator(9)) + else: + ax.xaxis.set_major_locator(MultipleLocator(xstep)) + if self.xlabel is not None: + ax.set_xlabel(self.xlabel) + ax.set_ylabel(self.ylabel) + ax.firsttime = False + if self.showprofile: + self.pf_axes[n].set_ylim(ymin, ymax) + self.pf_axes[n].set_xlim(self.zmin, self.zmax) + self.pf_axes[n].set_xlabel('dB') + self.pf_axes[n].grid(b=True, axis='x') + [tick.set_visible(False) + for tick in self.pf_axes[n].get_yticklabels()] + if self.colorbar: + ax.cbar = plt.colorbar( + ax.plt, ax=ax, fraction=0.05, pad=0.02, aspect=10) + ax.cbar.ax.tick_params(labelsize=8) + ax.cbar.ax.press = None + if self.cb_label: + ax.cbar.set_label(self.cb_label, size=8) + elif self.cb_labels: + ax.cbar.set_label(self.cb_labels[n], size=8) + else: + ax.cbar = None + if self.grid: + ax.grid(True) + + if not self.polar: + ax.set_xlim(xmin, xmax) + ax.set_ylim(ymin, ymax) + ax.set_title('{} {} {}'.format( + self.titles[n], + self.getDateTime(self.data.max_time).strftime( + '%Y-%m-%d %H:%M:%S'), + self.time_label), + size=8) + else: + ax.set_title('{}'.format(self.titles[n]), size=8) + ax.set_ylim(0, 90) + ax.set_yticks(numpy.arange(0, 90, 20)) + ax.yaxis.labelpad = 40 + + if self.firsttime: + for n, fig in enumerate(self.figures): + fig.subplots_adjust(**self.plots_adjust) + self.firsttime = False + + def clear_figures(self): + ''' + Reset axes for redraw plots + ''' + + for ax in self.axes+self.pf_axes+self.cb_axes: + ax.clear() + ax.firsttime = True + if hasattr(ax, 'cbar') and ax.cbar: + ax.cbar.remove() + + def __plot(self): + ''' + Main function to plot, format and save figures + ''' + + self.plot() + self.format() + + for n, fig in enumerate(self.figures): + if self.nrows == 0 or self.nplots == 0: + log.warning('No data', self.name) + fig.text(0.5, 0.5, 'No Data', fontsize='large', ha='center') + fig.canvas.manager.set_window_title(self.CODE) + continue + + fig.canvas.manager.set_window_title('{} - {}'.format(self.title, + self.getDateTime(self.data.max_time).strftime('%Y/%m/%d'))) + fig.canvas.draw() + if self.show: + fig.show() + figpause(0.01) + + if self.save: + self.save_figure(n) + + if self.plot_server: + self.send_to_server() + + def save_figure(self, n): + ''' + ''' + + if self.save_counter < self.save_period: + self.save_counter += 1 + return + + self.save_counter = 1 + + fig = self.figures[n] + + if self.save_code: + if isinstance(self.save_code, str): + labels = [self.save_code for x in self.figures] + else: + labels = self.save_code + else: + labels = [self.CODE for x in self.figures] + + figname = os.path.join( + self.save, + labels[n], + '{}_{}.png'.format( + labels[n], + self.getDateTime(self.data.max_time).strftime( + '%Y%m%d_%H%M%S' + ), + ) + ) + log.log('Saving figure: {}'.format(figname), self.name) + if not os.path.isdir(os.path.dirname(figname)): + os.makedirs(os.path.dirname(figname)) + fig.savefig(figname) + + if self.throttle == 0: + figname = os.path.join( + self.save, + '{}_{}.png'.format( + labels[n], + self.getDateTime(self.data.min_time).strftime( + '%Y%m%d' + ), + ) + ) + fig.savefig(figname) + + def send_to_server(self): + ''' + ''' + + interval = self.data.tm - self.sender_time + if interval < self.sender_period: + return + + self.sender_time = self.data.tm + + attrs = ['titles', 'zmin', 'zmax', 'tag', 'ymin', 'ymax'] + for attr in attrs: + value = getattr(self, attr) + if value: + if isinstance(value, (numpy.float32, numpy.float64)): + value = round(float(value), 2) + self.data.meta[attr] = value + if self.colormap == 'jet': + self.data.meta['colormap'] = 'Jet' + elif 'RdBu' in self.colormap: + self.data.meta['colormap'] = 'RdBu' + else: + self.data.meta['colormap'] = 'Viridis' + self.data.meta['interval'] = int(interval) + # msg = self.data.jsonify(self.data.tm, self.plot_name, self.plot_type) + try: + self.sender_queue.put(self.data.tm, block=False) + except: + tm = self.sender_queue.get() + self.sender_queue.put(self.data.tm) + + while True: + if self.sender_queue.empty(): + break + tm = self.sender_queue.get() + try: + msg = self.data.jsonify(tm, self.plot_name, self.plot_type) + except: + continue + self.socket.send_string(msg) + socks = dict(self.poll.poll(5000)) + if socks.get(self.socket) == zmq.POLLIN: + reply = self.socket.recv_string() + if reply == 'ok': + log.log("Response from server ok", self.name) + time.sleep(0.2) + continue + else: + log.warning( + "Malformed reply from server: {}".format(reply), self.name) + else: + log.warning( + "No response from server, retrying...", self.name) + self.sender_queue.put(self.data.tm) + self.socket.setsockopt(zmq.LINGER, 0) + self.socket.close() + self.poll.unregister(self.socket) + time.sleep(0.1) + self.socket = self.context.socket(zmq.REQ) + self.socket.connect(self.plot_server) + self.poll.register(self.socket, zmq.POLLIN) + break + + def setup(self): + ''' + This method should be implemented in the child class, the following + attributes should be set: + + self.nrows: number of rows + self.ncols: number of cols + self.nplots: number of plots (channels or pairs) + self.ylabel: label for Y axes + self.titles: list of axes title + + ''' + raise NotImplementedError + + def plot(self): + ''' + Must be defined in the child class + ''' + raise NotImplementedError + + def run(self, dataOut, **kwargs): + ''' + Main plotting routine + ''' + + if self.isConfig is False: + self.__setup(**kwargs) + + t = getattr(dataOut, self.attr_time) + + if self.localtime: + self.getDateTime = datetime.datetime.fromtimestamp + else: + self.getDateTime = datetime.datetime.utcfromtimestamp + + if self.xmin is None: + self.tmin = t + if 'buffer' in self.plot_type: + self.xmin = self.getDateTime(t).hour + else: + self.tmin = ( + self.getDateTime(t).replace( + hour=int(self.xmin), + minute=0, + second=0) - self.getDateTime(0)).total_seconds() + + self.data.setup() + self.isConfig = True + if self.plot_server: + self.context = zmq.Context() + self.socket = self.context.socket(zmq.REQ) + self.socket.connect(self.plot_server) + self.poll = zmq.Poller() + self.poll.register(self.socket, zmq.POLLIN) + + tm = getattr(dataOut, self.attr_time) + + if self.data and (tm - self.tmin) >= self.xrange*60*60: + self.save_counter = self.save_period + self.__plot() + if 'time' in self.xaxis: + self.xmin += self.xrange + if self.xmin >= 24: + self.xmin -= 24 + self.tmin += self.xrange*60*60 + self.data.setup() + self.clear_figures() + + self.data.update(dataOut, tm) + + if self.isPlotConfig is False: + self.__setup_plot() + self.isPlotConfig = True + + if self.throttle == 0: + self.__plot() + else: + self.__throttle_plot(self.__plot)#, coerce=coerce) + + def close(self): + + if self.data and not self.data.flagNoData: + self.save_counter = self.save_period + self.__plot() + if self.data and not self.data.flagNoData and self.pause: + figpause(10) + diff --git a/schainpy/model/graphics/jroplot_correlation.py b/schainpy/model/graphics/jroplot_correlation.py index 759f8c6..dd97d76 100644 --- a/schainpy/model/graphics/jroplot_correlation.py +++ b/schainpy/model/graphics/jroplot_correlation.py @@ -2,10 +2,10 @@ import os import datetime import numpy import copy -from schainpy.model import * -from figure import Figure, isRealtime +from schainpy.model.graphics.jroplot_base import Plot -class CorrelationPlot(Figure): + +class CorrelationPlot(Plot): isConfig = None __nsubplots = None @@ -99,7 +99,7 @@ class CorrelationPlot(Figure): if realtime: if not(isRealtime(utcdatatime = dataOut.utctime)): - print 'Skipping this plot function' + print('Skipping this plot function') return if channelList == None: @@ -108,7 +108,7 @@ class CorrelationPlot(Figure): channelIndexList = [] for channel in channelList: if channel not in dataOut.channelList: - raise ValueError, "Channel %d is not in dataOut.channelList" + raise ValueError("Channel %d is not in dataOut.channelList") channelIndexList.append(dataOut.channelList.index(channel)) factor = dataOut.normFactor @@ -184,4 +184,4 @@ class CorrelationPlot(Figure): save=save, ftp=ftp, wr_period=wr_period, - thisDatetime=thisDatetime) + thisDatetime=thisDatetime) \ No newline at end of file diff --git a/schainpy/model/graphics/jroplot_data.py b/schainpy/model/graphics/jroplot_data.py deleted file mode 100644 index 1e30239..0000000 --- a/schainpy/model/graphics/jroplot_data.py +++ /dev/null @@ -1,1154 +0,0 @@ - -import os -import time -import glob -import datetime -from multiprocessing import Process - -import zmq -import numpy -import matplotlib -import matplotlib.pyplot as plt -from matplotlib.patches import Polygon -from mpl_toolkits.axes_grid1 import make_axes_locatable -from matplotlib.ticker import FuncFormatter, LinearLocator, MultipleLocator - -from schainpy.model.proc.jroproc_base import Operation -from schainpy.utils import log - -jet_values = matplotlib.pyplot.get_cmap('jet', 100)(numpy.arange(100))[10:90] -blu_values = matplotlib.pyplot.get_cmap( - 'seismic_r', 20)(numpy.arange(20))[10:15] -ncmap = matplotlib.colors.LinearSegmentedColormap.from_list( - 'jro', numpy.vstack((blu_values, jet_values))) -matplotlib.pyplot.register_cmap(cmap=ncmap) - -CMAPS = [plt.get_cmap(s) for s in ('jro', 'jet', 'viridis', 'plasma', 'inferno', 'Greys', 'seismic', 'bwr', 'coolwarm')] - -EARTH_RADIUS = 6.3710e3 - -def ll2xy(lat1, lon1, lat2, lon2): - - p = 0.017453292519943295 - a = 0.5 - numpy.cos((lat2 - lat1) * p)/2 + numpy.cos(lat1 * p) * numpy.cos(lat2 * p) * (1 - numpy.cos((lon2 - lon1) * p)) / 2 - r = 12742 * numpy.arcsin(numpy.sqrt(a)) - theta = numpy.arctan2(numpy.sin((lon2-lon1)*p)*numpy.cos(lat2*p), numpy.cos(lat1*p)*numpy.sin(lat2*p)-numpy.sin(lat1*p)*numpy.cos(lat2*p)*numpy.cos((lon2-lon1)*p)) - theta = -theta + numpy.pi/2 - return r*numpy.cos(theta), r*numpy.sin(theta) - -def km2deg(km): - ''' - Convert distance in km to degrees - ''' - - return numpy.rad2deg(km/EARTH_RADIUS) - -def figpause(interval): - backend = plt.rcParams['backend'] - if backend in matplotlib.rcsetup.interactive_bk: - figManager = matplotlib._pylab_helpers.Gcf.get_active() - if figManager is not None: - canvas = figManager.canvas - if canvas.figure.stale: - canvas.draw() - try: - canvas.start_event_loop(interval) - except: - pass - return - -def popup(message): - ''' - ''' - - fig = plt.figure(figsize=(12, 8), facecolor='r') - text = '\n'.join([s.strip() for s in message.split(':')]) - fig.text(0.01, 0.5, text, ha='left', va='center', size='20', weight='heavy', color='w') - fig.show() - figpause(1000) - - -class PlotData(Operation, Process): - ''' - Base class for Schain plotting operations - ''' - - CODE = 'Figure' - colormap = 'jro' - bgcolor = 'white' - CONFLATE = False - __missing = 1E30 - - __attrs__ = ['show', 'save', 'xmin', 'xmax', 'ymin', 'ymax', 'zmin', 'zmax', - 'zlimits', 'xlabel', 'ylabel', 'xaxis','cb_label', 'title', - 'colorbar', 'bgcolor', 'width', 'height', 'localtime', 'oneFigure', - 'showprofile', 'decimation', 'ftp'] - - def __init__(self, **kwargs): - - Operation.__init__(self, plot=True, **kwargs) - Process.__init__(self) - - self.kwargs['code'] = self.CODE - self.mp = False - self.data = None - self.isConfig = False - self.figures = [] - self.axes = [] - self.cb_axes = [] - self.localtime = kwargs.pop('localtime', True) - self.show = kwargs.get('show', True) - self.save = kwargs.get('save', False) - self.ftp = kwargs.get('ftp', False) - self.colormap = kwargs.get('colormap', self.colormap) - self.colormap_coh = kwargs.get('colormap_coh', 'jet') - self.colormap_phase = kwargs.get('colormap_phase', 'RdBu_r') - self.colormaps = kwargs.get('colormaps', None) - self.bgcolor = kwargs.get('bgcolor', self.bgcolor) - self.showprofile = kwargs.get('showprofile', False) - self.title = kwargs.get('wintitle', self.CODE.upper()) - self.cb_label = kwargs.get('cb_label', None) - self.cb_labels = kwargs.get('cb_labels', None) - self.labels = kwargs.get('labels', None) - self.xaxis = kwargs.get('xaxis', 'frequency') - self.zmin = kwargs.get('zmin', None) - self.zmax = kwargs.get('zmax', None) - self.zlimits = kwargs.get('zlimits', None) - self.xmin = kwargs.get('xmin', None) - self.xmax = kwargs.get('xmax', None) - self.xrange = kwargs.get('xrange', 24) - self.xscale = kwargs.get('xscale', None) - self.ymin = kwargs.get('ymin', None) - self.ymax = kwargs.get('ymax', None) - self.yscale = kwargs.get('yscale', None) - self.xlabel = kwargs.get('xlabel', None) - self.decimation = kwargs.get('decimation', None) - self.showSNR = kwargs.get('showSNR', False) - self.oneFigure = kwargs.get('oneFigure', True) - self.width = kwargs.get('width', None) - self.height = kwargs.get('height', None) - self.colorbar = kwargs.get('colorbar', True) - self.factors = kwargs.get('factors', [1, 1, 1, 1, 1, 1, 1, 1]) - self.channels = kwargs.get('channels', None) - self.titles = kwargs.get('titles', []) - self.polar = False - self.grid = kwargs.get('grid', False) - - def __fmtTime(self, x, pos): - ''' - ''' - - return '{}'.format(self.getDateTime(x).strftime('%H:%M')) - - def __setup(self): - ''' - Common setup for all figures, here figures and axes are created - ''' - - if self.CODE not in self.data: - raise ValueError(log.error('Missing data for {}'.format(self.CODE), - self.name)) - - self.setup() - - self.time_label = 'LT' if self.localtime else 'UTC' - if self.data.localtime: - self.getDateTime = datetime.datetime.fromtimestamp - else: - self.getDateTime = datetime.datetime.utcfromtimestamp - - if self.width is None: - self.width = 8 - - self.figures = [] - self.axes = [] - self.cb_axes = [] - self.pf_axes = [] - self.cmaps = [] - - size = '15%' if self.ncols == 1 else '30%' - pad = '4%' if self.ncols == 1 else '8%' - - if self.oneFigure: - if self.height is None: - self.height = 1.4 * self.nrows + 1 - fig = plt.figure(figsize=(self.width, self.height), - edgecolor='k', - facecolor='w') - self.figures.append(fig) - for n in range(self.nplots): - ax = fig.add_subplot(self.nrows, self.ncols, - n + 1, polar=self.polar) - ax.tick_params(labelsize=8) - ax.firsttime = True - ax.index = 0 - ax.press = None - self.axes.append(ax) - if self.showprofile: - cax = self.__add_axes(ax, size=size, pad=pad) - cax.tick_params(labelsize=8) - self.pf_axes.append(cax) - else: - if self.height is None: - self.height = 3 - for n in range(self.nplots): - fig = plt.figure(figsize=(self.width, self.height), - edgecolor='k', - facecolor='w') - ax = fig.add_subplot(1, 1, 1, polar=self.polar) - ax.tick_params(labelsize=8) - ax.firsttime = True - ax.index = 0 - ax.press = None - self.figures.append(fig) - self.axes.append(ax) - if self.showprofile: - cax = self.__add_axes(ax, size=size, pad=pad) - cax.tick_params(labelsize=8) - self.pf_axes.append(cax) - - for n in range(self.nrows): - if self.colormaps is not None: - cmap = plt.get_cmap(self.colormaps[n]) - else: - cmap = plt.get_cmap(self.colormap) - cmap.set_bad(self.bgcolor, 1.) - self.cmaps.append(cmap) - - for fig in self.figures: - fig.canvas.mpl_connect('key_press_event', self.OnKeyPress) - fig.canvas.mpl_connect('scroll_event', self.OnBtnScroll) - fig.canvas.mpl_connect('button_press_event', self.onBtnPress) - fig.canvas.mpl_connect('motion_notify_event', self.onMotion) - fig.canvas.mpl_connect('button_release_event', self.onBtnRelease) - if self.show: - fig.show() - - def OnKeyPress(self, event): - ''' - Event for pressing keys (up, down) change colormap - ''' - ax = event.inaxes - if ax in self.axes: - if event.key == 'down': - ax.index += 1 - elif event.key == 'up': - ax.index -= 1 - if ax.index < 0: - ax.index = len(CMAPS) - 1 - elif ax.index == len(CMAPS): - ax.index = 0 - cmap = CMAPS[ax.index] - ax.cbar.set_cmap(cmap) - ax.cbar.draw_all() - ax.plt.set_cmap(cmap) - ax.cbar.patch.figure.canvas.draw() - self.colormap = cmap.name - - def OnBtnScroll(self, event): - ''' - Event for scrolling, scale figure - ''' - cb_ax = event.inaxes - if cb_ax in [ax.cbar.ax for ax in self.axes if ax.cbar]: - ax = [ax for ax in self.axes if cb_ax == ax.cbar.ax][0] - pt = ax.cbar.ax.bbox.get_points()[:, 1] - nrm = ax.cbar.norm - vmin, vmax, p0, p1, pS = ( - nrm.vmin, nrm.vmax, pt[0], pt[1], event.y) - scale = 2 if event.step == 1 else 0.5 - point = vmin + (vmax - vmin) / (p1 - p0) * (pS - p0) - ax.cbar.norm.vmin = point - scale * (point - vmin) - ax.cbar.norm.vmax = point - scale * (point - vmax) - ax.plt.set_norm(ax.cbar.norm) - ax.cbar.draw_all() - ax.cbar.patch.figure.canvas.draw() - - def onBtnPress(self, event): - ''' - Event for mouse button press - ''' - cb_ax = event.inaxes - if cb_ax is None: - return - - if cb_ax in [ax.cbar.ax for ax in self.axes if ax.cbar]: - cb_ax.press = event.x, event.y - else: - cb_ax.press = None - - def onMotion(self, event): - ''' - Event for move inside colorbar - ''' - cb_ax = event.inaxes - if cb_ax is None: - return - if cb_ax not in [ax.cbar.ax for ax in self.axes if ax.cbar]: - return - if cb_ax.press is None: - return - - ax = [ax for ax in self.axes if cb_ax == ax.cbar.ax][0] - xprev, yprev = cb_ax.press - dx = event.x - xprev - dy = event.y - yprev - cb_ax.press = event.x, event.y - scale = ax.cbar.norm.vmax - ax.cbar.norm.vmin - perc = 0.03 - - if event.button == 1: - ax.cbar.norm.vmin -= (perc * scale) * numpy.sign(dy) - ax.cbar.norm.vmax -= (perc * scale) * numpy.sign(dy) - elif event.button == 3: - ax.cbar.norm.vmin -= (perc * scale) * numpy.sign(dy) - ax.cbar.norm.vmax += (perc * scale) * numpy.sign(dy) - - ax.cbar.draw_all() - ax.plt.set_norm(ax.cbar.norm) - ax.cbar.patch.figure.canvas.draw() - - def onBtnRelease(self, event): - ''' - Event for mouse button release - ''' - cb_ax = event.inaxes - if cb_ax is not None: - cb_ax.press = None - - def __add_axes(self, ax, size='30%', pad='8%'): - ''' - Add new axes to the given figure - ''' - divider = make_axes_locatable(ax) - nax = divider.new_horizontal(size=size, pad=pad) - ax.figure.add_axes(nax) - return nax - - self.setup() - - def setup(self): - ''' - This method should be implemented in the child class, the following - attributes should be set: - - self.nrows: number of rows - self.ncols: number of cols - self.nplots: number of plots (channels or pairs) - self.ylabel: label for Y axes - self.titles: list of axes title - - ''' - raise(NotImplementedError, 'Implement this method in child class') - - def fill_gaps(self, x_buffer, y_buffer, z_buffer): - ''' - Create a masked array for missing data - ''' - if x_buffer.shape[0] < 2: - return x_buffer, y_buffer, z_buffer - - deltas = x_buffer[1:] - x_buffer[0:-1] - x_median = numpy.median(deltas) - - index = numpy.where(deltas > 5 * x_median) - - if len(index[0]) != 0: - z_buffer[::, index[0], ::] = self.__missing - z_buffer = numpy.ma.masked_inside(z_buffer, - 0.99 * self.__missing, - 1.01 * self.__missing) - - return x_buffer, y_buffer, z_buffer - - def decimate(self): - - # dx = int(len(self.x)/self.__MAXNUMX) + 1 - dy = int(len(self.y) / self.decimation) + 1 - - # x = self.x[::dx] - x = self.x - y = self.y[::dy] - z = self.z[::, ::, ::dy] - - return x, y, z - - def format(self): - ''' - Set min and max values, labels, ticks and titles - ''' - - if self.xmin is None: - xmin = self.min_time - else: - if self.xaxis is 'time': - dt = self.getDateTime(self.min_time) - xmin = (dt.replace(hour=int(self.xmin), minute=0, second=0) - - datetime.datetime(1970, 1, 1)).total_seconds() - if self.data.localtime: - xmin += time.timezone - else: - xmin = self.xmin - - if self.xmax is None: - xmax = xmin + self.xrange * 60 * 60 - else: - if self.xaxis is 'time': - dt = self.getDateTime(self.max_time) - xmax = (dt.replace(hour=int(self.xmax), minute=59, second=59) - - datetime.datetime(1970, 1, 1) + datetime.timedelta(seconds=1)).total_seconds() - if self.data.localtime: - xmax += time.timezone - else: - xmax = self.xmax - - ymin = self.ymin if self.ymin else numpy.nanmin(self.y) - ymax = self.ymax if self.ymax else numpy.nanmax(self.y) - - Y = numpy.array([1, 2, 5, 10, 20, 50, 100, 200, 500, 1000, 2000, 5000]) - i = 1 if numpy.where(abs(ymax-ymin) <= Y)[0][0] < 0 else numpy.where(abs(ymax-ymin) <= Y)[0][0] - ystep = Y[i] / 10. - - if self.xaxis is not 'time': - X = numpy.array([1, 2, 5, 10, 20, 50, 100, 200, 500, 1000, 2000, 5000])/2. - i = 1 if numpy.where(abs(xmax-xmin) <= X)[0][0] < 0 else numpy.where(abs(xmax-xmin) <= X)[0][0] - xstep = X[i] / 10. - - for n, ax in enumerate(self.axes): - if ax.firsttime: - ax.set_facecolor(self.bgcolor) - ax.yaxis.set_major_locator(MultipleLocator(ystep)) - if self.xscale: - ax.xaxis.set_major_formatter(FuncFormatter(lambda x, pos: '{0:g}'.format(x*self.xscale))) - if self.xscale: - ax.yaxis.set_major_formatter(FuncFormatter(lambda x, pos: '{0:g}'.format(x*self.yscale))) - if self.xaxis is 'time': - ax.xaxis.set_major_formatter(FuncFormatter(self.__fmtTime)) - ax.xaxis.set_major_locator(LinearLocator(9)) - else: - ax.xaxis.set_major_locator(MultipleLocator(xstep)) - if self.xlabel is not None: - ax.set_xlabel(self.xlabel) - ax.set_ylabel(self.ylabel) - ax.firsttime = False - if self.showprofile: - self.pf_axes[n].set_ylim(ymin, ymax) - self.pf_axes[n].set_xlim(self.zmin, self.zmax) - self.pf_axes[n].set_xlabel('dB') - self.pf_axes[n].grid(b=True, axis='x') - [tick.set_visible(False) - for tick in self.pf_axes[n].get_yticklabels()] - if self.colorbar: - ax.cbar = plt.colorbar( - ax.plt, ax=ax, fraction=0.05, pad=0.02, aspect=10) - ax.cbar.ax.tick_params(labelsize=8) - ax.cbar.ax.press = None - if self.cb_label: - ax.cbar.set_label(self.cb_label, size=8) - elif self.cb_labels: - ax.cbar.set_label(self.cb_labels[n], size=8) - else: - ax.cbar = None - if self.grid: - ax.grid(True) - - if not self.polar: - ax.set_xlim(xmin, xmax) - ax.set_ylim(ymin, ymax) - ax.set_title('{} {} {}'.format( - self.titles[n], - self.getDateTime(self.max_time).strftime('%Y-%m-%dT%H:%M:%S'), - self.time_label), - size=8) - else: - ax.set_title('{}'.format(self.titles[n]), size=8) - ax.set_ylim(0, 90) - ax.set_yticks(numpy.arange(0, 90, 20)) - ax.yaxis.labelpad = 40 - - def __plot(self): - ''' - ''' - log.log('Plotting', self.name) - - try: - self.plot() - self.format() - except Exception as e: - log.warning('{} Plot could not be updated... check data'.format(self.CODE), self.name) - log.error(str(e), '') - return - - for n, fig in enumerate(self.figures): - if self.nrows == 0 or self.nplots == 0: - log.warning('No data', self.name) - fig.text(0.5, 0.5, 'No Data', fontsize='large', ha='center') - fig.canvas.manager.set_window_title(self.CODE) - continue - - fig.tight_layout() - fig.canvas.manager.set_window_title('{} - {}'.format(self.title, - self.getDateTime(self.max_time).strftime('%Y/%m/%d'))) - fig.canvas.draw() - - if self.save and (self.data.ended or not self.data.buffering): - - if self.save_labels: - labels = self.save_labels - else: - labels = range(self.nrows) - - if self.oneFigure: - label = '' - else: - label = '-{}'.format(labels[n]) - figname = os.path.join( - self.save, - self.CODE, - '{}{}_{}.png'.format( - self.CODE, - label, - self.getDateTime(self.saveTime).strftime( - '%Y%m%d_%H%M%S'), - ) - ) - log.log('Saving figure: {}'.format(figname), self.name) - if not os.path.isdir(os.path.dirname(figname)): - os.makedirs(os.path.dirname(figname)) - fig.savefig(figname) - - def plot(self): - ''' - ''' - raise(NotImplementedError, 'Implement this method in child class') - - def run(self): - - log.log('Starting', self.name) - - context = zmq.Context() - receiver = context.socket(zmq.SUB) - receiver.setsockopt(zmq.SUBSCRIBE, '') - receiver.setsockopt(zmq.CONFLATE, self.CONFLATE) - - if 'server' in self.kwargs['parent']: - receiver.connect( - 'ipc:///tmp/{}.plots'.format(self.kwargs['parent']['server'])) - else: - receiver.connect("ipc:///tmp/zmq.plots") - - while True: - try: - self.data = receiver.recv_pyobj(flags=zmq.NOBLOCK) - if self.data.localtime and self.localtime: - self.times = self.data.times - elif self.data.localtime and not self.localtime: - self.times = self.data.times + time.timezone - elif not self.data.localtime and self.localtime: - self.times = self.data.times - time.timezone - else: - self.times = self.data.times - - self.min_time = self.times[0] - self.max_time = self.times[-1] - - if self.isConfig is False: - self.__setup() - self.isConfig = True - - self.__plot() - - except zmq.Again as e: - if self.data and self.data.ended: - break - log.log('Waiting for data...') - if self.data: - figpause(self.data.throttle) - else: - time.sleep(2) - - def close(self): - if self.data: - self.__plot() - - -class PlotSpectraData(PlotData): - ''' - Plot for Spectra data - ''' - - CODE = 'spc' - colormap = 'jro' - - def setup(self): - self.nplots = len(self.data.channels) - self.ncols = int(numpy.sqrt(self.nplots) + 0.9) - self.nrows = int((1.0 * self.nplots / self.ncols) + 0.9) - self.width = 3.4 * self.ncols - self.height = 3 * self.nrows - self.cb_label = 'dB' - if self.showprofile: - self.width += 0.8 * self.ncols - - self.ylabel = 'Range [km]' - - def plot(self): - if self.xaxis == "frequency": - x = self.data.xrange[0] - self.xlabel = "Frequency (kHz)" - elif self.xaxis == "time": - x = self.data.xrange[1] - self.xlabel = "Time (ms)" - else: - x = self.data.xrange[2] - self.xlabel = "Velocity (m/s)" - - if self.CODE == 'spc_mean': - x = self.data.xrange[2] - self.xlabel = "Velocity (m/s)" - - self.titles = [] - - y = self.data.heights - self.y = y - z = self.data['spc'] - - for n, ax in enumerate(self.axes): - noise = self.data['noise'][n][-1] - if self.CODE == 'spc_mean': - mean = self.data['mean'][n][-1] - if ax.firsttime: - self.xmax = self.xmax if self.xmax else numpy.nanmax(x) - self.xmin = self.xmin if self.xmin else -self.xmax - self.zmin = self.zmin if self.zmin else numpy.nanmin(z) - self.zmax = self.zmax if self.zmax else numpy.nanmax(z) - ax.plt = ax.pcolormesh(x, y, z[n].T, - vmin=self.zmin, - vmax=self.zmax, - cmap=plt.get_cmap(self.colormap) - ) - - if self.showprofile: - ax.plt_profile = self.pf_axes[n].plot( - self.data['rti'][n][-1], y)[0] - ax.plt_noise = self.pf_axes[n].plot(numpy.repeat(noise, len(y)), y, - color="k", linestyle="dashed", lw=1)[0] - if self.CODE == 'spc_mean': - ax.plt_mean = ax.plot(mean, y, color='k')[0] - else: - ax.plt.set_array(z[n].T.ravel()) - if self.showprofile: - ax.plt_profile.set_data(self.data['rti'][n][-1], y) - ax.plt_noise.set_data(numpy.repeat(noise, len(y)), y) - if self.CODE == 'spc_mean': - ax.plt_mean.set_data(mean, y) - - self.titles.append('CH {}: {:3.2f}dB'.format(n, noise)) - self.saveTime = self.max_time - - -class PlotCrossSpectraData(PlotData): - - CODE = 'cspc' - zmin_coh = None - zmax_coh = None - zmin_phase = None - zmax_phase = None - - def setup(self): - - self.ncols = 4 - self.nrows = len(self.data.pairs) - self.nplots = self.nrows * 4 - self.width = 3.4 * self.ncols - self.height = 3 * self.nrows - self.ylabel = 'Range [km]' - self.showprofile = False - - def plot(self): - - if self.xaxis == "frequency": - x = self.data.xrange[0] - self.xlabel = "Frequency (kHz)" - elif self.xaxis == "time": - x = self.data.xrange[1] - self.xlabel = "Time (ms)" - else: - x = self.data.xrange[2] - self.xlabel = "Velocity (m/s)" - - self.titles = [] - - y = self.data.heights - self.y = y - spc = self.data['spc'] - cspc = self.data['cspc'] - - for n in range(self.nrows): - noise = self.data['noise'][n][-1] - pair = self.data.pairs[n] - ax = self.axes[4 * n] - ax3 = self.axes[4 * n + 3] - if ax.firsttime: - self.xmax = self.xmax if self.xmax else numpy.nanmax(x) - self.xmin = self.xmin if self.xmin else -self.xmax - self.zmin = self.zmin if self.zmin else numpy.nanmin(spc) - self.zmax = self.zmax if self.zmax else numpy.nanmax(spc) - ax.plt = ax.pcolormesh(x, y, spc[pair[0]].T, - vmin=self.zmin, - vmax=self.zmax, - cmap=plt.get_cmap(self.colormap) - ) - else: - ax.plt.set_array(spc[pair[0]].T.ravel()) - self.titles.append('CH {}: {:3.2f}dB'.format(n, noise)) - - ax = self.axes[4 * n + 1] - if ax.firsttime: - ax.plt = ax.pcolormesh(x, y, spc[pair[1]].T, - vmin=self.zmin, - vmax=self.zmax, - cmap=plt.get_cmap(self.colormap) - ) - else: - ax.plt.set_array(spc[pair[1]].T.ravel()) - self.titles.append('CH {}: {:3.2f}dB'.format(n, noise)) - - out = cspc[n] / numpy.sqrt(spc[pair[0]] * spc[pair[1]]) - coh = numpy.abs(out) - phase = numpy.arctan2(out.imag, out.real) * 180 / numpy.pi - - ax = self.axes[4 * n + 2] - if ax.firsttime: - ax.plt = ax.pcolormesh(x, y, coh.T, - vmin=0, - vmax=1, - cmap=plt.get_cmap(self.colormap_coh) - ) - else: - ax.plt.set_array(coh.T.ravel()) - self.titles.append( - 'Coherence Ch{} * Ch{}'.format(pair[0], pair[1])) - - ax = self.axes[4 * n + 3] - if ax.firsttime: - ax.plt = ax.pcolormesh(x, y, phase.T, - vmin=-180, - vmax=180, - cmap=plt.get_cmap(self.colormap_phase) - ) - else: - ax.plt.set_array(phase.T.ravel()) - self.titles.append('Phase CH{} * CH{}'.format(pair[0], pair[1])) - - self.saveTime = self.max_time - - -class PlotSpectraMeanData(PlotSpectraData): - ''' - Plot for Spectra and Mean - ''' - CODE = 'spc_mean' - colormap = 'jro' - - -class PlotRTIData(PlotData): - ''' - Plot for RTI data - ''' - - CODE = 'rti' - colormap = 'jro' - - def setup(self): - self.xaxis = 'time' - self.ncols = 1 - self.nrows = len(self.data.channels) - self.nplots = len(self.data.channels) - self.ylabel = 'Range [km]' - self.cb_label = 'dB' - self.titles = ['{} Channel {}'.format( - self.CODE.upper(), x) for x in range(self.nrows)] - - def plot(self): - self.x = self.times - self.y = self.data.heights - self.z = self.data[self.CODE] - self.z = numpy.ma.masked_invalid(self.z) - - if self.decimation is None: - x, y, z = self.fill_gaps(self.x, self.y, self.z) - else: - x, y, z = self.fill_gaps(*self.decimate()) - - for n, ax in enumerate(self.axes): - self.zmin = self.zmin if self.zmin else numpy.min(self.z) - self.zmax = self.zmax if self.zmax else numpy.max(self.z) - if ax.firsttime: - ax.plt = ax.pcolormesh(x, y, z[n].T, - vmin=self.zmin, - vmax=self.zmax, - cmap=plt.get_cmap(self.colormap) - ) - if self.showprofile: - ax.plot_profile = self.pf_axes[n].plot( - self.data['rti'][n][-1], self.y)[0] - ax.plot_noise = self.pf_axes[n].plot(numpy.repeat(self.data['noise'][n][-1], len(self.y)), self.y, - color="k", linestyle="dashed", lw=1)[0] - else: - ax.collections.remove(ax.collections[0]) - ax.plt = ax.pcolormesh(x, y, z[n].T, - vmin=self.zmin, - vmax=self.zmax, - cmap=plt.get_cmap(self.colormap) - ) - if self.showprofile: - ax.plot_profile.set_data(self.data['rti'][n][-1], self.y) - ax.plot_noise.set_data(numpy.repeat( - self.data['noise'][n][-1], len(self.y)), self.y) - - self.saveTime = self.min_time - - -class PlotCOHData(PlotRTIData): - ''' - Plot for Coherence data - ''' - - CODE = 'coh' - - def setup(self): - self.xaxis = 'time' - self.ncols = 1 - self.nrows = len(self.data.pairs) - self.nplots = len(self.data.pairs) - self.ylabel = 'Range [km]' - if self.CODE == 'coh': - self.cb_label = '' - self.titles = [ - 'Coherence Map Ch{} * Ch{}'.format(x[0], x[1]) for x in self.data.pairs] - else: - self.cb_label = 'Degrees' - self.titles = [ - 'Phase Map Ch{} * Ch{}'.format(x[0], x[1]) for x in self.data.pairs] - - -class PlotPHASEData(PlotCOHData): - ''' - Plot for Phase map data - ''' - - CODE = 'phase' - colormap = 'seismic' - - -class PlotNoiseData(PlotData): - ''' - Plot for noise - ''' - - CODE = 'noise' - - def setup(self): - self.xaxis = 'time' - self.ncols = 1 - self.nrows = 1 - self.nplots = 1 - self.ylabel = 'Intensity [dB]' - self.titles = ['Noise'] - self.colorbar = False - - def plot(self): - - x = self.times - xmin = self.min_time - xmax = xmin + self.xrange * 60 * 60 - Y = self.data[self.CODE] - - if self.axes[0].firsttime: - for ch in self.data.channels: - y = Y[ch] - self.axes[0].plot(x, y, lw=1, label='Ch{}'.format(ch)) - plt.legend() - else: - for ch in self.data.channels: - y = Y[ch] - self.axes[0].lines[ch].set_data(x, y) - - self.ymin = numpy.nanmin(Y) - 5 - self.ymax = numpy.nanmax(Y) + 5 - self.saveTime = self.min_time - - -class PlotSNRData(PlotRTIData): - ''' - Plot for SNR Data - ''' - - CODE = 'snr' - colormap = 'jet' - - -class PlotDOPData(PlotRTIData): - ''' - Plot for DOPPLER Data - ''' - - CODE = 'dop' - colormap = 'jet' - - -class PlotSkyMapData(PlotData): - ''' - Plot for meteors detection data - ''' - - CODE = 'param' - - def setup(self): - - self.ncols = 1 - self.nrows = 1 - self.width = 7.2 - self.height = 7.2 - self.nplots = 1 - self.xlabel = 'Zonal Zenith Angle (deg)' - self.ylabel = 'Meridional Zenith Angle (deg)' - self.polar = True - self.ymin = -180 - self.ymax = 180 - self.colorbar = False - - def plot(self): - - arrayParameters = numpy.concatenate(self.data['param']) - error = arrayParameters[:, -1] - indValid = numpy.where(error == 0)[0] - finalMeteor = arrayParameters[indValid, :] - finalAzimuth = finalMeteor[:, 3] - finalZenith = finalMeteor[:, 4] - - x = finalAzimuth * numpy.pi / 180 - y = finalZenith - - ax = self.axes[0] - - if ax.firsttime: - ax.plot = ax.plot(x, y, 'bo', markersize=5)[0] - else: - ax.plot.set_data(x, y) - - dt1 = self.getDateTime(self.min_time).strftime('%y/%m/%d %H:%M:%S') - dt2 = self.getDateTime(self.max_time).strftime('%y/%m/%d %H:%M:%S') - title = 'Meteor Detection Sky Map\n %s - %s \n Number of events: %5.0f\n' % (dt1, - dt2, - len(x)) - self.titles[0] = title - self.saveTime = self.max_time - - -class PlotParamData(PlotRTIData): - ''' - Plot for data_param object - ''' - - CODE = 'param' - colormap = 'seismic' - - def setup(self): - self.xaxis = 'time' - self.ncols = 1 - self.nrows = self.data.shape(self.CODE)[0] - self.nplots = self.nrows - if self.showSNR: - self.nrows += 1 - self.nplots += 1 - - self.ylabel = 'Height [km]' - if not self.titles: - self.titles = self.data.parameters \ - if self.data.parameters else ['Param {}'.format(x) for x in xrange(self.nrows)] - if self.showSNR: - self.titles.append('SNR') - - def plot(self): - self.data.normalize_heights() - self.x = self.times - self.y = self.data.heights - if self.showSNR: - self.z = numpy.concatenate( - (self.data[self.CODE], self.data['snr']) - ) - else: - self.z = self.data[self.CODE] - - self.z = numpy.ma.masked_invalid(self.z) - - if self.decimation is None: - x, y, z = self.fill_gaps(self.x, self.y, self.z) - else: - x, y, z = self.fill_gaps(*self.decimate()) - - for n, ax in enumerate(self.axes): - - self.zmax = self.zmax if self.zmax is not None else numpy.max( - self.z[n]) - self.zmin = self.zmin if self.zmin is not None else numpy.min( - self.z[n]) - - if ax.firsttime: - if self.zlimits is not None: - self.zmin, self.zmax = self.zlimits[n] - - ax.plt = ax.pcolormesh(x, y, z[n].T * self.factors[n], - vmin=self.zmin, - vmax=self.zmax, - cmap=self.cmaps[n] - ) - else: - if self.zlimits is not None: - self.zmin, self.zmax = self.zlimits[n] - ax.collections.remove(ax.collections[0]) - ax.plt = ax.pcolormesh(x, y, z[n].T * self.factors[n], - vmin=self.zmin, - vmax=self.zmax, - cmap=self.cmaps[n] - ) - - self.saveTime = self.min_time - - -class PlotOutputData(PlotParamData): - ''' - Plot data_output object - ''' - - CODE = 'output' - colormap = 'seismic' - - -class PlotPolarMapData(PlotData): - ''' - Plot for meteors detection data - ''' - - CODE = 'param' - colormap = 'seismic' - - def setup(self): - self.ncols = 1 - self.nrows = 1 - self.width = 9 - self.height = 8 - self.mode = self.data.meta['mode'] - if self.channels is not None: - self.nplots = len(self.channels) - self.nrows = len(self.channels) - else: - self.nplots = self.data.shape(self.CODE)[0] - self.nrows = self.nplots - self.channels = range(self.nplots) - if self.mode == 'E': - self.xlabel = 'Longitude' - self.ylabel = 'Latitude' - else: - self.xlabel = 'Range (km)' - self.ylabel = 'Height (km)' - self.bgcolor = 'white' - self.cb_labels = self.data.meta['units'] - self.lat = self.data.meta['latitude'] - self.lon = self.data.meta['longitude'] - self.xmin, self.xmax = float(km2deg(self.xmin) + self.lon), float(km2deg(self.xmax) + self.lon) - self.ymin, self.ymax = float(km2deg(self.ymin) + self.lat), float(km2deg(self.ymax) + self.lat) - # self.polar = True - - def plot(self): - - for n, ax in enumerate(self.axes): - data = self.data['param'][self.channels[n]] - - zeniths = numpy.linspace(0, self.data.meta['max_range'], data.shape[1]) - if self.mode == 'E': - azimuths = -numpy.radians(self.data.heights)+numpy.pi/2 - r, theta = numpy.meshgrid(zeniths, azimuths) - x, y = r*numpy.cos(theta)*numpy.cos(numpy.radians(self.data.meta['elevation'])), r*numpy.sin(theta)*numpy.cos(numpy.radians(self.data.meta['elevation'])) - x = km2deg(x) + self.lon - y = km2deg(y) + self.lat - else: - azimuths = numpy.radians(self.data.heights) - r, theta = numpy.meshgrid(zeniths, azimuths) - x, y = r*numpy.cos(theta), r*numpy.sin(theta) - self.y = zeniths - - if ax.firsttime: - if self.zlimits is not None: - self.zmin, self.zmax = self.zlimits[n] - ax.plt = ax.pcolormesh(#r, theta, numpy.ma.array(data, mask=numpy.isnan(data)), - x, y, numpy.ma.array(data, mask=numpy.isnan(data)), - vmin=self.zmin, - vmax=self.zmax, - cmap=self.cmaps[n]) - else: - if self.zlimits is not None: - self.zmin, self.zmax = self.zlimits[n] - ax.collections.remove(ax.collections[0]) - ax.plt = ax.pcolormesh(# r, theta, numpy.ma.array(data, mask=numpy.isnan(data)), - x, y, numpy.ma.array(data, mask=numpy.isnan(data)), - vmin=self.zmin, - vmax=self.zmax, - cmap=self.cmaps[n]) - - if self.mode == 'A': - continue - - # plot district names - f = open('/data/workspace/schain_scripts/distrito.csv') - for line in f: - label, lon, lat = [s.strip() for s in line.split(',') if s] - lat = float(lat) - lon = float(lon) - # ax.plot(lon, lat, '.b', ms=2) - ax.text(lon, lat, label.decode('utf8'), ha='center', va='bottom', size='8', color='black') - - # plot limites - limites =[] - tmp = [] - for line in open('/data/workspace/schain_scripts/lima.csv'): - if '#' in line: - if tmp: - limites.append(tmp) - tmp = [] - continue - values = line.strip().split(',') - tmp.append((float(values[0]), float(values[1]))) - for points in limites: - ax.add_patch(Polygon(points, ec='k', fc='none', ls='--', lw=0.5)) - - # plot Cuencas - for cuenca in ('rimac', 'lurin', 'mala', 'chillon', 'chilca', 'chancay-huaral'): - f = open('/data/workspace/schain_scripts/{}.csv'.format(cuenca)) - values = [line.strip().split(',') for line in f] - points = [(float(s[0]), float(s[1])) for s in values] - ax.add_patch(Polygon(points, ec='b', fc='none')) - - # plot grid - for r in (15, 30, 45, 60): - ax.add_artist(plt.Circle((self.lon, self.lat), km2deg(r), color='0.6', fill=False, lw=0.2)) - ax.text( - self.lon + (km2deg(r))*numpy.cos(60*numpy.pi/180), - self.lat + (km2deg(r))*numpy.sin(60*numpy.pi/180), - '{}km'.format(r), - ha='center', va='bottom', size='8', color='0.6', weight='heavy') - - if self.mode == 'E': - title = 'El={}$^\circ$'.format(self.data.meta['elevation']) - label = 'E{:02d}'.format(int(self.data.meta['elevation'])) - else: - title = 'Az={}$^\circ$'.format(self.data.meta['azimuth']) - label = 'A{:02d}'.format(int(self.data.meta['azimuth'])) - - self.save_labels = ['{}-{}'.format(lbl, label) for lbl in self.labels] - self.titles = ['{} {}'.format(self.data.parameters[x], title) for x in self.channels] - self.saveTime = self.max_time - - diff --git a/schainpy/model/graphics/jroplot_heispectra.py b/schainpy/model/graphics/jroplot_heispectra.py index b0c9c5b..44d5bea 100644 --- a/schainpy/model/graphics/jroplot_heispectra.py +++ b/schainpy/model/graphics/jroplot_heispectra.py @@ -7,10 +7,10 @@ import os import datetime import numpy -from figure import Figure, isRealtime -from plotting_codes import * +from schainpy.model.graphics.jroplot_base import Plot -class SpectraHeisScope(Figure): + +class SpectraHeisScope(Plot): isConfig = None @@ -20,9 +20,9 @@ class SpectraHeisScope(Figure): HEIGHTPROF = None PREFIX = 'spc' - def __init__(self, **kwargs): + def __init__(self):#, **kwargs): - Figure.__init__(self, **kwargs) + Plot.__init__(self)#, **kwargs) self.isConfig = False self.__nsubplots = 1 @@ -96,9 +96,12 @@ class SpectraHeisScope(Figure): ymax : None, """ + if dataOut.flagNoData: + return dataOut + if dataOut.realtime: if not(isRealtime(utcdatatime = dataOut.utctime)): - print 'Skipping this plot function' + print('Skipping this plot function') return if channelList == None: @@ -107,7 +110,7 @@ class SpectraHeisScope(Figure): channelIndexList = [] for channel in channelList: if channel not in dataOut.channelList: - raise ValueError, "Channel %d is not in dataOut.channelList" + raise ValueError("Channel %d is not in dataOut.channelList") channelIndexList.append(dataOut.channelList.index(channel)) # x = dataOut.heightList @@ -173,15 +176,18 @@ class SpectraHeisScope(Figure): wr_period=wr_period, thisDatetime=thisDatetime) -class RTIfromSpectraHeis(Figure): + return dataOut + + +class RTIfromSpectraHeis(Plot): isConfig = None __nsubplots = None PREFIX = 'rtinoise' - def __init__(self, **kwargs): - Figure.__init__(self, **kwargs) + def __init__(self):#, **kwargs): + Plot.__init__(self)#, **kwargs) self.timerange = 24*60*60 self.isConfig = False self.__nsubplots = 1 @@ -231,6 +237,10 @@ class RTIfromSpectraHeis(Figure): server=None, folder=None, username=None, password=None, ftp_wei=0, exp_code=0, sub_exp_code=0, plot_pos=0): + if dataOut.flagNoData: + return dataOut + + if channelList == None: channelIndexList = dataOut.channelIndexList channelList = dataOut.channelList @@ -238,7 +248,7 @@ class RTIfromSpectraHeis(Figure): channelIndexList = [] for channel in channelList: if channel not in dataOut.channelList: - raise ValueError, "Channel %d is not in dataOut.channelList" + raise ValueError("Channel %d is not in dataOut.channelList") channelIndexList.append(dataOut.channelList.index(channel)) if timerange != None: @@ -327,3 +337,6 @@ class RTIfromSpectraHeis(Figure): wr_period=wr_period, thisDatetime=thisDatetime, update_figfile=update_figfile) + + + return dataOut \ No newline at end of file diff --git a/schainpy/model/graphics/jroplot_parameters.py b/schainpy/model/graphics/jroplot_parameters.py index 4e810c6..ed29a71 100644 --- a/schainpy/model/graphics/jroplot_parameters.py +++ b/schainpy/model/graphics/jroplot_parameters.py @@ -1,2151 +1,346 @@ import os import datetime import numpy -import inspect -from figure import Figure, isRealtime, isTimeInHourRange -from plotting_codes import * +from schainpy.model.graphics.jroplot_base import Plot, plt +from schainpy.model.graphics.jroplot_spectra import SpectraPlot, RTIPlot, CoherencePlot +from schainpy.utils import log -class FitGauPlot(Figure): +EARTH_RADIUS = 6.3710e3 - isConfig = None - __nsubplots = None - WIDTHPROF = None - HEIGHTPROF = None - PREFIX = 'fitgau' +def ll2xy(lat1, lon1, lat2, lon2): - def __init__(self, **kwargs): - Figure.__init__(self, **kwargs) - self.isConfig = False - self.__nsubplots = 1 + p = 0.017453292519943295 + a = 0.5 - numpy.cos((lat2 - lat1) * p)/2 + numpy.cos(lat1 * p) * \ + numpy.cos(lat2 * p) * (1 - numpy.cos((lon2 - lon1) * p)) / 2 + r = 12742 * numpy.arcsin(numpy.sqrt(a)) + theta = numpy.arctan2(numpy.sin((lon2-lon1)*p)*numpy.cos(lat2*p), numpy.cos(lat1*p) + * numpy.sin(lat2*p)-numpy.sin(lat1*p)*numpy.cos(lat2*p)*numpy.cos((lon2-lon1)*p)) + theta = -theta + numpy.pi/2 + return r*numpy.cos(theta), r*numpy.sin(theta) - self.WIDTH = 250 - self.HEIGHT = 250 - self.WIDTHPROF = 120 - self.HEIGHTPROF = 0 - self.counter_imagwr = 0 - self.PLOT_CODE = SPEC_CODE +def km2deg(km): + ''' + Convert distance in km to degrees + ''' - self.FTP_WEI = None - self.EXP_CODE = None - self.SUB_EXP_CODE = None - self.PLOT_POS = None + return numpy.rad2deg(km/EARTH_RADIUS) - self.__xfilter_ena = False - self.__yfilter_ena = False - def getSubplots(self): - ncol = int(numpy.sqrt(self.nplots)+0.9) - nrow = int(self.nplots*1./ncol + 0.9) +class SpectralMomentsPlot(SpectraPlot): + ''' + Plot for Spectral Moments + ''' + CODE = 'spc_moments' + colormap = 'jet' + plot_name = 'SpectralMoments' + plot_type = 'pcolor' - return nrow, ncol - def setup(self, id, nplots, wintitle, showprofile=True, show=True): +class SnrPlot(RTIPlot): + ''' + Plot for SNR Data + ''' - self.__showprofile = showprofile - self.nplots = nplots + CODE = 'snr' + colormap = 'jet' + plot_name = 'SNR' - ncolspan = 1 - colspan = 1 - if showprofile: - ncolspan = 3 - colspan = 2 - self.__nsubplots = 2 - self.createFigure(id = id, - wintitle = wintitle, - widthplot = self.WIDTH + self.WIDTHPROF, - heightplot = self.HEIGHT + self.HEIGHTPROF, - show=show) +class DopplerPlot(RTIPlot): + ''' + Plot for DOPPLER Data (1st moment) + ''' - nrow, ncol = self.getSubplots() + CODE = 'dop' + colormap = 'jet' + plot_name = 'DopplerShift' - counter = 0 - for y in range(nrow): - for x in range(ncol): - if counter >= self.nplots: - break +class PowerPlot(RTIPlot): + ''' + Plot for Power Data (0 moment) + ''' - self.addAxes(nrow, ncol*ncolspan, y, x*ncolspan, colspan, 1) + CODE = 'pow' + colormap = 'jet' + plot_name = 'TotalPower' - if showprofile: - self.addAxes(nrow, ncol*ncolspan, y, x*ncolspan+colspan, 1, 1) - counter += 1 +class SpectralWidthPlot(RTIPlot): + ''' + Plot for Spectral Width Data (2nd moment) + ''' - def run(self, dataOut, id, wintitle="", channelList=None, showprofile=True, - xmin=None, xmax=None, ymin=None, ymax=None, zmin=None, zmax=None, - save=False, figpath='./', figfile=None, show=True, ftp=False, wr_period=1, - server=None, folder=None, username=None, password=None, - ftp_wei=0, exp_code=0, sub_exp_code=0, plot_pos=0, realtime=False, - xaxis="frequency", colormap='jet', normFactor=None , GauSelector = 1): + CODE = 'width' + colormap = 'jet' + plot_name = 'SpectralWidth' - """ - Input: - dataOut : - id : - wintitle : - channelList : - showProfile : - xmin : None, - xmax : None, - ymin : None, - ymax : None, - zmin : None, - zmax : None - """ - if realtime: - if not(isRealtime(utcdatatime = dataOut.utctime)): - print 'Skipping this plot function' - return +class SkyMapPlot(Plot): + ''' + Plot for meteors detection data + ''' - if channelList == None: - channelIndexList = dataOut.channelIndexList - else: - channelIndexList = [] - for channel in channelList: - if channel not in dataOut.channelList: - raise ValueError, "Channel %d is not in dataOut.channelList" %channel - channelIndexList.append(dataOut.channelList.index(channel)) - -# if normFactor is None: -# factor = dataOut.normFactor -# else: -# factor = normFactor - if xaxis == "frequency": - x = dataOut.spc_range[0] - xlabel = "Frequency (kHz)" - - elif xaxis == "time": - x = dataOut.spc_range[1] - xlabel = "Time (ms)" - - else: - x = dataOut.spc_range[2] - xlabel = "Velocity (m/s)" - - ylabel = "Range (Km)" - - y = dataOut.getHeiRange() - - z = dataOut.GauSPC[:,GauSelector,:,:] #GauSelector] #dataOut.data_spc/factor - print 'GausSPC', z[0,32,10:40] - z = numpy.where(numpy.isfinite(z), z, numpy.NAN) - zdB = 10*numpy.log10(z) - - avg = numpy.average(z, axis=1) - avgdB = 10*numpy.log10(avg) - - noise = dataOut.spc_noise - noisedB = 10*numpy.log10(noise) - - thisDatetime = datetime.datetime.utcfromtimestamp(dataOut.getTimeRange()[0]) - title = wintitle + " Spectra" - if ((dataOut.azimuth!=None) and (dataOut.zenith!=None)): - title = title + '_' + 'azimuth,zenith=%2.2f,%2.2f'%(dataOut.azimuth, dataOut.zenith) - - if not self.isConfig: - - nplots = len(channelIndexList) - - self.setup(id=id, - nplots=nplots, - wintitle=wintitle, - showprofile=showprofile, - show=show) - - if xmin == None: xmin = numpy.nanmin(x) - if xmax == None: xmax = numpy.nanmax(x) - if ymin == None: ymin = numpy.nanmin(y) - if ymax == None: ymax = numpy.nanmax(y) - if zmin == None: zmin = numpy.floor(numpy.nanmin(noisedB)) - 3 - if zmax == None: zmax = numpy.ceil(numpy.nanmax(avgdB)) + 3 - - self.FTP_WEI = ftp_wei - self.EXP_CODE = exp_code - self.SUB_EXP_CODE = sub_exp_code - self.PLOT_POS = plot_pos - - self.isConfig = True - - self.setWinTitle(title) - - for i in range(self.nplots): - index = channelIndexList[i] - str_datetime = '%s %s'%(thisDatetime.strftime("%Y/%m/%d"),thisDatetime.strftime("%H:%M:%S")) - title = "Channel %d: %4.2fdB: %s" %(dataOut.channelList[index], noisedB[index], str_datetime) - if len(dataOut.beam.codeList) != 0: - title = "Ch%d:%4.2fdB,%2.2f,%2.2f:%s" %(dataOut.channelList[index], noisedB[index], dataOut.beam.azimuthList[index], dataOut.beam.zenithList[index], str_datetime) + CODE = 'param' - axes = self.axesList[i*self.__nsubplots] - axes.pcolor(x, y, zdB[index,:,:], - xmin=xmin, xmax=xmax, ymin=ymin, ymax=ymax, zmin=zmin, zmax=zmax, - xlabel=xlabel, ylabel=ylabel, title=title, colormap=colormap, - ticksize=9, cblabel='') + def setup(self): - if self.__showprofile: - axes = self.axesList[i*self.__nsubplots +1] - axes.pline(avgdB[index,:], y, - xmin=zmin, xmax=zmax, ymin=ymin, ymax=ymax, - xlabel='dB', ylabel='', title='', - ytick_visible=False, - grid='x') + self.ncols = 1 + self.nrows = 1 + self.width = 7.2 + self.height = 7.2 + self.nplots = 1 + self.xlabel = 'Zonal Zenith Angle (deg)' + self.ylabel = 'Meridional Zenith Angle (deg)' + self.polar = True + self.ymin = -180 + self.ymax = 180 + self.colorbar = False - noiseline = numpy.repeat(noisedB[index], len(y)) - axes.addpline(noiseline, y, idline=1, color="black", linestyle="dashed", lw=2) + def plot(self): - self.draw() - - if figfile == None: - str_datetime = thisDatetime.strftime("%Y%m%d_%H%M%S") - name = str_datetime - if ((dataOut.azimuth!=None) and (dataOut.zenith!=None)): - name = name + '_az' + '_%2.2f'%(dataOut.azimuth) + '_zn' + '_%2.2f'%(dataOut.zenith) - figfile = self.getFilename(name) - - self.save(figpath=figpath, - figfile=figfile, - save=save, - ftp=ftp, - wr_period=wr_period, - thisDatetime=thisDatetime) - - - -class MomentsPlot(Figure): - - isConfig = None - __nsubplots = None - - WIDTHPROF = None - HEIGHTPROF = None - PREFIX = 'prm' - def __init__(self, **kwargs): - Figure.__init__(self, **kwargs) - self.isConfig = False - self.__nsubplots = 1 - - self.WIDTH = 280 - self.HEIGHT = 250 - self.WIDTHPROF = 120 - self.HEIGHTPROF = 0 - self.counter_imagwr = 0 - - self.PLOT_CODE = MOMENTS_CODE - - self.FTP_WEI = None - self.EXP_CODE = None - self.SUB_EXP_CODE = None - self.PLOT_POS = None - - def getSubplots(self): - - ncol = int(numpy.sqrt(self.nplots)+0.9) - nrow = int(self.nplots*1./ncol + 0.9) - - return nrow, ncol - - def setup(self, id, nplots, wintitle, showprofile=True, show=True): - - self.__showprofile = showprofile - self.nplots = nplots - - ncolspan = 1 - colspan = 1 - if showprofile: - ncolspan = 3 - colspan = 2 - self.__nsubplots = 2 - - self.createFigure(id = id, - wintitle = wintitle, - widthplot = self.WIDTH + self.WIDTHPROF, - heightplot = self.HEIGHT + self.HEIGHTPROF, - show=show) - - nrow, ncol = self.getSubplots() - - counter = 0 - for y in range(nrow): - for x in range(ncol): - - if counter >= self.nplots: - break - - self.addAxes(nrow, ncol*ncolspan, y, x*ncolspan, colspan, 1) - - if showprofile: - self.addAxes(nrow, ncol*ncolspan, y, x*ncolspan+colspan, 1, 1) - - counter += 1 - - def run(self, dataOut, id, wintitle="", channelList=None, showprofile=True, - xmin=None, xmax=None, ymin=None, ymax=None, zmin=None, zmax=None, - save=False, figpath='./', figfile=None, show=True, ftp=False, wr_period=1, - server=None, folder=None, username=None, password=None, - ftp_wei=0, exp_code=0, sub_exp_code=0, plot_pos=0, realtime=False): - - """ - - Input: - dataOut : - id : - wintitle : - channelList : - showProfile : - xmin : None, - xmax : None, - ymin : None, - ymax : None, - zmin : None, - zmax : None - """ - - if dataOut.flagNoData: - return None - - if realtime: - if not(isRealtime(utcdatatime = dataOut.utctime)): - print 'Skipping this plot function' - return - - if channelList == None: - channelIndexList = dataOut.channelIndexList - else: - channelIndexList = [] - for channel in channelList: - if channel not in dataOut.channelList: - raise ValueError, "Channel %d is not in dataOut.channelList" - channelIndexList.append(dataOut.channelList.index(channel)) - - factor = dataOut.normFactor - x = dataOut.abscissaList - y = dataOut.heightList - - z = dataOut.data_pre[channelIndexList,:,:]/factor - z = numpy.where(numpy.isfinite(z), z, numpy.NAN) - avg = numpy.average(z, axis=1) - noise = dataOut.noise/factor - - zdB = 10*numpy.log10(z) - avgdB = 10*numpy.log10(avg) - noisedB = 10*numpy.log10(noise) - - #thisDatetime = dataOut.datatime - thisDatetime = datetime.datetime.utcfromtimestamp(dataOut.getTimeRange()[0]) - title = wintitle + " Parameters" - xlabel = "Velocity (m/s)" - ylabel = "Range (Km)" - - update_figfile = False - - if not self.isConfig: - - nplots = len(channelIndexList) - - self.setup(id=id, - nplots=nplots, - wintitle=wintitle, - showprofile=showprofile, - show=show) - - if xmin == None: xmin = numpy.nanmin(x) - if xmax == None: xmax = numpy.nanmax(x) - if ymin == None: ymin = numpy.nanmin(y) - if ymax == None: ymax = numpy.nanmax(y) - if zmin == None: zmin = numpy.nanmin(avgdB)*0.9 - if zmax == None: zmax = numpy.nanmax(avgdB)*0.9 - - self.FTP_WEI = ftp_wei - self.EXP_CODE = exp_code - self.SUB_EXP_CODE = sub_exp_code - self.PLOT_POS = plot_pos - - self.isConfig = True - update_figfile = True - - self.setWinTitle(title) - - for i in range(self.nplots): - str_datetime = '%s %s'%(thisDatetime.strftime("%Y/%m/%d"),thisDatetime.strftime("%H:%M:%S")) - title = "Channel %d: %4.2fdB: %s" %(dataOut.channelList[i], noisedB[i], str_datetime) - axes = self.axesList[i*self.__nsubplots] - axes.pcolor(x, y, zdB[i,:,:], - xmin=xmin, xmax=xmax, ymin=ymin, ymax=ymax, zmin=zmin, zmax=zmax, - xlabel=xlabel, ylabel=ylabel, title=title, - ticksize=9, cblabel='') - #Mean Line - mean = dataOut.data_param[i, 1, :] - axes.addpline(mean, y, idline=0, color="black", linestyle="solid", lw=1) - - if self.__showprofile: - axes = self.axesList[i*self.__nsubplots +1] - axes.pline(avgdB[i], y, - xmin=zmin, xmax=zmax, ymin=ymin, ymax=ymax, - xlabel='dB', ylabel='', title='', - ytick_visible=False, - grid='x') - - noiseline = numpy.repeat(noisedB[i], len(y)) - axes.addpline(noiseline, y, idline=1, color="black", linestyle="dashed", lw=2) - - self.draw() - - self.save(figpath=figpath, - figfile=figfile, - save=save, - ftp=ftp, - wr_period=wr_period, - thisDatetime=thisDatetime) - - - -class SkyMapPlot(Figure): - - __isConfig = None - __nsubplots = None - - WIDTHPROF = None - HEIGHTPROF = None - PREFIX = 'mmap' - - def __init__(self, **kwargs): - Figure.__init__(self, **kwargs) - self.isConfig = False - self.__nsubplots = 1 - -# self.WIDTH = 280 -# self.HEIGHT = 250 - self.WIDTH = 600 - self.HEIGHT = 600 - self.WIDTHPROF = 120 - self.HEIGHTPROF = 0 - self.counter_imagwr = 0 - - self.PLOT_CODE = MSKYMAP_CODE - - self.FTP_WEI = None - self.EXP_CODE = None - self.SUB_EXP_CODE = None - self.PLOT_POS = None - - def getSubplots(self): - - ncol = int(numpy.sqrt(self.nplots)+0.9) - nrow = int(self.nplots*1./ncol + 0.9) - - return nrow, ncol - - def setup(self, id, nplots, wintitle, showprofile=False, show=True): - - self.__showprofile = showprofile - self.nplots = nplots - - ncolspan = 1 - colspan = 1 - - self.createFigure(id = id, - wintitle = wintitle, - widthplot = self.WIDTH, #+ self.WIDTHPROF, - heightplot = self.HEIGHT,# + self.HEIGHTPROF, - show=show) - - nrow, ncol = 1,1 - counter = 0 - x = 0 - y = 0 - self.addAxes(1, 1, 0, 0, 1, 1, True) - - def run(self, dataOut, id, wintitle="", channelList=None, showprofile=False, - tmin=0, tmax=24, timerange=None, - save=False, figpath='./', figfile=None, show=True, ftp=False, wr_period=1, - server=None, folder=None, username=None, password=None, - ftp_wei=0, exp_code=0, sub_exp_code=0, plot_pos=0, realtime=False): - - """ - - Input: - dataOut : - id : - wintitle : - channelList : - showProfile : - xmin : None, - xmax : None, - ymin : None, - ymax : None, - zmin : None, - zmax : None - """ - - arrayParameters = dataOut.data_param - error = arrayParameters[:,-1] + arrayParameters = numpy.concatenate(self.data['param']) + error = arrayParameters[:, -1] indValid = numpy.where(error == 0)[0] - finalMeteor = arrayParameters[indValid,:] - finalAzimuth = finalMeteor[:,3] - finalZenith = finalMeteor[:,4] + finalMeteor = arrayParameters[indValid, :] + finalAzimuth = finalMeteor[:, 3] + finalZenith = finalMeteor[:, 4] - x = finalAzimuth*numpy.pi/180 + x = finalAzimuth * numpy.pi / 180 y = finalZenith - x1 = [dataOut.ltctime, dataOut.ltctime] - - #thisDatetime = dataOut.datatime - thisDatetime = datetime.datetime.utcfromtimestamp(dataOut.ltctime) - title = wintitle + " Parameters" - xlabel = "Zonal Zenith Angle (deg) " - ylabel = "Meridional Zenith Angle (deg)" - update_figfile = False - - if not self.isConfig: - - nplots = 1 - - self.setup(id=id, - nplots=nplots, - wintitle=wintitle, - showprofile=showprofile, - show=show) - - if self.xmin is None and self.xmax is None: - self.xmin, self.xmax = self.getTimeLim(x1, tmin, tmax, timerange) - - if timerange != None: - self.timerange = timerange - else: - self.timerange = self.xmax - self.xmin - - self.FTP_WEI = ftp_wei - self.EXP_CODE = exp_code - self.SUB_EXP_CODE = sub_exp_code - self.PLOT_POS = plot_pos - self.name = thisDatetime.strftime("%Y%m%d_%H%M%S") - self.firstdate = '%s %s'%(thisDatetime.strftime("%Y/%m/%d"),thisDatetime.strftime("%H:%M:%S")) - self.isConfig = True - update_figfile = True - - self.setWinTitle(title) - - i = 0 - str_datetime = '%s %s'%(thisDatetime.strftime("%Y/%m/%d"),thisDatetime.strftime("%H:%M:%S")) - - axes = self.axesList[i*self.__nsubplots] - nevents = axes.x_buffer.shape[0] + x.shape[0] - title = "Meteor Detection Sky Map\n %s - %s \n Number of events: %5.0f\n" %(self.firstdate,str_datetime,nevents) - axes.polar(x, y, - title=title, xlabel=xlabel, ylabel=ylabel, - ticksize=9, cblabel='') - - self.draw() - - self.save(figpath=figpath, - figfile=figfile, - save=save, - ftp=ftp, - wr_period=wr_period, - thisDatetime=thisDatetime, - update_figfile=update_figfile) - - if dataOut.ltctime >= self.xmax: - self.isConfigmagwr = wr_period - self.isConfig = False - update_figfile = True - axes.__firsttime = True - self.xmin += self.timerange - self.xmax += self.timerange - - - - -class WindProfilerPlot(Figure): - - __isConfig = None - __nsubplots = None - - WIDTHPROF = None - HEIGHTPROF = None - PREFIX = 'wind' - - def __init__(self, **kwargs): - Figure.__init__(self, **kwargs) - self.timerange = None - self.isConfig = False - self.__nsubplots = 1 - - self.WIDTH = 800 - self.HEIGHT = 300 - self.WIDTHPROF = 120 - self.HEIGHTPROF = 0 - self.counter_imagwr = 0 - - self.PLOT_CODE = WIND_CODE - - self.FTP_WEI = None - self.EXP_CODE = None - self.SUB_EXP_CODE = None - self.PLOT_POS = None - self.tmin = None - self.tmax = None - - self.xmin = None - self.xmax = None - - self.figfile = None - - def getSubplots(self): - - ncol = 1 - nrow = self.nplots - - return nrow, ncol - - def setup(self, id, nplots, wintitle, showprofile=True, show=True): - - self.__showprofile = showprofile - self.nplots = nplots - - ncolspan = 1 - colspan = 1 - - self.createFigure(id = id, - wintitle = wintitle, - widthplot = self.WIDTH + self.WIDTHPROF, - heightplot = self.HEIGHT + self.HEIGHTPROF, - show=show) - - nrow, ncol = self.getSubplots() - - counter = 0 - for y in range(nrow): - if counter >= self.nplots: - break - - self.addAxes(nrow, ncol*ncolspan, y, 0, colspan, 1) - counter += 1 - - def run(self, dataOut, id, wintitle="", channelList=None, showprofile='False', - xmin=None, xmax=None, ymin=None, ymax=None, zmin=None, zmax=None, - zmax_ver = None, zmin_ver = None, SNRmin = None, SNRmax = None, - timerange=None, SNRthresh = None, - save=False, figpath='./', lastone=0,figfile=None, ftp=False, wr_period=1, show=True, - server=None, folder=None, username=None, password=None, - ftp_wei=0, exp_code=0, sub_exp_code=0, plot_pos=0): - """ - - Input: - dataOut : - id : - wintitle : - channelList : - showProfile : - xmin : None, - xmax : None, - ymin : None, - ymax : None, - zmin : None, - zmax : None - """ - -# if timerange is not None: -# self.timerange = timerange -# -# tmin = None -# tmax = None - - x = dataOut.getTimeRange1(dataOut.paramInterval) - y = dataOut.heightList - z = dataOut.data_output.copy() - nplots = z.shape[0] #Number of wind dimensions estimated - nplotsw = nplots - - - #If there is a SNR function defined - if dataOut.data_SNR is not None: - nplots += 1 - SNR = dataOut.data_SNR - SNRavg = numpy.average(SNR, axis=0) - - SNRdB = 10*numpy.log10(SNR) - SNRavgdB = 10*numpy.log10(SNRavg) - - if SNRthresh == None: SNRthresh = -5.0 - ind = numpy.where(SNRavg < 10**(SNRthresh/10))[0] - - for i in range(nplotsw): - z[i,ind] = numpy.nan - - thisDatetime = datetime.datetime.utcfromtimestamp(dataOut.ltctime) - #thisDatetime = datetime.datetime.now() - title = wintitle + "Wind" - xlabel = "" - ylabel = "Height (km)" - update_figfile = False - - if not self.isConfig: - - self.setup(id=id, - nplots=nplots, - wintitle=wintitle, - showprofile=showprofile, - show=show) - - if timerange is not None: - self.timerange = timerange - - self.xmin, self.xmax = self.getTimeLim(x, xmin, xmax, timerange) - - if ymin == None: ymin = numpy.nanmin(y) - if ymax == None: ymax = numpy.nanmax(y) - - if zmax == None: zmax = numpy.nanmax(abs(z[range(2),:])) - #if numpy.isnan(zmax): zmax = 50 - if zmin == None: zmin = -zmax - - if nplotsw == 3: - if zmax_ver == None: zmax_ver = numpy.nanmax(abs(z[2,:])) - if zmin_ver == None: zmin_ver = -zmax_ver - - if dataOut.data_SNR is not None: - if SNRmin == None: SNRmin = numpy.nanmin(SNRavgdB) - if SNRmax == None: SNRmax = numpy.nanmax(SNRavgdB) - - - self.FTP_WEI = ftp_wei - self.EXP_CODE = exp_code - self.SUB_EXP_CODE = sub_exp_code - self.PLOT_POS = plot_pos - - self.name = thisDatetime.strftime("%Y%m%d_%H%M%S") - self.isConfig = True - self.figfile = figfile - update_figfile = True - - self.setWinTitle(title) - - if ((self.xmax - x[1]) < (x[1]-x[0])): - x[1] = self.xmax - - strWind = ['Zonal', 'Meridional', 'Vertical'] - strCb = ['Velocity (m/s)','Velocity (m/s)','Velocity (cm/s)'] - zmaxVector = [zmax, zmax, zmax_ver] - zminVector = [zmin, zmin, zmin_ver] - windFactor = [1,1,100] - - for i in range(nplotsw): - - title = "%s Wind: %s" %(strWind[i], thisDatetime.strftime("%Y/%m/%d %H:%M:%S")) - axes = self.axesList[i*self.__nsubplots] - - z1 = z[i,:].reshape((1,-1))*windFactor[i] - #z1=numpy.ma.masked_where(z1==0.,z1) - - axes.pcolorbuffer(x, y, z1, - xmin=self.xmin, xmax=self.xmax, ymin=ymin, ymax=ymax, zmin=zminVector[i], zmax=zmaxVector[i], - xlabel=xlabel, ylabel=ylabel, title=title, rti=True, XAxisAsTime=True, - ticksize=9, cblabel=strCb[i], cbsize="1%", colormap="seismic" ) - - if dataOut.data_SNR is not None: - i += 1 - title = "Signal Noise Ratio (SNR): %s" %(thisDatetime.strftime("%Y/%m/%d %H:%M:%S")) - axes = self.axesList[i*self.__nsubplots] - SNRavgdB = SNRavgdB.reshape((1,-1)) - axes.pcolorbuffer(x, y, SNRavgdB, - xmin=self.xmin, xmax=self.xmax, ymin=ymin, ymax=ymax, zmin=SNRmin, zmax=SNRmax, - xlabel=xlabel, ylabel=ylabel, title=title, rti=True, XAxisAsTime=True, - ticksize=9, cblabel='', cbsize="1%", colormap="jet") - - self.draw() - self.save(figpath=figpath, - figfile=figfile, - save=save, - ftp=ftp, - wr_period=wr_period, - thisDatetime=thisDatetime, - update_figfile=update_figfile) + ax = self.axes[0] - if dataOut.ltctime + dataOut.paramInterval >= self.xmax: - self.counter_imagwr = wr_period - self.isConfig = False - update_figfile = True - - -class ParametersPlot(Figure): - - __isConfig = None - __nsubplots = None - - WIDTHPROF = None - HEIGHTPROF = None - PREFIX = 'param' - - nplots = None - nchan = None - - def __init__(self, **kwargs): - Figure.__init__(self, **kwargs) - self.timerange = None - self.isConfig = False - self.__nsubplots = 1 - - self.WIDTH = 800 - self.HEIGHT = 180 - self.WIDTHPROF = 120 - self.HEIGHTPROF = 0 - self.counter_imagwr = 0 - - self.PLOT_CODE = RTI_CODE - - self.FTP_WEI = None - self.EXP_CODE = None - self.SUB_EXP_CODE = None - self.PLOT_POS = None - self.tmin = None - self.tmax = None - - self.xmin = None - self.xmax = None - - self.figfile = None - - def getSubplots(self): - - ncol = 1 - nrow = self.nplots - - return nrow, ncol - - def setup(self, id, nplots, wintitle, show=True): - - self.nplots = nplots - - ncolspan = 1 - colspan = 1 - - self.createFigure(id = id, - wintitle = wintitle, - widthplot = self.WIDTH + self.WIDTHPROF, - heightplot = self.HEIGHT + self.HEIGHTPROF, - show=show) - - nrow, ncol = self.getSubplots() - - counter = 0 - for y in range(nrow): - for x in range(ncol): - - if counter >= self.nplots: - break - - self.addAxes(nrow, ncol*ncolspan, y, x*ncolspan, colspan, 1) - - counter += 1 - - def run(self, dataOut, id, wintitle="", channelList=None, paramIndex = 0, colormap="jet", - xmin=None, xmax=None, ymin=None, ymax=None, zmin=None, zmax=None, timerange=None, - showSNR=False, SNRthresh = -numpy.inf, SNRmin=None, SNRmax=None, - save=False, figpath='./', lastone=0,figfile=None, ftp=False, wr_period=1, show=True, - server=None, folder=None, username=None, password=None, - ftp_wei=0, exp_code=0, sub_exp_code=0, plot_pos=0, HEIGHT=None): - """ - - Input: - dataOut : - id : - wintitle : - channelList : - showProfile : - xmin : None, - xmax : None, - ymin : None, - ymax : None, - zmin : None, - zmax : None - """ - - if HEIGHT is not None: - self.HEIGHT = HEIGHT + if ax.firsttime: + ax.plot = ax.plot(x, y, 'bo', markersize=5)[0] + else: + ax.plot.set_data(x, y) + + dt1 = self.getDateTime(self.data.min_time).strftime('%y/%m/%d %H:%M:%S') + dt2 = self.getDateTime(self.data.max_time).strftime('%y/%m/%d %H:%M:%S') + title = 'Meteor Detection Sky Map\n %s - %s \n Number of events: %5.0f\n' % (dt1, + dt2, + len(x)) + self.titles[0] = title + + +class ParametersPlot(RTIPlot): + ''' + Plot for data_param object + ''' + + CODE = 'param' + colormap = 'seismic' + plot_name = 'Parameters' + + def setup(self): + self.xaxis = 'time' + self.ncols = 1 + self.nrows = self.data.shape(self.CODE)[0] + self.nplots = self.nrows + self.plots_adjust.update({'hspace':0.8, 'left': 0.1, 'bottom': 0.08, 'right':0.95, 'top': 0.95}) + if not self.xlabel: + self.xlabel = 'Time' - if not isTimeInHourRange(dataOut.datatime, xmin, xmax): - return - - if channelList == None: - channelIndexList = range(dataOut.data_param.shape[0]) + if self.showSNR: + self.nrows += 1 + self.nplots += 1 + + self.ylabel = 'Height [km]' + if not self.titles: + self.titles = self.data.parameters \ + if self.data.parameters else ['Param {}'.format(x) for x in range(self.nrows)] + if self.showSNR: + self.titles.append('SNR') + + def plot(self): + self.data.normalize_heights() + self.x = self.data.times + self.y = self.data.heights + if self.showSNR: + self.z = numpy.concatenate( + (self.data[self.CODE], self.data['snr']) + ) else: - channelIndexList = [] - for channel in channelList: - if channel not in dataOut.channelList: - raise ValueError, "Channel %d is not in dataOut.channelList" - channelIndexList.append(dataOut.channelList.index(channel)) + self.z = self.data[self.CODE] - x = dataOut.getTimeRange1(dataOut.paramInterval) - y = dataOut.getHeiRange() + self.z = numpy.ma.masked_invalid(self.z) - if dataOut.data_param.ndim == 3: - z = dataOut.data_param[channelIndexList,paramIndex,:] + if self.decimation is None: + x, y, z = self.fill_gaps(self.x, self.y, self.z) else: - z = dataOut.data_param[channelIndexList,:] - - if showSNR: - #SNR data - SNRarray = dataOut.data_SNR[channelIndexList,:] - SNRdB = 10*numpy.log10(SNRarray) - ind = numpy.where(SNRdB < SNRthresh) - z[ind] = numpy.nan - - thisDatetime = dataOut.datatime -# thisDatetime = datetime.datetime.utcfromtimestamp(dataOut.getTimeRange()[0]) - title = wintitle + " Parameters Plot" #: %s" %(thisDatetime.strftime("%d-%b-%Y")) - xlabel = "" - ylabel = "Range (Km)" - - update_figfile = False - - if not self.isConfig: - - nchan = len(channelIndexList) - self.nchan = nchan - self.plotFact = 1 - nplots = nchan - - if showSNR: - nplots = nchan*2 - self.plotFact = 2 - if SNRmin == None: SNRmin = numpy.nanmin(SNRdB) - if SNRmax == None: SNRmax = numpy.nanmax(SNRdB) - - self.setup(id=id, - nplots=nplots, - wintitle=wintitle, - show=show) - - if timerange != None: - self.timerange = timerange - - self.xmin, self.xmax = self.getTimeLim(x, xmin, xmax, timerange) - - if ymin == None: ymin = numpy.nanmin(y) - if ymax == None: ymax = numpy.nanmax(y) - if zmin == None: zmin = numpy.nanmin(z) - if zmax == None: zmax = numpy.nanmax(z) - - self.FTP_WEI = ftp_wei - self.EXP_CODE = exp_code - self.SUB_EXP_CODE = sub_exp_code - self.PLOT_POS = plot_pos - - self.name = thisDatetime.strftime("%Y%m%d_%H%M%S") - self.isConfig = True - self.figfile = figfile - update_figfile = True - - self.setWinTitle(title) - - for i in range(self.nchan): - index = channelIndexList[i] - title = "Channel %d: %s" %(dataOut.channelList[index], thisDatetime.strftime("%Y/%m/%d %H:%M:%S")) - axes = self.axesList[i*self.plotFact] - z1 = z[i,:].reshape((1,-1)) - axes.pcolorbuffer(x, y, z1, - xmin=self.xmin, xmax=self.xmax, ymin=ymin, ymax=ymax, zmin=zmin, zmax=zmax, - xlabel=xlabel, ylabel=ylabel, title=title, rti=True, XAxisAsTime=True, - ticksize=9, cblabel='', cbsize="1%",colormap=colormap) - - if showSNR: - title = "Channel %d SNR: %s" %(dataOut.channelList[index], thisDatetime.strftime("%Y/%m/%d %H:%M:%S")) - axes = self.axesList[i*self.plotFact + 1] - SNRdB1 = SNRdB[i,:].reshape((1,-1)) - axes.pcolorbuffer(x, y, SNRdB1, - xmin=self.xmin, xmax=self.xmax, ymin=ymin, ymax=ymax, zmin=SNRmin, zmax=SNRmax, - xlabel=xlabel, ylabel=ylabel, title=title, rti=True, XAxisAsTime=True, - ticksize=9, cblabel='', cbsize="1%",colormap='jet') - - - self.draw() + x, y, z = self.fill_gaps(*self.decimate()) - if dataOut.ltctime >= self.xmax: - self.counter_imagwr = wr_period - self.isConfig = False - update_figfile = True - - self.save(figpath=figpath, - figfile=figfile, - save=save, - ftp=ftp, - wr_period=wr_period, - thisDatetime=thisDatetime, - update_figfile=update_figfile) - - - -class Parameters1Plot(Figure): - - __isConfig = None - __nsubplots = None - - WIDTHPROF = None - HEIGHTPROF = None - PREFIX = 'prm' - - def __init__(self, **kwargs): - Figure.__init__(self, **kwargs) - self.timerange = 2*60*60 - self.isConfig = False - self.__nsubplots = 1 - - self.WIDTH = 800 - self.HEIGHT = 180 - self.WIDTHPROF = 120 - self.HEIGHTPROF = 0 - self.counter_imagwr = 0 + for n, ax in enumerate(self.axes): - self.PLOT_CODE = PARMS_CODE - - self.FTP_WEI = None - self.EXP_CODE = None - self.SUB_EXP_CODE = None - self.PLOT_POS = None - self.tmin = None - self.tmax = None - - self.xmin = None - self.xmax = None - - self.figfile = None - - def getSubplots(self): - - ncol = 1 - nrow = self.nplots + self.zmax = self.zmax if self.zmax is not None else numpy.max( + self.z[n]) + self.zmin = self.zmin if self.zmin is not None else numpy.min( + self.z[n]) - return nrow, ncol - - def setup(self, id, nplots, wintitle, showprofile=True, show=True): + if ax.firsttime: + if self.zlimits is not None: + self.zmin, self.zmax = self.zlimits[n] - self.__showprofile = showprofile - self.nplots = nplots - - ncolspan = 1 - colspan = 1 - - self.createFigure(id = id, - wintitle = wintitle, - widthplot = self.WIDTH + self.WIDTHPROF, - heightplot = self.HEIGHT + self.HEIGHTPROF, - show=show) - - nrow, ncol = self.getSubplots() - - counter = 0 - for y in range(nrow): - for x in range(ncol): - - if counter >= self.nplots: - break - - self.addAxes(nrow, ncol*ncolspan, y, x*ncolspan, colspan, 1) - - if showprofile: - self.addAxes(nrow, ncol*ncolspan, y, x*ncolspan+colspan, 1, 1) - - counter += 1 - - def run(self, dataOut, id, wintitle="", channelList=None, showprofile=False, - xmin=None, xmax=None, ymin=None, ymax=None, zmin=None, zmax=None,timerange=None, - parameterIndex = None, onlyPositive = False, - SNRthresh = -numpy.inf, SNR = True, SNRmin = None, SNRmax = None, onlySNR = False, - DOP = True, - zlabel = "", parameterName = "", parameterObject = "data_param", - save=False, figpath='./', lastone=0,figfile=None, ftp=False, wr_period=1, show=True, - server=None, folder=None, username=None, password=None, - ftp_wei=0, exp_code=0, sub_exp_code=0, plot_pos=0): - #print inspect.getargspec(self.run).args - """ - - Input: - dataOut : - id : - wintitle : - channelList : - showProfile : - xmin : None, - xmax : None, - ymin : None, - ymax : None, - zmin : None, - zmax : None - """ - - data_param = getattr(dataOut, parameterObject) - - if channelList == None: - channelIndexList = numpy.arange(data_param.shape[0]) + ax.plt = ax.pcolormesh(x, y, z[n].T * self.factors[n], + vmin=self.zmin, + vmax=self.zmax, + cmap=self.cmaps[n] + ) + else: + if self.zlimits is not None: + self.zmin, self.zmax = self.zlimits[n] + ax.collections.remove(ax.collections[0]) + ax.plt = ax.pcolormesh(x, y, z[n].T * self.factors[n], + vmin=self.zmin, + vmax=self.zmax, + cmap=self.cmaps[n] + ) + + +class OutputPlot(ParametersPlot): + ''' + Plot data_output object + ''' + + CODE = 'output' + colormap = 'seismic' + plot_name = 'Output' + + +class PolarMapPlot(Plot): + ''' + Plot for weather radar + ''' + + CODE = 'param' + colormap = 'seismic' + + def setup(self): + self.ncols = 1 + self.nrows = 1 + self.width = 9 + self.height = 8 + self.mode = self.data.meta['mode'] + if self.channels is not None: + self.nplots = len(self.channels) + self.nrows = len(self.channels) else: - channelIndexList = numpy.array(channelList) - - nchan = len(channelIndexList) #Number of channels being plotted - - if nchan < 1: - return - - nGraphsByChannel = 0 - - if SNR: - nGraphsByChannel += 1 - if DOP: - nGraphsByChannel += 1 - - if nGraphsByChannel < 1: - return - - nplots = nGraphsByChannel*nchan - - if timerange is not None: - self.timerange = timerange - - #tmin = None - #tmax = None - if parameterIndex == None: - parameterIndex = 1 - - x = dataOut.getTimeRange1(dataOut.paramInterval) - y = dataOut.heightList - - if dataOut.data_param.ndim == 3: - z = dataOut.data_param[channelIndexList,parameterIndex,:] + self.nplots = self.data.shape(self.CODE)[0] + self.nrows = self.nplots + self.channels = list(range(self.nplots)) + if self.mode == 'E': + self.xlabel = 'Longitude' + self.ylabel = 'Latitude' else: - z = dataOut.data_param[channelIndexList,:] - - if dataOut.data_SNR is not None: - if dataOut.data_SNR.ndim == 2: - SNRavg = numpy.average(dataOut.data_SNR, axis=0) + self.xlabel = 'Range (km)' + self.ylabel = 'Height (km)' + self.bgcolor = 'white' + self.cb_labels = self.data.meta['units'] + self.lat = self.data.meta['latitude'] + self.lon = self.data.meta['longitude'] + self.xmin, self.xmax = float( + km2deg(self.xmin) + self.lon), float(km2deg(self.xmax) + self.lon) + self.ymin, self.ymax = float( + km2deg(self.ymin) + self.lat), float(km2deg(self.ymax) + self.lat) + # self.polar = True + + def plot(self): + + for n, ax in enumerate(self.axes): + data = self.data['param'][self.channels[n]] + + zeniths = numpy.linspace( + 0, self.data.meta['max_range'], data.shape[1]) + if self.mode == 'E': + azimuths = -numpy.radians(self.data.heights)+numpy.pi/2 + r, theta = numpy.meshgrid(zeniths, azimuths) + x, y = r*numpy.cos(theta)*numpy.cos(numpy.radians(self.data.meta['elevation'])), r*numpy.sin( + theta)*numpy.cos(numpy.radians(self.data.meta['elevation'])) + x = km2deg(x) + self.lon + y = km2deg(y) + self.lat else: - SNRavg = dataOut.data_SNR - SNRdB = 10*numpy.log10(SNRavg) - - thisDatetime = datetime.datetime.utcfromtimestamp(dataOut.getTimeRange()[0]) - title = wintitle + " Parameters Plot" #: %s" %(thisDatetime.strftime("%d-%b-%Y")) - xlabel = "" - ylabel = "Range (Km)" - - if onlyPositive: - colormap = "jet" - zmin = 0 - else: colormap = "RdBu_r" - - if not self.isConfig: - - self.setup(id=id, - nplots=nplots, - wintitle=wintitle, - showprofile=showprofile, - show=show) - - self.xmin, self.xmax = self.getTimeLim(x, xmin, xmax, timerange) - - if ymin == None: ymin = numpy.nanmin(y) - if ymax == None: ymax = numpy.nanmax(y) - if zmin == None: zmin = numpy.nanmin(z) - if zmax == None: zmax = numpy.nanmax(z) - - if SNR: - if SNRmin == None: SNRmin = numpy.nanmin(SNRdB) - if SNRmax == None: SNRmax = numpy.nanmax(SNRdB) - - self.FTP_WEI = ftp_wei - self.EXP_CODE = exp_code - self.SUB_EXP_CODE = sub_exp_code - self.PLOT_POS = plot_pos - - self.name = thisDatetime.strftime("%Y%m%d_%H%M%S") - self.isConfig = True - self.figfile = figfile - - self.setWinTitle(title) - - if ((self.xmax - x[1]) < (x[1]-x[0])): - x[1] = self.xmax - - for i in range(nchan): - - if (SNR and not onlySNR): j = 2*i - else: j = i - - j = nGraphsByChannel*i - - if ((dataOut.azimuth!=None) and (dataOut.zenith!=None)): - title = title + '_' + 'azimuth,zenith=%2.2f,%2.2f'%(dataOut.azimuth, dataOut.zenith) - - if not onlySNR: - axes = self.axesList[j*self.__nsubplots] - z1 = z[i,:].reshape((1,-1)) - axes.pcolorbuffer(x, y, z1, - xmin=self.xmin, xmax=self.xmax, ymin=ymin, ymax=ymax, zmin=zmin, zmax=zmax, - xlabel=xlabel, ylabel=ylabel, title=title, rti=True, XAxisAsTime=True,colormap=colormap, - ticksize=9, cblabel=zlabel, cbsize="1%") - - if DOP: - title = "%s Channel %d: %s" %(parameterName, channelIndexList[i], thisDatetime.strftime("%Y/%m/%d %H:%M:%S")) - - if ((dataOut.azimuth!=None) and (dataOut.zenith!=None)): - title = title + '_' + 'azimuth,zenith=%2.2f,%2.2f'%(dataOut.azimuth, dataOut.zenith) - axes = self.axesList[j] - z1 = z[i,:].reshape((1,-1)) - axes.pcolorbuffer(x, y, z1, - xmin=self.xmin, xmax=self.xmax, ymin=ymin, ymax=ymax, zmin=zmin, zmax=zmax, - xlabel=xlabel, ylabel=ylabel, title=title, rti=True, XAxisAsTime=True,colormap=colormap, - ticksize=9, cblabel=zlabel, cbsize="1%") - - if SNR: - title = "Channel %d Signal Noise Ratio (SNR): %s" %(channelIndexList[i], thisDatetime.strftime("%Y/%m/%d %H:%M:%S")) - axes = self.axesList[(j)*self.__nsubplots] - if not onlySNR: - axes = self.axesList[(j + 1)*self.__nsubplots] - - axes = self.axesList[(j + nGraphsByChannel-1)] - z1 = SNRdB.reshape((1,-1)) - axes.pcolorbuffer(x, y, z1, - xmin=self.xmin, xmax=self.xmax, ymin=ymin, ymax=ymax, zmin=SNRmin, zmax=SNRmax, - xlabel=xlabel, ylabel=ylabel, title=title, rti=True, XAxisAsTime=True,colormap="jet", - ticksize=9, cblabel=zlabel, cbsize="1%") - - - - self.draw() - - if x[1] >= self.axesList[0].xmax: - self.counter_imagwr = wr_period - self.isConfig = False - self.figfile = None - - self.save(figpath=figpath, - figfile=figfile, - save=save, - ftp=ftp, - wr_period=wr_period, - thisDatetime=thisDatetime, - update_figfile=False) - -class SpectralFittingPlot(Figure): - - __isConfig = None - __nsubplots = None - - WIDTHPROF = None - HEIGHTPROF = None - PREFIX = 'prm' - - - N = None - ippSeconds = None - - def __init__(self, **kwargs): - Figure.__init__(self, **kwargs) - self.isConfig = False - self.__nsubplots = 1 - - self.PLOT_CODE = SPECFIT_CODE - - self.WIDTH = 450 - self.HEIGHT = 250 - self.WIDTHPROF = 0 - self.HEIGHTPROF = 0 - - def getSubplots(self): - - ncol = int(numpy.sqrt(self.nplots)+0.9) - nrow = int(self.nplots*1./ncol + 0.9) - - return nrow, ncol - - def setup(self, id, nplots, wintitle, showprofile=False, show=True): - - showprofile = False - self.__showprofile = showprofile - self.nplots = nplots - - ncolspan = 5 - colspan = 4 - if showprofile: - ncolspan = 5 - colspan = 4 - self.__nsubplots = 2 - - self.createFigure(id = id, - wintitle = wintitle, - widthplot = self.WIDTH + self.WIDTHPROF, - heightplot = self.HEIGHT + self.HEIGHTPROF, - show=show) - - nrow, ncol = self.getSubplots() - - counter = 0 - for y in range(nrow): - for x in range(ncol): - - if counter >= self.nplots: - break - - self.addAxes(nrow, ncol*ncolspan, y, x*ncolspan, colspan, 1) - - if showprofile: - self.addAxes(nrow, ncol*ncolspan, y, x*ncolspan+colspan, 1, 1) - - counter += 1 - - def run(self, dataOut, id, cutHeight=None, fit=False, wintitle="", channelList=None, showprofile=True, - xmin=None, xmax=None, ymin=None, ymax=None, - save=False, figpath='./', figfile=None, show=True): - - """ - - Input: - dataOut : - id : - wintitle : - channelList : - showProfile : - xmin : None, - xmax : None, - zmin : None, - zmax : None - """ - - if cutHeight==None: - h=270 - heightindex = numpy.abs(cutHeight - dataOut.heightList).argmin() - cutHeight = dataOut.heightList[heightindex] - - factor = dataOut.normFactor - x = dataOut.abscissaList[:-1] - #y = dataOut.getHeiRange() - - z = dataOut.data_pre[:,:,heightindex]/factor - z = numpy.where(numpy.isfinite(z), z, numpy.NAN) - avg = numpy.average(z, axis=1) - listChannels = z.shape[0] - - #Reconstruct Function - if fit==True: - groupArray = dataOut.groupList - listChannels = groupArray.reshape((groupArray.size)) - listChannels.sort() - spcFitLine = numpy.zeros(z.shape) - constants = dataOut.constants - - nGroups = groupArray.shape[0] - nChannels = groupArray.shape[1] - nProfiles = z.shape[1] - - for f in range(nGroups): - groupChann = groupArray[f,:] - p = dataOut.data_param[f,:,heightindex] -# p = numpy.array([ 89.343967,0.14036615,0.17086219,18.89835291,1.58388365,1.55099167]) - fitLineAux = dataOut.library.modelFunction(p, constants)*nProfiles - fitLineAux = fitLineAux.reshape((nChannels,nProfiles)) - spcFitLine[groupChann,:] = fitLineAux -# spcFitLine = spcFitLine/factor - - z = z[listChannels,:] - spcFitLine = spcFitLine[listChannels,:] - spcFitLinedB = 10*numpy.log10(spcFitLine) - - zdB = 10*numpy.log10(z) - #thisDatetime = dataOut.datatime - thisDatetime = datetime.datetime.utcfromtimestamp(dataOut.getTimeRange()[0]) - title = wintitle + " Doppler Spectra: %s" %(thisDatetime.strftime("%d-%b-%Y %H:%M:%S")) - xlabel = "Velocity (m/s)" - ylabel = "Spectrum" - - if not self.isConfig: - - nplots = listChannels.size - - self.setup(id=id, - nplots=nplots, - wintitle=wintitle, - showprofile=showprofile, - show=show) - - if xmin == None: xmin = numpy.nanmin(x) - if xmax == None: xmax = numpy.nanmax(x) - if ymin == None: ymin = numpy.nanmin(zdB) - if ymax == None: ymax = numpy.nanmax(zdB)+2 - - self.isConfig = True - - self.setWinTitle(title) - for i in range(self.nplots): -# title = "Channel %d: %4.2fdB" %(dataOut.channelList[i]+1, noisedB[i]) - title = "Height %4.1f km\nChannel %d:" %(cutHeight, listChannels[i]) - axes = self.axesList[i*self.__nsubplots] - if fit == False: - axes.pline(x, zdB[i,:], - xmin=xmin, xmax=xmax, ymin=ymin, ymax=ymax, - xlabel=xlabel, ylabel=ylabel, title=title - ) - if fit == True: - fitline=spcFitLinedB[i,:] - y=numpy.vstack([zdB[i,:],fitline] ) - legendlabels=['Data','Fitting'] - axes.pmultilineyaxis(x, y, - xmin=xmin, xmax=xmax, ymin=ymin, ymax=ymax, - xlabel=xlabel, ylabel=ylabel, title=title, - legendlabels=legendlabels, marker=None, - linestyle='solid', grid='both') - - self.draw() - - self.save(figpath=figpath, - figfile=figfile, - save=save, - ftp=ftp, - wr_period=wr_period, - thisDatetime=thisDatetime) - - -class EWDriftsPlot(Figure): - - __isConfig = None - __nsubplots = None - - WIDTHPROF = None - HEIGHTPROF = None - PREFIX = 'drift' - - def __init__(self, **kwargs): - Figure.__init__(self, **kwargs) - self.timerange = 2*60*60 - self.isConfig = False - self.__nsubplots = 1 - - self.WIDTH = 800 - self.HEIGHT = 150 - self.WIDTHPROF = 120 - self.HEIGHTPROF = 0 - self.counter_imagwr = 0 - - self.PLOT_CODE = EWDRIFT_CODE - - self.FTP_WEI = None - self.EXP_CODE = None - self.SUB_EXP_CODE = None - self.PLOT_POS = None - self.tmin = None - self.tmax = None - - self.xmin = None - self.xmax = None - - self.figfile = None - - def getSubplots(self): - - ncol = 1 - nrow = self.nplots - - return nrow, ncol - - def setup(self, id, nplots, wintitle, showprofile=True, show=True): - - self.__showprofile = showprofile - self.nplots = nplots - - ncolspan = 1 - colspan = 1 - - self.createFigure(id = id, - wintitle = wintitle, - widthplot = self.WIDTH + self.WIDTHPROF, - heightplot = self.HEIGHT + self.HEIGHTPROF, - show=show) - - nrow, ncol = self.getSubplots() - - counter = 0 - for y in range(nrow): - if counter >= self.nplots: - break - - self.addAxes(nrow, ncol*ncolspan, y, 0, colspan, 1) - counter += 1 - - def run(self, dataOut, id, wintitle="", channelList=None, - xmin=None, xmax=None, ymin=None, ymax=None, zmin=None, zmax=None, - zmaxVertical = None, zminVertical = None, zmaxZonal = None, zminZonal = None, - timerange=None, SNRthresh = -numpy.inf, SNRmin = None, SNRmax = None, SNR_1 = False, - save=False, figpath='./', lastone=0,figfile=None, ftp=False, wr_period=1, show=True, - server=None, folder=None, username=None, password=None, - ftp_wei=0, exp_code=0, sub_exp_code=0, plot_pos=0): - """ - - Input: - dataOut : - id : - wintitle : - channelList : - showProfile : - xmin : None, - xmax : None, - ymin : None, - ymax : None, - zmin : None, - zmax : None - """ - - if timerange is not None: - self.timerange = timerange - - tmin = None - tmax = None - - x = dataOut.getTimeRange1(dataOut.outputInterval) -# y = dataOut.heightList - y = dataOut.heightList - - z = dataOut.data_output - nplots = z.shape[0] #Number of wind dimensions estimated - nplotsw = nplots - - #If there is a SNR function defined - if dataOut.data_SNR is not None: - nplots += 1 - SNR = dataOut.data_SNR - - if SNR_1: - SNR += 1 - - SNRavg = numpy.average(SNR, axis=0) - - SNRdB = 10*numpy.log10(SNR) - SNRavgdB = 10*numpy.log10(SNRavg) - - ind = numpy.where(SNRavg < 10**(SNRthresh/10))[0] - - for i in range(nplotsw): - z[i,ind] = numpy.nan - - - showprofile = False -# thisDatetime = dataOut.datatime - thisDatetime = datetime.datetime.utcfromtimestamp(x[1]) - title = wintitle + " EW Drifts" - xlabel = "" - ylabel = "Height (Km)" - - if not self.isConfig: - - self.setup(id=id, - nplots=nplots, - wintitle=wintitle, - showprofile=showprofile, - show=show) - - self.xmin, self.xmax = self.getTimeLim(x, xmin, xmax, timerange) - - if ymin == None: ymin = numpy.nanmin(y) - if ymax == None: ymax = numpy.nanmax(y) - - if zmaxZonal == None: zmaxZonal = numpy.nanmax(abs(z[0,:])) - if zminZonal == None: zminZonal = -zmaxZonal - if zmaxVertical == None: zmaxVertical = numpy.nanmax(abs(z[1,:])) - if zminVertical == None: zminVertical = -zmaxVertical - - if dataOut.data_SNR is not None: - if SNRmin == None: SNRmin = numpy.nanmin(SNRavgdB) - if SNRmax == None: SNRmax = numpy.nanmax(SNRavgdB) - - self.FTP_WEI = ftp_wei - self.EXP_CODE = exp_code - self.SUB_EXP_CODE = sub_exp_code - self.PLOT_POS = plot_pos - - self.name = thisDatetime.strftime("%Y%m%d_%H%M%S") - self.isConfig = True - - - self.setWinTitle(title) - - if ((self.xmax - x[1]) < (x[1]-x[0])): - x[1] = self.xmax - - strWind = ['Zonal','Vertical'] - strCb = 'Velocity (m/s)' - zmaxVector = [zmaxZonal, zmaxVertical] - zminVector = [zminZonal, zminVertical] - - for i in range(nplotsw): - - title = "%s Drifts: %s" %(strWind[i], thisDatetime.strftime("%Y/%m/%d %H:%M:%S")) - axes = self.axesList[i*self.__nsubplots] - - z1 = z[i,:].reshape((1,-1)) - - axes.pcolorbuffer(x, y, z1, - xmin=self.xmin, xmax=self.xmax, ymin=ymin, ymax=ymax, zmin=zminVector[i], zmax=zmaxVector[i], - xlabel=xlabel, ylabel=ylabel, title=title, rti=True, XAxisAsTime=True, - ticksize=9, cblabel=strCb, cbsize="1%", colormap="RdBu_r") - - if dataOut.data_SNR is not None: - i += 1 - if SNR_1: - title = "Signal Noise Ratio + 1 (SNR+1): %s" %(thisDatetime.strftime("%Y/%m/%d %H:%M:%S")) + azimuths = numpy.radians(self.data.heights) + r, theta = numpy.meshgrid(zeniths, azimuths) + x, y = r*numpy.cos(theta), r*numpy.sin(theta) + self.y = zeniths + + if ax.firsttime: + if self.zlimits is not None: + self.zmin, self.zmax = self.zlimits[n] + ax.plt = ax.pcolormesh( # r, theta, numpy.ma.array(data, mask=numpy.isnan(data)), + x, y, numpy.ma.array(data, mask=numpy.isnan(data)), + vmin=self.zmin, + vmax=self.zmax, + cmap=self.cmaps[n]) else: - title = "Signal Noise Ratio (SNR): %s" %(thisDatetime.strftime("%Y/%m/%d %H:%M:%S")) - axes = self.axesList[i*self.__nsubplots] - SNRavgdB = SNRavgdB.reshape((1,-1)) - - axes.pcolorbuffer(x, y, SNRavgdB, - xmin=self.xmin, xmax=self.xmax, ymin=ymin, ymax=ymax, zmin=SNRmin, zmax=SNRmax, - xlabel=xlabel, ylabel=ylabel, title=title, rti=True, XAxisAsTime=True, - ticksize=9, cblabel='', cbsize="1%", colormap="jet") - - self.draw() - - if x[1] >= self.axesList[0].xmax: - self.counter_imagwr = wr_period - self.isConfig = False - self.figfile = None - - - - -class PhasePlot(Figure): - - __isConfig = None - __nsubplots = None - - PREFIX = 'mphase' - - - def __init__(self, **kwargs): - Figure.__init__(self, **kwargs) - self.timerange = 24*60*60 - self.isConfig = False - self.__nsubplots = 1 - self.counter_imagwr = 0 - self.WIDTH = 600 - self.HEIGHT = 300 - self.WIDTHPROF = 120 - self.HEIGHTPROF = 0 - self.xdata = None - self.ydata = None - - self.PLOT_CODE = MPHASE_CODE - - self.FTP_WEI = None - self.EXP_CODE = None - self.SUB_EXP_CODE = None - self.PLOT_POS = None - - - self.filename_phase = None - - self.figfile = None - - def getSubplots(self): - - ncol = 1 - nrow = 1 - - return nrow, ncol - - def setup(self, id, nplots, wintitle, showprofile=True, show=True): - - self.__showprofile = showprofile - self.nplots = nplots - - ncolspan = 7 - colspan = 6 - self.__nsubplots = 2 - - self.createFigure(id = id, - wintitle = wintitle, - widthplot = self.WIDTH+self.WIDTHPROF, - heightplot = self.HEIGHT+self.HEIGHTPROF, - show=show) - - nrow, ncol = self.getSubplots() - - self.addAxes(nrow, ncol*ncolspan, 0, 0, colspan, 1) - - - def run(self, dataOut, id, wintitle="", pairsList=None, showprofile='True', - xmin=None, xmax=None, ymin=None, ymax=None, - timerange=None, - save=False, figpath='./', figfile=None, show=True, ftp=False, wr_period=1, - server=None, folder=None, username=None, password=None, - ftp_wei=0, exp_code=0, sub_exp_code=0, plot_pos=0): - - - tmin = None - tmax = None - x = dataOut.getTimeRange1(dataOut.outputInterval) - y = dataOut.getHeiRange() - - - #thisDatetime = dataOut.datatime - thisDatetime = datetime.datetime.utcfromtimestamp(dataOut.ltctime) - title = wintitle + " Phase of Beacon Signal" # : %s" %(thisDatetime.strftime("%d-%b-%Y")) - xlabel = "Local Time" - ylabel = "Phase" - - - #phase = numpy.zeros((len(pairsIndexList),len(dataOut.beacon_heiIndexList))) - phase_beacon = dataOut.data_output - update_figfile = False - - if not self.isConfig: - - self.nplots = phase_beacon.size - - self.setup(id=id, - nplots=self.nplots, - wintitle=wintitle, - showprofile=showprofile, - show=show) - - if timerange is not None: - self.timerange = timerange - - self.xmin, self.xmax = self.getTimeLim(x, xmin, xmax, timerange) - - if ymin == None: ymin = numpy.nanmin(phase_beacon) - 10.0 - if ymax == None: ymax = numpy.nanmax(phase_beacon) + 10.0 - - self.FTP_WEI = ftp_wei - self.EXP_CODE = exp_code - self.SUB_EXP_CODE = sub_exp_code - self.PLOT_POS = plot_pos - - self.name = thisDatetime.strftime("%Y%m%d_%H%M%S") - self.isConfig = True - self.figfile = figfile - self.xdata = numpy.array([]) - self.ydata = numpy.array([]) - - #open file beacon phase - path = '%s%03d' %(self.PREFIX, self.id) - beacon_file = os.path.join(path,'%s.txt'%self.name) - self.filename_phase = os.path.join(figpath,beacon_file) - update_figfile = True - - - #store data beacon phase - #self.save_data(self.filename_phase, phase_beacon, thisDatetime) - - self.setWinTitle(title) - - - title = "Phase Offset %s" %(thisDatetime.strftime("%Y/%m/%d %H:%M:%S")) - - legendlabels = ["phase %d"%(chan) for chan in numpy.arange(self.nplots)] - - axes = self.axesList[0] - - self.xdata = numpy.hstack((self.xdata, x[0:1])) - - if len(self.ydata)==0: - self.ydata = phase_beacon.reshape(-1,1) + if self.zlimits is not None: + self.zmin, self.zmax = self.zlimits[n] + ax.collections.remove(ax.collections[0]) + ax.plt = ax.pcolormesh( # r, theta, numpy.ma.array(data, mask=numpy.isnan(data)), + x, y, numpy.ma.array(data, mask=numpy.isnan(data)), + vmin=self.zmin, + vmax=self.zmax, + cmap=self.cmaps[n]) + + if self.mode == 'A': + continue + + # plot district names + f = open('/data/workspace/schain_scripts/distrito.csv') + for line in f: + label, lon, lat = [s.strip() for s in line.split(',') if s] + lat = float(lat) + lon = float(lon) + # ax.plot(lon, lat, '.b', ms=2) + ax.text(lon, lat, label.decode('utf8'), ha='center', + va='bottom', size='8', color='black') + + # plot limites + limites = [] + tmp = [] + for line in open('/data/workspace/schain_scripts/lima.csv'): + if '#' in line: + if tmp: + limites.append(tmp) + tmp = [] + continue + values = line.strip().split(',') + tmp.append((float(values[0]), float(values[1]))) + for points in limites: + ax.add_patch( + Polygon(points, ec='k', fc='none', ls='--', lw=0.5)) + + # plot Cuencas + for cuenca in ('rimac', 'lurin', 'mala', 'chillon', 'chilca', 'chancay-huaral'): + f = open('/data/workspace/schain_scripts/{}.csv'.format(cuenca)) + values = [line.strip().split(',') for line in f] + points = [(float(s[0]), float(s[1])) for s in values] + ax.add_patch(Polygon(points, ec='b', fc='none')) + + # plot grid + for r in (15, 30, 45, 60): + ax.add_artist(plt.Circle((self.lon, self.lat), + km2deg(r), color='0.6', fill=False, lw=0.2)) + ax.text( + self.lon + (km2deg(r))*numpy.cos(60*numpy.pi/180), + self.lat + (km2deg(r))*numpy.sin(60*numpy.pi/180), + '{}km'.format(r), + ha='center', va='bottom', size='8', color='0.6', weight='heavy') + + if self.mode == 'E': + title = 'El={}$^\circ$'.format(self.data.meta['elevation']) + label = 'E{:02d}'.format(int(self.data.meta['elevation'])) else: - self.ydata = numpy.hstack((self.ydata, phase_beacon.reshape(-1,1))) - - - axes.pmultilineyaxis(x=self.xdata, y=self.ydata, - xmin=self.xmin, xmax=self.xmax, ymin=ymin, ymax=ymax, - xlabel=xlabel, ylabel=ylabel, title=title, legendlabels=legendlabels, marker='x', markersize=8, linestyle="solid", - XAxisAsTime=True, grid='both' - ) - - self.draw() - - self.save(figpath=figpath, - figfile=figfile, - save=save, - ftp=ftp, - wr_period=wr_period, - thisDatetime=thisDatetime, - update_figfile=update_figfile) - - if dataOut.ltctime + dataOut.outputInterval >= self.xmax: - self.counter_imagwr = wr_period - self.isConfig = False - update_figfile = True - - - -class NSMeteorDetection1Plot(Figure): - - isConfig = None - __nsubplots = None - - WIDTHPROF = None - HEIGHTPROF = None - PREFIX = 'nsm' - - zminList = None - zmaxList = None - cmapList = None - titleList = None - nPairs = None - nChannels = None - nParam = None - - def __init__(self, **kwargs): - Figure.__init__(self, **kwargs) - self.isConfig = False - self.__nsubplots = 1 - - self.WIDTH = 750 - self.HEIGHT = 250 - self.WIDTHPROF = 120 - self.HEIGHTPROF = 0 - self.counter_imagwr = 0 - - self.PLOT_CODE = SPEC_CODE - - self.FTP_WEI = None - self.EXP_CODE = None - self.SUB_EXP_CODE = None - self.PLOT_POS = None - - self.__xfilter_ena = False - self.__yfilter_ena = False - - def getSubplots(self): - - ncol = 3 - nrow = int(numpy.ceil(self.nplots/3.0)) - - return nrow, ncol - - def setup(self, id, nplots, wintitle, show=True): - - self.nplots = nplots - - ncolspan = 1 - colspan = 1 - - self.createFigure(id = id, - wintitle = wintitle, - widthplot = self.WIDTH + self.WIDTHPROF, - heightplot = self.HEIGHT + self.HEIGHTPROF, - show=show) - - nrow, ncol = self.getSubplots() - - counter = 0 - for y in range(nrow): - for x in range(ncol): - - if counter >= self.nplots: - break - - self.addAxes(nrow, ncol*ncolspan, y, x*ncolspan, colspan, 1) - - counter += 1 - - def run(self, dataOut, id, wintitle="", channelList=None, showprofile=True, - xmin=None, xmax=None, ymin=None, ymax=None, SNRmin=None, SNRmax=None, - vmin=None, vmax=None, wmin=None, wmax=None, mode = 'SA', - save=False, figpath='./', figfile=None, show=True, ftp=False, wr_period=1, - server=None, folder=None, username=None, password=None, - ftp_wei=0, exp_code=0, sub_exp_code=0, plot_pos=0, realtime=False, - xaxis="frequency"): - - """ - - Input: - dataOut : - id : - wintitle : - channelList : - showProfile : - xmin : None, - xmax : None, - ymin : None, - ymax : None, - zmin : None, - zmax : None - """ - #SEPARAR EN DOS PLOTS - nParam = dataOut.data_param.shape[1] - 3 - - utctime = dataOut.data_param[0,0] - tmet = dataOut.data_param[:,1].astype(int) - hmet = dataOut.data_param[:,2].astype(int) - - x = dataOut.abscissaList - y = dataOut.heightList - - z = numpy.zeros((nParam, y.size, x.size - 1)) - z[:,:] = numpy.nan - z[:,hmet,tmet] = dataOut.data_param[:,3:].T - z[0,:,:] = 10*numpy.log10(z[0,:,:]) - - xlabel = "Time (s)" - ylabel = "Range (km)" - - thisDatetime = datetime.datetime.utcfromtimestamp(dataOut.ltctime) - - if not self.isConfig: - - nplots = nParam - - self.setup(id=id, - nplots=nplots, - wintitle=wintitle, - show=show) - - if xmin is None: xmin = numpy.nanmin(x) - if xmax is None: xmax = numpy.nanmax(x) - if ymin is None: ymin = numpy.nanmin(y) - if ymax is None: ymax = numpy.nanmax(y) - if SNRmin is None: SNRmin = numpy.nanmin(z[0,:]) - if SNRmax is None: SNRmax = numpy.nanmax(z[0,:]) - if vmax is None: vmax = numpy.nanmax(numpy.abs(z[1,:])) - if vmin is None: vmin = -vmax - if wmin is None: wmin = 0 - if wmax is None: wmax = 50 - - pairsList = dataOut.groupList - self.nPairs = len(dataOut.groupList) - - zminList = [SNRmin, vmin, cmin] + [pmin]*self.nPairs - zmaxList = [SNRmax, vmax, cmax] + [pmax]*self.nPairs - titleList = ["SNR","Radial Velocity","Coherence"] - cmapList = ["jet","RdBu_r","jet"] + title = 'Az={}$^\circ$'.format(self.data.meta['azimuth']) + label = 'A{:02d}'.format(int(self.data.meta['azimuth'])) - for i in range(self.nPairs): - strAux1 = "Phase Difference "+ str(pairsList[i][0]) + str(pairsList[i][1]) - titleList = titleList + [strAux1] - cmapList = cmapList + ["RdBu_r"] - - self.zminList = zminList - self.zmaxList = zmaxList - self.cmapList = cmapList - self.titleList = titleList - - self.FTP_WEI = ftp_wei - self.EXP_CODE = exp_code - self.SUB_EXP_CODE = sub_exp_code - self.PLOT_POS = plot_pos + self.save_labels = ['{}-{}'.format(lbl, label) for lbl in self.labels] + self.titles = ['{} {}'.format( + self.data.parameters[x], title) for x in self.channels] - self.isConfig = True - - str_datetime = '%s %s'%(thisDatetime.strftime("%Y/%m/%d"),thisDatetime.strftime("%H:%M:%S")) - - for i in range(nParam): - title = self.titleList[i] + ": " +str_datetime - axes = self.axesList[i] - axes.pcolor(x, y, z[i,:].T, - xmin=xmin, xmax=xmax, ymin=ymin, ymax=ymax, zmin=self.zminList[i], zmax=self.zmaxList[i], - xlabel=xlabel, ylabel=ylabel, title=title, colormap=self.cmapList[i],ticksize=9, cblabel='') - self.draw() - - if figfile == None: - str_datetime = thisDatetime.strftime("%Y%m%d_%H%M%S") - name = str_datetime - if ((dataOut.azimuth!=None) and (dataOut.zenith!=None)): - name = name + '_az' + '_%2.2f'%(dataOut.azimuth) + '_zn' + '_%2.2f'%(dataOut.zenith) - figfile = self.getFilename(name) - - self.save(figpath=figpath, - figfile=figfile, - save=save, - ftp=ftp, - wr_period=wr_period, - thisDatetime=thisDatetime) - - -class NSMeteorDetection2Plot(Figure): - - isConfig = None - __nsubplots = None - - WIDTHPROF = None - HEIGHTPROF = None - PREFIX = 'nsm' - - zminList = None - zmaxList = None - cmapList = None - titleList = None - nPairs = None - nChannels = None - nParam = None - - def __init__(self, **kwargs): - Figure.__init__(self, **kwargs) - self.isConfig = False - self.__nsubplots = 1 - - self.WIDTH = 750 - self.HEIGHT = 250 - self.WIDTHPROF = 120 - self.HEIGHTPROF = 0 - self.counter_imagwr = 0 - - self.PLOT_CODE = SPEC_CODE - - self.FTP_WEI = None - self.EXP_CODE = None - self.SUB_EXP_CODE = None - self.PLOT_POS = None - - self.__xfilter_ena = False - self.__yfilter_ena = False - - def getSubplots(self): - - ncol = 3 - nrow = int(numpy.ceil(self.nplots/3.0)) - - return nrow, ncol - - def setup(self, id, nplots, wintitle, show=True): - - self.nplots = nplots - - ncolspan = 1 - colspan = 1 - - self.createFigure(id = id, - wintitle = wintitle, - widthplot = self.WIDTH + self.WIDTHPROF, - heightplot = self.HEIGHT + self.HEIGHTPROF, - show=show) - - nrow, ncol = self.getSubplots() - - counter = 0 - for y in range(nrow): - for x in range(ncol): - - if counter >= self.nplots: - break - - self.addAxes(nrow, ncol*ncolspan, y, x*ncolspan, colspan, 1) - - counter += 1 - - def run(self, dataOut, id, wintitle="", channelList=None, showprofile=True, - xmin=None, xmax=None, ymin=None, ymax=None, SNRmin=None, SNRmax=None, - vmin=None, vmax=None, wmin=None, wmax=None, mode = 'SA', - save=False, figpath='./', figfile=None, show=True, ftp=False, wr_period=1, - server=None, folder=None, username=None, password=None, - ftp_wei=0, exp_code=0, sub_exp_code=0, plot_pos=0, realtime=False, - xaxis="frequency"): - - """ - - Input: - dataOut : - id : - wintitle : - channelList : - showProfile : - xmin : None, - xmax : None, - ymin : None, - ymax : None, - zmin : None, - zmax : None - """ - #Rebuild matrix - utctime = dataOut.data_param[0,0] - cmet = dataOut.data_param[:,1].astype(int) - tmet = dataOut.data_param[:,2].astype(int) - hmet = dataOut.data_param[:,3].astype(int) - - nParam = 3 - nChan = len(dataOut.groupList) - x = dataOut.abscissaList - y = dataOut.heightList - - z = numpy.full((nChan, nParam, y.size, x.size - 1),numpy.nan) - z[cmet,:,hmet,tmet] = dataOut.data_param[:,4:] - z[:,0,:,:] = 10*numpy.log10(z[:,0,:,:]) #logarithmic scale - z = numpy.reshape(z, (nChan*nParam, y.size, x.size-1)) - - xlabel = "Time (s)" - ylabel = "Range (km)" - - thisDatetime = datetime.datetime.utcfromtimestamp(dataOut.ltctime) - - if not self.isConfig: - - nplots = nParam*nChan - - self.setup(id=id, - nplots=nplots, - wintitle=wintitle, - show=show) - - if xmin is None: xmin = numpy.nanmin(x) - if xmax is None: xmax = numpy.nanmax(x) - if ymin is None: ymin = numpy.nanmin(y) - if ymax is None: ymax = numpy.nanmax(y) - if SNRmin is None: SNRmin = numpy.nanmin(z[0,:]) - if SNRmax is None: SNRmax = numpy.nanmax(z[0,:]) - if vmax is None: vmax = numpy.nanmax(numpy.abs(z[1,:])) - if vmin is None: vmin = -vmax - if wmin is None: wmin = 0 - if wmax is None: wmax = 50 - - self.nChannels = nChan - - zminList = [] - zmaxList = [] - titleList = [] - cmapList = [] - for i in range(self.nChannels): - strAux1 = "SNR Channel "+ str(i) - strAux2 = "Radial Velocity Channel "+ str(i) - strAux3 = "Spectral Width Channel "+ str(i) - - titleList = titleList + [strAux1,strAux2,strAux3] - cmapList = cmapList + ["jet","RdBu_r","jet"] - zminList = zminList + [SNRmin,vmin,wmin] - zmaxList = zmaxList + [SNRmax,vmax,wmax] - - self.zminList = zminList - self.zmaxList = zmaxList - self.cmapList = cmapList - self.titleList = titleList - - self.FTP_WEI = ftp_wei - self.EXP_CODE = exp_code - self.SUB_EXP_CODE = sub_exp_code - self.PLOT_POS = plot_pos - - self.isConfig = True - - str_datetime = '%s %s'%(thisDatetime.strftime("%Y/%m/%d"),thisDatetime.strftime("%H:%M:%S")) - - for i in range(self.nplots): - title = self.titleList[i] + ": " +str_datetime - axes = self.axesList[i] - axes.pcolor(x, y, z[i,:].T, - xmin=xmin, xmax=xmax, ymin=ymin, ymax=ymax, zmin=self.zminList[i], zmax=self.zmaxList[i], - xlabel=xlabel, ylabel=ylabel, title=title, colormap=self.cmapList[i],ticksize=9, cblabel='') - self.draw() - - if figfile == None: - str_datetime = thisDatetime.strftime("%Y%m%d_%H%M%S") - name = str_datetime - if ((dataOut.azimuth!=None) and (dataOut.zenith!=None)): - name = name + '_az' + '_%2.2f'%(dataOut.azimuth) + '_zn' + '_%2.2f'%(dataOut.zenith) - figfile = self.getFilename(name) - - self.save(figpath=figpath, - figfile=figfile, - save=save, - ftp=ftp, - wr_period=wr_period, - thisDatetime=thisDatetime) diff --git a/schainpy/model/graphics/jroplot_spectra.py b/schainpy/model/graphics/jroplot_spectra.py index eafac9f..3ef5c6e 100644 --- a/schainpy/model/graphics/jroplot_spectra.py +++ b/schainpy/model/graphics/jroplot_spectra.py @@ -1,1326 +1,431 @@ ''' Created on Jul 9, 2014 +Modified on May 10, 2020 -@author: roj-idl71 +@author: Juan C. Espinoza ''' -import os -import datetime -import numpy - -from figure import Figure, isRealtime, isTimeInHourRange -from plotting_codes import * - - -class SpectraPlot(Figure): - - isConfig = None - __nsubplots = None - - WIDTHPROF = None - HEIGHTPROF = None - PREFIX = 'spc' - - def __init__(self, **kwargs): - Figure.__init__(self, **kwargs) - self.isConfig = False - self.__nsubplots = 1 - - self.WIDTH = 250 - self.HEIGHT = 250 - self.WIDTHPROF = 120 - self.HEIGHTPROF = 0 - self.counter_imagwr = 0 - - self.PLOT_CODE = SPEC_CODE - - self.FTP_WEI = None - self.EXP_CODE = None - self.SUB_EXP_CODE = None - self.PLOT_POS = None - - self.__xfilter_ena = False - self.__yfilter_ena = False - - def getSubplots(self): - - ncol = int(numpy.sqrt(self.nplots)+0.9) - nrow = int(self.nplots*1./ncol + 0.9) - - return nrow, ncol - - def setup(self, id, nplots, wintitle, showprofile=True, show=True): - - self.__showprofile = showprofile - self.nplots = nplots - - ncolspan = 1 - colspan = 1 - if showprofile: - ncolspan = 3 - colspan = 2 - self.__nsubplots = 2 - - self.createFigure(id = id, - wintitle = wintitle, - widthplot = self.WIDTH + self.WIDTHPROF, - heightplot = self.HEIGHT + self.HEIGHTPROF, - show=show) - - nrow, ncol = self.getSubplots() - - counter = 0 - for y in range(nrow): - for x in range(ncol): - - if counter >= self.nplots: - break - - self.addAxes(nrow, ncol*ncolspan, y, x*ncolspan, colspan, 1) - - if showprofile: - self.addAxes(nrow, ncol*ncolspan, y, x*ncolspan+colspan, 1, 1) - - counter += 1 - - def run(self, dataOut, id, wintitle="", channelList=None, showprofile=True, - xmin=None, xmax=None, ymin=None, ymax=None, zmin=None, zmax=None, - save=False, figpath='./', figfile=None, show=True, ftp=False, wr_period=1, - server=None, folder=None, username=None, password=None, - ftp_wei=0, exp_code=0, sub_exp_code=0, plot_pos=0, realtime=False, - xaxis="frequency", colormap='jet', normFactor=None): - - """ - - Input: - dataOut : - id : - wintitle : - channelList : - showProfile : - xmin : None, - xmax : None, - ymin : None, - ymax : None, - zmin : None, - zmax : None - """ - if realtime: - if not(isRealtime(utcdatatime = dataOut.utctime)): - print 'Skipping this plot function' - return - - if channelList == None: - channelIndexList = dataOut.channelIndexList - else: - channelIndexList = [] - for channel in channelList: - if channel not in dataOut.channelList: - raise ValueError, "Channel %d is not in dataOut.channelList" %channel - channelIndexList.append(dataOut.channelList.index(channel)) - - if normFactor is None: - factor = dataOut.normFactor - else: - factor = normFactor - if xaxis == "frequency": - x = dataOut.getFreqRange(1)/1000. - xlabel = "Frequency (kHz)" - - elif xaxis == "time": - x = dataOut.getAcfRange(1) - xlabel = "Time (ms)" - - else: - x = dataOut.getVelRange(1) - xlabel = "Velocity (m/s)" - - ylabel = "Range (Km)" - - y = dataOut.getHeiRange() - - z = dataOut.data_spc/factor - z = numpy.where(numpy.isfinite(z), z, numpy.NAN) - zdB = 10*numpy.log10(z) - - avg = numpy.average(z, axis=1) - avgdB = 10*numpy.log10(avg) - - noise = dataOut.getNoise()/factor - noisedB = 10*numpy.log10(noise) - - thisDatetime = datetime.datetime.utcfromtimestamp(dataOut.getTimeRange()[0]) - title = wintitle + " Spectra" - if ((dataOut.azimuth!=None) and (dataOut.zenith!=None)): - title = title + '_' + 'azimuth,zenith=%2.2f,%2.2f'%(dataOut.azimuth, dataOut.zenith) - - if not self.isConfig: - - nplots = len(channelIndexList) - - self.setup(id=id, - nplots=nplots, - wintitle=wintitle, - showprofile=showprofile, - show=show) - - if xmin == None: xmin = numpy.nanmin(x) - if xmax == None: xmax = numpy.nanmax(x) - if ymin == None: ymin = numpy.nanmin(y) - if ymax == None: ymax = numpy.nanmax(y) - if zmin == None: zmin = numpy.floor(numpy.nanmin(noisedB)) - 3 - if zmax == None: zmax = numpy.ceil(numpy.nanmax(avgdB)) + 3 - - self.FTP_WEI = ftp_wei - self.EXP_CODE = exp_code - self.SUB_EXP_CODE = sub_exp_code - self.PLOT_POS = plot_pos - - self.isConfig = True - - self.setWinTitle(title) - - for i in range(self.nplots): - index = channelIndexList[i] - str_datetime = '%s %s'%(thisDatetime.strftime("%Y/%m/%d"),thisDatetime.strftime("%H:%M:%S")) - title = "Channel %d: %4.2fdB: %s" %(dataOut.channelList[index], noisedB[index], str_datetime) - if len(dataOut.beam.codeList) != 0: - title = "Ch%d:%4.2fdB,%2.2f,%2.2f:%s" %(dataOut.channelList[index], noisedB[index], dataOut.beam.azimuthList[index], dataOut.beam.zenithList[index], str_datetime) - - axes = self.axesList[i*self.__nsubplots] - axes.pcolor(x, y, zdB[index,:,:], - xmin=xmin, xmax=xmax, ymin=ymin, ymax=ymax, zmin=zmin, zmax=zmax, - xlabel=xlabel, ylabel=ylabel, title=title, colormap=colormap, - ticksize=9, cblabel='') - - if self.__showprofile: - axes = self.axesList[i*self.__nsubplots +1] - axes.pline(avgdB[index,:], y, - xmin=zmin, xmax=zmax, ymin=ymin, ymax=ymax, - xlabel='dB', ylabel='', title='', - ytick_visible=False, - grid='x') - - noiseline = numpy.repeat(noisedB[index], len(y)) - axes.addpline(noiseline, y, idline=1, color="black", linestyle="dashed", lw=2) - - self.draw() - - if figfile == None: - str_datetime = thisDatetime.strftime("%Y%m%d_%H%M%S") - name = str_datetime - if ((dataOut.azimuth!=None) and (dataOut.zenith!=None)): - name = name + '_az' + '_%2.2f'%(dataOut.azimuth) + '_zn' + '_%2.2f'%(dataOut.zenith) - figfile = self.getFilename(name) - - self.save(figpath=figpath, - figfile=figfile, - save=save, - ftp=ftp, - wr_period=wr_period, - thisDatetime=thisDatetime) - -class CrossSpectraPlot(Figure): - - isConfig = None - __nsubplots = None - - WIDTH = None - HEIGHT = None - WIDTHPROF = None - HEIGHTPROF = None - PREFIX = 'cspc' - - def __init__(self, **kwargs): - Figure.__init__(self, **kwargs) - self.isConfig = False - self.__nsubplots = 4 - self.counter_imagwr = 0 - self.WIDTH = 250 - self.HEIGHT = 250 - self.WIDTHPROF = 0 - self.HEIGHTPROF = 0 - - self.PLOT_CODE = CROSS_CODE - self.FTP_WEI = None - self.EXP_CODE = None - self.SUB_EXP_CODE = None - self.PLOT_POS = None - - def getSubplots(self): - - ncol = 4 - nrow = self.nplots - - return nrow, ncol - - def setup(self, id, nplots, wintitle, showprofile=True, show=True): - - self.__showprofile = showprofile - self.nplots = nplots - - ncolspan = 1 - colspan = 1 - - self.createFigure(id = id, - wintitle = wintitle, - widthplot = self.WIDTH + self.WIDTHPROF, - heightplot = self.HEIGHT + self.HEIGHTPROF, - show=True) - - nrow, ncol = self.getSubplots() - - counter = 0 - for y in range(nrow): - for x in range(ncol): - self.addAxes(nrow, ncol*ncolspan, y, x*ncolspan, colspan, 1) - - counter += 1 - - def run(self, dataOut, id, wintitle="", pairsList=None, - xmin=None, xmax=None, ymin=None, ymax=None, zmin=None, zmax=None, - coh_min=None, coh_max=None, phase_min=None, phase_max=None, - save=False, figpath='./', figfile=None, ftp=False, wr_period=1, - power_cmap='jet', coherence_cmap='jet', phase_cmap='RdBu_r', show=True, - server=None, folder=None, username=None, password=None, - ftp_wei=0, exp_code=0, sub_exp_code=0, plot_pos=0, normFactor=None, - xaxis='frequency'): - - """ - - Input: - dataOut : - id : - wintitle : - channelList : - showProfile : - xmin : None, - xmax : None, - ymin : None, - ymax : None, - zmin : None, - zmax : None - """ - - if pairsList == None: - pairsIndexList = dataOut.pairsIndexList - else: - pairsIndexList = [] - for pair in pairsList: - if pair not in dataOut.pairsList: - raise ValueError, "Pair %s is not in dataOut.pairsList" %str(pair) - pairsIndexList.append(dataOut.pairsList.index(pair)) - - if not pairsIndexList: - return - - if len(pairsIndexList) > 4: - pairsIndexList = pairsIndexList[0:4] - - if normFactor is None: - factor = dataOut.normFactor - else: - factor = normFactor - x = dataOut.getVelRange(1) - y = dataOut.getHeiRange() - z = dataOut.data_spc[:,:,:]/factor - z = numpy.where(numpy.isfinite(z), z, numpy.NAN) - - noise = dataOut.noise/factor - - zdB = 10*numpy.log10(z) - noisedB = 10*numpy.log10(noise) - - if coh_min == None: - coh_min = 0.0 - if coh_max == None: - coh_max = 1.0 - - if phase_min == None: - phase_min = -180 - if phase_max == None: - phase_max = 180 - - #thisDatetime = dataOut.datatime - thisDatetime = datetime.datetime.utcfromtimestamp(dataOut.getTimeRange()[0]) - title = wintitle + " Cross-Spectra: %s" %(thisDatetime.strftime("%d-%b-%Y %H:%M:%S")) -# xlabel = "Velocity (m/s)" - ylabel = "Range (Km)" - - if xaxis == "frequency": - x = dataOut.getFreqRange(1)/1000. - xlabel = "Frequency (kHz)" - - elif xaxis == "time": - x = dataOut.getAcfRange(1) - xlabel = "Time (ms)" - - else: - x = dataOut.getVelRange(1) - xlabel = "Velocity (m/s)" - - if not self.isConfig: - - nplots = len(pairsIndexList) - - self.setup(id=id, - nplots=nplots, - wintitle=wintitle, - showprofile=False, - show=show) - - avg = numpy.abs(numpy.average(z, axis=1)) - avgdB = 10*numpy.log10(avg) - - if xmin == None: xmin = numpy.nanmin(x) - if xmax == None: xmax = numpy.nanmax(x) - if ymin == None: ymin = numpy.nanmin(y) - if ymax == None: ymax = numpy.nanmax(y) - if zmin == None: zmin = numpy.floor(numpy.nanmin(noisedB)) - 3 - if zmax == None: zmax = numpy.ceil(numpy.nanmax(avgdB)) + 3 - - self.FTP_WEI = ftp_wei - self.EXP_CODE = exp_code - self.SUB_EXP_CODE = sub_exp_code - self.PLOT_POS = plot_pos - - self.isConfig = True - - self.setWinTitle(title) - - for i in range(self.nplots): - pair = dataOut.pairsList[pairsIndexList[i]] - - chan_index0 = dataOut.channelList.index(pair[0]) - chan_index1 = dataOut.channelList.index(pair[1]) - - str_datetime = '%s %s'%(thisDatetime.strftime("%Y/%m/%d"),thisDatetime.strftime("%H:%M:%S")) - title = "Ch%d: %4.2fdB: %s" %(pair[0], noisedB[chan_index0], str_datetime) - zdB = 10.*numpy.log10(dataOut.data_spc[chan_index0,:,:]/factor) - axes0 = self.axesList[i*self.__nsubplots] - axes0.pcolor(x, y, zdB, - xmin=xmin, xmax=xmax, ymin=ymin, ymax=ymax, zmin=zmin, zmax=zmax, - xlabel=xlabel, ylabel=ylabel, title=title, - ticksize=9, colormap=power_cmap, cblabel='') - - title = "Ch%d: %4.2fdB: %s" %(pair[1], noisedB[chan_index1], str_datetime) - zdB = 10.*numpy.log10(dataOut.data_spc[chan_index1,:,:]/factor) - axes0 = self.axesList[i*self.__nsubplots+1] - axes0.pcolor(x, y, zdB, - xmin=xmin, xmax=xmax, ymin=ymin, ymax=ymax, zmin=zmin, zmax=zmax, - xlabel=xlabel, ylabel=ylabel, title=title, - ticksize=9, colormap=power_cmap, cblabel='') - - coherenceComplex = dataOut.data_cspc[pairsIndexList[i],:,:]/numpy.sqrt(dataOut.data_spc[chan_index0,:,:]*dataOut.data_spc[chan_index1,:,:]) - coherence = numpy.abs(coherenceComplex) -# phase = numpy.arctan(-1*coherenceComplex.imag/coherenceComplex.real)*180/numpy.pi - phase = numpy.arctan2(coherenceComplex.imag, coherenceComplex.real)*180/numpy.pi - - title = "Coherence Ch%d * Ch%d" %(pair[0], pair[1]) - axes0 = self.axesList[i*self.__nsubplots+2] - axes0.pcolor(x, y, coherence, - xmin=xmin, xmax=xmax, ymin=ymin, ymax=ymax, zmin=coh_min, zmax=coh_max, - xlabel=xlabel, ylabel=ylabel, title=title, - ticksize=9, colormap=coherence_cmap, cblabel='') - - title = "Phase Ch%d * Ch%d" %(pair[0], pair[1]) - axes0 = self.axesList[i*self.__nsubplots+3] - axes0.pcolor(x, y, phase, - xmin=xmin, xmax=xmax, ymin=ymin, ymax=ymax, zmin=phase_min, zmax=phase_max, - xlabel=xlabel, ylabel=ylabel, title=title, - ticksize=9, colormap=phase_cmap, cblabel='') - - - - self.draw() - - self.save(figpath=figpath, - figfile=figfile, - save=save, - ftp=ftp, - wr_period=wr_period, - thisDatetime=thisDatetime) - - -class RTIPlot(Figure): - - __isConfig = None - __nsubplots = None - - WIDTHPROF = None - HEIGHTPROF = None - PREFIX = 'rti' - - def __init__(self, **kwargs): - - Figure.__init__(self, **kwargs) - self.timerange = None - self.isConfig = False - self.__nsubplots = 1 - - self.WIDTH = 800 - self.HEIGHT = 180 - self.WIDTHPROF = 120 - self.HEIGHTPROF = 0 - self.counter_imagwr = 0 - - self.PLOT_CODE = RTI_CODE - - self.FTP_WEI = None - self.EXP_CODE = None - self.SUB_EXP_CODE = None - self.PLOT_POS = None - self.tmin = None - self.tmax = None - - self.xmin = None - self.xmax = None - - self.figfile = None - - def getSubplots(self): - - ncol = 1 - nrow = self.nplots - - return nrow, ncol - - def setup(self, id, nplots, wintitle, showprofile=True, show=True): - - self.__showprofile = showprofile - self.nplots = nplots - - ncolspan = 1 - colspan = 1 - if showprofile: - ncolspan = 7 - colspan = 6 - self.__nsubplots = 2 - - self.createFigure(id = id, - wintitle = wintitle, - widthplot = self.WIDTH + self.WIDTHPROF, - heightplot = self.HEIGHT + self.HEIGHTPROF, - show=show) - - nrow, ncol = self.getSubplots() - - counter = 0 - for y in range(nrow): - for x in range(ncol): - - if counter >= self.nplots: - break - - self.addAxes(nrow, ncol*ncolspan, y, x*ncolspan, colspan, 1) - - if showprofile: - self.addAxes(nrow, ncol*ncolspan, y, x*ncolspan+colspan, 1, 1) - - counter += 1 - - def run(self, dataOut, id, wintitle="", channelList=None, showprofile='True', - xmin=None, xmax=None, ymin=None, ymax=None, zmin=None, zmax=None, - timerange=None, colormap='jet', - save=False, figpath='./', lastone=0,figfile=None, ftp=False, wr_period=1, show=True, - server=None, folder=None, username=None, password=None, - ftp_wei=0, exp_code=0, sub_exp_code=0, plot_pos=0, normFactor=None, HEIGHT=None): - - """ - - Input: - dataOut : - id : - wintitle : - channelList : - showProfile : - xmin : None, - xmax : None, - ymin : None, - ymax : None, - zmin : None, - zmax : None - """ - - #colormap = kwargs.get('colormap', 'jet') - if HEIGHT is not None: - self.HEIGHT = HEIGHT - - if not isTimeInHourRange(dataOut.datatime, xmin, xmax): - return - - if channelList == None: - channelIndexList = dataOut.channelIndexList - else: - channelIndexList = [] - for channel in channelList: - if channel not in dataOut.channelList: - raise ValueError, "Channel %d is not in dataOut.channelList" - channelIndexList.append(dataOut.channelList.index(channel)) - - if normFactor is None: - factor = dataOut.normFactor - else: - factor = normFactor - -# factor = dataOut.normFactor - x = dataOut.getTimeRange() - y = dataOut.getHeiRange() - - z = dataOut.data_spc/factor - z = numpy.where(numpy.isfinite(z), z, numpy.NAN) - avg = numpy.average(z, axis=1) - avgdB = 10.*numpy.log10(avg) - # avgdB = dataOut.getPower() - - - thisDatetime = dataOut.datatime -# thisDatetime = datetime.datetime.utcfromtimestamp(dataOut.getTimeRange()[0]) - title = wintitle + " RTI" #: %s" %(thisDatetime.strftime("%d-%b-%Y")) - xlabel = "" - ylabel = "Range (Km)" - - update_figfile = False - - if dataOut.ltctime >= self.xmax: - self.counter_imagwr = wr_period - self.isConfig = False - update_figfile = True - - if not self.isConfig: - - nplots = len(channelIndexList) - - self.setup(id=id, - nplots=nplots, - wintitle=wintitle, - showprofile=showprofile, - show=show) - - if timerange != None: - self.timerange = timerange - - self.xmin, self.xmax = self.getTimeLim(x, xmin, xmax, timerange) - - noise = dataOut.noise/factor - noisedB = 10*numpy.log10(noise) - - if ymin == None: ymin = numpy.nanmin(y) - if ymax == None: ymax = numpy.nanmax(y) - if zmin == None: zmin = numpy.floor(numpy.nanmin(noisedB)) - 3 - if zmax == None: zmax = numpy.ceil(numpy.nanmax(avgdB)) + 3 - - self.FTP_WEI = ftp_wei - self.EXP_CODE = exp_code - self.SUB_EXP_CODE = sub_exp_code - self.PLOT_POS = plot_pos - - self.name = thisDatetime.strftime("%Y%m%d_%H%M%S") - self.isConfig = True - self.figfile = figfile - update_figfile = True - - self.setWinTitle(title) - - for i in range(self.nplots): - index = channelIndexList[i] - title = "Channel %d: %s" %(dataOut.channelList[index], thisDatetime.strftime("%Y/%m/%d %H:%M:%S")) - if ((dataOut.azimuth!=None) and (dataOut.zenith!=None)): - title = title + '_' + 'azimuth,zenith=%2.2f,%2.2f'%(dataOut.azimuth, dataOut.zenith) - axes = self.axesList[i*self.__nsubplots] - zdB = avgdB[index].reshape((1,-1)) - axes.pcolorbuffer(x, y, zdB, - xmin=self.xmin, xmax=self.xmax, ymin=ymin, ymax=ymax, zmin=zmin, zmax=zmax, - xlabel=xlabel, ylabel=ylabel, title=title, rti=True, XAxisAsTime=True, - ticksize=9, cblabel='', cbsize="1%", colormap=colormap) - - if self.__showprofile: - axes = self.axesList[i*self.__nsubplots +1] - axes.pline(avgdB[index], y, - xmin=zmin, xmax=zmax, ymin=ymin, ymax=ymax, - xlabel='dB', ylabel='', title='', - ytick_visible=False, - grid='x') - - self.draw() - - self.save(figpath=figpath, - figfile=figfile, - save=save, - ftp=ftp, - wr_period=wr_period, - thisDatetime=thisDatetime, - update_figfile=update_figfile) - -class CoherenceMap(Figure): - isConfig = None - __nsubplots = None - - WIDTHPROF = None - HEIGHTPROF = None - PREFIX = 'cmap' - - def __init__(self, **kwargs): - Figure.__init__(self, **kwargs) - self.timerange = 2*60*60 - self.isConfig = False - self.__nsubplots = 1 - - self.WIDTH = 800 - self.HEIGHT = 180 - self.WIDTHPROF = 120 - self.HEIGHTPROF = 0 - self.counter_imagwr = 0 - - self.PLOT_CODE = COH_CODE - - self.FTP_WEI = None - self.EXP_CODE = None - self.SUB_EXP_CODE = None - self.PLOT_POS = None - self.counter_imagwr = 0 - - self.xmin = None - self.xmax = None - - def getSubplots(self): - ncol = 1 - nrow = self.nplots*2 - - return nrow, ncol - - def setup(self, id, nplots, wintitle, showprofile=True, show=True): - self.__showprofile = showprofile - self.nplots = nplots - - ncolspan = 1 - colspan = 1 - if showprofile: - ncolspan = 7 - colspan = 6 - self.__nsubplots = 2 - - self.createFigure(id = id, - wintitle = wintitle, - widthplot = self.WIDTH + self.WIDTHPROF, - heightplot = self.HEIGHT + self.HEIGHTPROF, - show=True) - - nrow, ncol = self.getSubplots() - - for y in range(nrow): - for x in range(ncol): - - self.addAxes(nrow, ncol*ncolspan, y, x*ncolspan, colspan, 1) - - if showprofile: - self.addAxes(nrow, ncol*ncolspan, y, x*ncolspan+colspan, 1, 1) - - def run(self, dataOut, id, wintitle="", pairsList=None, showprofile='True', - xmin=None, xmax=None, ymin=None, ymax=None, zmin=None, zmax=None, - timerange=None, phase_min=None, phase_max=None, - save=False, figpath='./', figfile=None, ftp=False, wr_period=1, - coherence_cmap='jet', phase_cmap='RdBu_r', show=True, - server=None, folder=None, username=None, password=None, - ftp_wei=0, exp_code=0, sub_exp_code=0, plot_pos=0): - - if not isTimeInHourRange(dataOut.datatime, xmin, xmax): - return - - if pairsList == None: - pairsIndexList = dataOut.pairsIndexList - else: - pairsIndexList = [] - for pair in pairsList: - if pair not in dataOut.pairsList: - raise ValueError, "Pair %s is not in dataOut.pairsList" %(pair) - pairsIndexList.append(dataOut.pairsList.index(pair)) - - if pairsIndexList == []: - return - - if len(pairsIndexList) > 4: - pairsIndexList = pairsIndexList[0:4] - - if phase_min == None: - phase_min = -180 - if phase_max == None: - phase_max = 180 - - x = dataOut.getTimeRange() - y = dataOut.getHeiRange() - - thisDatetime = dataOut.datatime - - title = wintitle + " CoherenceMap" #: %s" %(thisDatetime.strftime("%d-%b-%Y")) - xlabel = "" - ylabel = "Range (Km)" - update_figfile = False - - if not self.isConfig: - nplots = len(pairsIndexList) - self.setup(id=id, - nplots=nplots, - wintitle=wintitle, - showprofile=showprofile, - show=show) - - if timerange != None: - self.timerange = timerange - - self.xmin, self.xmax = self.getTimeLim(x, xmin, xmax, timerange) - - if ymin == None: ymin = numpy.nanmin(y) - if ymax == None: ymax = numpy.nanmax(y) - if zmin == None: zmin = 0. - if zmax == None: zmax = 1. - - self.FTP_WEI = ftp_wei - self.EXP_CODE = exp_code - self.SUB_EXP_CODE = sub_exp_code - self.PLOT_POS = plot_pos - - self.name = thisDatetime.strftime("%Y%m%d_%H%M%S") - - self.isConfig = True - update_figfile = True - - self.setWinTitle(title) - - for i in range(self.nplots): - - pair = dataOut.pairsList[pairsIndexList[i]] - - ccf = numpy.average(dataOut.data_cspc[pairsIndexList[i],:,:],axis=0) - powa = numpy.average(dataOut.data_spc[pair[0],:,:],axis=0) - powb = numpy.average(dataOut.data_spc[pair[1],:,:],axis=0) - - - avgcoherenceComplex = ccf/numpy.sqrt(powa*powb) - coherence = numpy.abs(avgcoherenceComplex) - - z = coherence.reshape((1,-1)) - - counter = 0 - - title = "Coherence Ch%d * Ch%d: %s" %(pair[0], pair[1], thisDatetime.strftime("%d-%b-%Y %H:%M:%S")) - axes = self.axesList[i*self.__nsubplots*2] - axes.pcolorbuffer(x, y, z, - xmin=self.xmin, xmax=self.xmax, ymin=ymin, ymax=ymax, zmin=zmin, zmax=zmax, - xlabel=xlabel, ylabel=ylabel, title=title, rti=True, XAxisAsTime=True, - ticksize=9, cblabel='', colormap=coherence_cmap, cbsize="1%") - - if self.__showprofile: - counter += 1 - axes = self.axesList[i*self.__nsubplots*2 + counter] - axes.pline(coherence, y, - xmin=zmin, xmax=zmax, ymin=ymin, ymax=ymax, - xlabel='', ylabel='', title='', ticksize=7, - ytick_visible=False, nxticks=5, - grid='x') - - counter += 1 - - phase = numpy.arctan2(avgcoherenceComplex.imag, avgcoherenceComplex.real)*180/numpy.pi - - z = phase.reshape((1,-1)) - - title = "Phase Ch%d * Ch%d: %s" %(pair[0], pair[1], thisDatetime.strftime("%d-%b-%Y %H:%M:%S")) - axes = self.axesList[i*self.__nsubplots*2 + counter] - axes.pcolorbuffer(x, y, z, - xmin=self.xmin, xmax=self.xmax, ymin=ymin, ymax=ymax, zmin=phase_min, zmax=phase_max, - xlabel=xlabel, ylabel=ylabel, title=title, rti=True, XAxisAsTime=True, - ticksize=9, cblabel='', colormap=phase_cmap, cbsize="1%") - - if self.__showprofile: - counter += 1 - axes = self.axesList[i*self.__nsubplots*2 + counter] - axes.pline(phase, y, - xmin=phase_min, xmax=phase_max, ymin=ymin, ymax=ymax, - xlabel='', ylabel='', title='', ticksize=7, - ytick_visible=False, nxticks=4, - grid='x') - - self.draw() - - if dataOut.ltctime >= self.xmax: - self.counter_imagwr = wr_period - self.isConfig = False - update_figfile = True - - self.save(figpath=figpath, - figfile=figfile, - save=save, - ftp=ftp, - wr_period=wr_period, - thisDatetime=thisDatetime, - update_figfile=update_figfile) - -class PowerProfilePlot(Figure): - - isConfig = None - __nsubplots = None - - WIDTHPROF = None - HEIGHTPROF = None - PREFIX = 'spcprofile' - - def __init__(self, **kwargs): - Figure.__init__(self, **kwargs) - self.isConfig = False - self.__nsubplots = 1 - - self.PLOT_CODE = POWER_CODE - - self.WIDTH = 300 - self.HEIGHT = 500 - self.counter_imagwr = 0 - - def getSubplots(self): - ncol = 1 - nrow = 1 - - return nrow, ncol - - def setup(self, id, nplots, wintitle, show): - - self.nplots = nplots - - ncolspan = 1 - colspan = 1 - self.createFigure(id = id, - wintitle = wintitle, - widthplot = self.WIDTH, - heightplot = self.HEIGHT, - show=show) +import os +import datetime +import numpy - nrow, ncol = self.getSubplots() +from schainpy.model.graphics.jroplot_base import Plot, plt - counter = 0 - for y in range(nrow): - for x in range(ncol): - self.addAxes(nrow, ncol*ncolspan, y, x*ncolspan, colspan, 1) - def run(self, dataOut, id, wintitle="", channelList=None, - xmin=None, xmax=None, ymin=None, ymax=None, - save=False, figpath='./', figfile=None, show=True, - ftp=False, wr_period=1, server=None, - folder=None, username=None, password=None): +class SpectraPlot(Plot): + ''' + Plot for Spectra data + ''' + CODE = 'spc' + colormap = 'jet' + plot_name = 'Spectra' + plot_type = 'pcolor' - if channelList == None: - channelIndexList = dataOut.channelIndexList - channelList = dataOut.channelList + def setup(self): + self.nplots = len(self.data.channels) + self.ncols = int(numpy.sqrt(self.nplots) + 0.9) + self.nrows = int((1.0 * self.nplots / self.ncols) + 0.9) + self.height = 3 * self.nrows + self.cb_label = 'dB' + if self.showprofile: + self.width = 4 * self.ncols else: - channelIndexList = [] - for channel in channelList: - if channel not in dataOut.channelList: - raise ValueError, "Channel %d is not in dataOut.channelList" - channelIndexList.append(dataOut.channelList.index(channel)) - - factor = dataOut.normFactor - - y = dataOut.getHeiRange() - - #for voltage - if dataOut.type == 'Voltage': - x = dataOut.data[channelIndexList,:] * numpy.conjugate(dataOut.data[channelIndexList,:]) - x = x.real - x = numpy.where(numpy.isfinite(x), x, numpy.NAN) - - #for spectra - if dataOut.type == 'Spectra': - x = dataOut.data_spc[channelIndexList,:,:]/factor - x = numpy.where(numpy.isfinite(x), x, numpy.NAN) - x = numpy.average(x, axis=1) - - - xdB = 10*numpy.log10(x) - - thisDatetime = datetime.datetime.utcfromtimestamp(dataOut.getTimeRange()[0]) - title = wintitle + " Power Profile %s" %(thisDatetime.strftime("%d-%b-%Y")) - xlabel = "dB" - ylabel = "Range (Km)" - - if not self.isConfig: - - nplots = 1 - - self.setup(id=id, - nplots=nplots, - wintitle=wintitle, - show=show) - - if ymin == None: ymin = numpy.nanmin(y) - if ymax == None: ymax = numpy.nanmax(y) - if xmin == None: xmin = numpy.nanmin(xdB)*0.9 - if xmax == None: xmax = numpy.nanmax(xdB)*1.1 - - self.isConfig = True - - self.setWinTitle(title) - - title = "Power Profile: %s" %(thisDatetime.strftime("%d-%b-%Y %H:%M:%S")) - axes = self.axesList[0] - - legendlabels = ["channel %d"%x for x in channelList] - axes.pmultiline(xdB, y, - xmin=xmin, xmax=xmax, ymin=ymin, ymax=ymax, - xlabel=xlabel, ylabel=ylabel, title=title, legendlabels=legendlabels, - ytick_visible=True, nxticks=5, - grid='x') - - self.draw() - - self.save(figpath=figpath, - figfile=figfile, - save=save, - ftp=ftp, - wr_period=wr_period, - thisDatetime=thisDatetime) - -class SpectraCutPlot(Figure): - - isConfig = None - __nsubplots = None - - WIDTHPROF = None - HEIGHTPROF = None - PREFIX = 'spc_cut' - - def __init__(self, **kwargs): - Figure.__init__(self, **kwargs) - self.isConfig = False - self.__nsubplots = 1 - - self.PLOT_CODE = POWER_CODE - - self.WIDTH = 700 - self.HEIGHT = 500 - self.counter_imagwr = 0 - - def getSubplots(self): - ncol = 1 - nrow = 1 - - return nrow, ncol - - def setup(self, id, nplots, wintitle, show): - - self.nplots = nplots - - ncolspan = 1 - colspan = 1 - - self.createFigure(id = id, - wintitle = wintitle, - widthplot = self.WIDTH, - heightplot = self.HEIGHT, - show=show) - - nrow, ncol = self.getSubplots() - - counter = 0 - for y in range(nrow): - for x in range(ncol): - self.addAxes(nrow, ncol*ncolspan, y, x*ncolspan, colspan, 1) - - def run(self, dataOut, id, wintitle="", channelList=None, - xmin=None, xmax=None, ymin=None, ymax=None, - save=False, figpath='./', figfile=None, show=True, - ftp=False, wr_period=1, server=None, - folder=None, username=None, password=None, - xaxis="frequency"): - - - if channelList == None: - channelIndexList = dataOut.channelIndexList - channelList = dataOut.channelList + self.width = 3.5 * self.ncols + self.plots_adjust.update({'wspace': 0.4, 'hspace':0.4, 'left': 0.1, 'right': 0.9, 'bottom': 0.08}) + self.ylabel = 'Range [km]' + + def plot(self): + if self.xaxis == "frequency": + x = self.data.xrange[0] + self.xlabel = "Frequency (kHz)" + elif self.xaxis == "time": + x = self.data.xrange[1] + self.xlabel = "Time (ms)" else: - channelIndexList = [] - for channel in channelList: - if channel not in dataOut.channelList: - raise ValueError, "Channel %d is not in dataOut.channelList" - channelIndexList.append(dataOut.channelList.index(channel)) - - factor = dataOut.normFactor - - y = dataOut.getHeiRange() - - z = dataOut.data_spc/factor - z = numpy.where(numpy.isfinite(z), z, numpy.NAN) - - hei_index = numpy.arange(25)*3 + 20 - - if xaxis == "frequency": - x = dataOut.getFreqRange()/1000. - zdB = 10*numpy.log10(z[0,:,hei_index]) - xlabel = "Frequency (kHz)" - ylabel = "Power (dB)" - - elif xaxis == "time": - x = dataOut.getAcfRange() - zdB = z[0,:,hei_index] - xlabel = "Time (ms)" - ylabel = "ACF" - + x = self.data.xrange[2] + self.xlabel = "Velocity (m/s)" + + if self.CODE == 'spc_moments': + x = self.data.xrange[2] + self.xlabel = "Velocity (m/s)" + + self.titles = [] + + y = self.data.heights + self.y = y + z = self.data['spc'] + + for n, ax in enumerate(self.axes): + noise = self.data['noise'][n][-1] + if self.CODE == 'spc_moments': + mean = self.data['moments'][n, :, 1, :][-1] + if ax.firsttime: + self.xmax = self.xmax if self.xmax else numpy.nanmax(x) + self.xmin = self.xmin if self.xmin else -self.xmax + self.zmin = self.zmin if self.zmin else numpy.nanmin(z) + self.zmax = self.zmax if self.zmax else numpy.nanmax(z) + ax.plt = ax.pcolormesh(x, y, z[n].T, + vmin=self.zmin, + vmax=self.zmax, + cmap=plt.get_cmap(self.colormap) + ) + + if self.showprofile: + ax.plt_profile = self.pf_axes[n].plot( + self.data['rti'][n][-1], y)[0] + ax.plt_noise = self.pf_axes[n].plot(numpy.repeat(noise, len(y)), y, + color="k", linestyle="dashed", lw=1)[0] + if self.CODE == 'spc_moments': + ax.plt_mean = ax.plot(mean, y, color='k')[0] + else: + ax.plt.set_array(z[n].T.ravel()) + if self.showprofile: + ax.plt_profile.set_data(self.data['rti'][n][-1], y) + ax.plt_noise.set_data(numpy.repeat(noise, len(y)), y) + if self.CODE == 'spc_moments': + ax.plt_mean.set_data(mean, y) + self.titles.append('CH {}: {:3.2f}dB'.format(n, noise)) + + +class CrossSpectraPlot(Plot): + + CODE = 'cspc' + colormap = 'jet' + plot_name = 'CrossSpectra' + plot_type = 'pcolor' + zmin_coh = None + zmax_coh = None + zmin_phase = None + zmax_phase = None + + def setup(self): + + self.ncols = 4 + self.nrows = len(self.data.pairs) + self.nplots = self.nrows * 4 + self.width = 3.4 * self.ncols + self.height = 3 * self.nrows + self.ylabel = 'Range [km]' + self.showprofile = False + self.plots_adjust.update({'bottom': 0.08}) + + def plot(self): + + if self.xaxis == "frequency": + x = self.data.xrange[0] + self.xlabel = "Frequency (kHz)" + elif self.xaxis == "time": + x = self.data.xrange[1] + self.xlabel = "Time (ms)" else: - x = dataOut.getVelRange() - zdB = 10*numpy.log10(z[0,:,hei_index]) - xlabel = "Velocity (m/s)" - ylabel = "Power (dB)" - - thisDatetime = datetime.datetime.utcfromtimestamp(dataOut.getTimeRange()[0]) - title = wintitle + " Range Cuts %s" %(thisDatetime.strftime("%d-%b-%Y")) - - if not self.isConfig: - - nplots = 1 - - self.setup(id=id, - nplots=nplots, - wintitle=wintitle, - show=show) - - if xmin == None: xmin = numpy.nanmin(x)*0.9 - if xmax == None: xmax = numpy.nanmax(x)*1.1 - if ymin == None: ymin = numpy.nanmin(zdB) - if ymax == None: ymax = numpy.nanmax(zdB) - - self.isConfig = True - - self.setWinTitle(title) - - title = "Spectra Cuts: %s" %(thisDatetime.strftime("%d-%b-%Y %H:%M:%S")) - axes = self.axesList[0] - - legendlabels = ["Range = %dKm" %y[i] for i in hei_index] - - axes.pmultilineyaxis( x, zdB, - xmin=xmin, xmax=xmax, ymin=ymin, ymax=ymax, - xlabel=xlabel, ylabel=ylabel, title=title, legendlabels=legendlabels, - ytick_visible=True, nxticks=5, - grid='x') - - self.draw() - - self.save(figpath=figpath, - figfile=figfile, - save=save, - ftp=ftp, - wr_period=wr_period, - thisDatetime=thisDatetime) - -class Noise(Figure): - - isConfig = None - __nsubplots = None - - PREFIX = 'noise' - - - def __init__(self, **kwargs): - Figure.__init__(self, **kwargs) - self.timerange = 24*60*60 - self.isConfig = False - self.__nsubplots = 1 - self.counter_imagwr = 0 - self.WIDTH = 800 - self.HEIGHT = 400 - self.WIDTHPROF = 120 - self.HEIGHTPROF = 0 - self.xdata = None - self.ydata = None - - self.PLOT_CODE = NOISE_CODE - - self.FTP_WEI = None - self.EXP_CODE = None - self.SUB_EXP_CODE = None - self.PLOT_POS = None - self.figfile = None - - self.xmin = None - self.xmax = None - - def getSubplots(self): - - ncol = 1 - nrow = 1 - - return nrow, ncol - - def openfile(self, filename): - dirname = os.path.dirname(filename) - - if not os.path.exists(dirname): - os.mkdir(dirname) - - f = open(filename,'w+') - f.write('\n\n') - f.write('JICAMARCA RADIO OBSERVATORY - Noise \n') - f.write('DD MM YYYY HH MM SS Channel0 Channel1 Channel2 Channel3\n\n' ) - f.close() - - def save_data(self, filename_phase, data, data_datetime): - - f=open(filename_phase,'a') - - timetuple_data = data_datetime.timetuple() - day = str(timetuple_data.tm_mday) - month = str(timetuple_data.tm_mon) - year = str(timetuple_data.tm_year) - hour = str(timetuple_data.tm_hour) - minute = str(timetuple_data.tm_min) - second = str(timetuple_data.tm_sec) - - data_msg = '' - for i in range(len(data)): - data_msg += str(data[i]) + ' ' - - f.write(day+' '+month+' '+year+' '+hour+' '+minute+' '+second+' ' + data_msg + '\n') - f.close() + x = self.data.xrange[2] + self.xlabel = "Velocity (m/s)" + + self.titles = [] + + y = self.data.heights + self.y = y + nspc = self.data['spc'] + spc = self.data['cspc'][0] + cspc = self.data['cspc'][1] + + for n in range(self.nrows): + noise = self.data['noise'][:,-1] + pair = self.data.pairs[n] + ax = self.axes[4 * n] + if ax.firsttime: + self.xmax = self.xmax if self.xmax else numpy.nanmax(x) + self.xmin = self.xmin if self.xmin else -self.xmax + self.zmin = self.zmin if self.zmin else numpy.nanmin(nspc) + self.zmax = self.zmax if self.zmax else numpy.nanmax(nspc) + ax.plt = ax.pcolormesh(x , y , nspc[pair[0]].T, + vmin=self.zmin, + vmax=self.zmax, + cmap=plt.get_cmap(self.colormap) + ) + else: + ax.plt.set_array(nspc[pair[0]].T.ravel()) + self.titles.append('CH {}: {:3.2f}dB'.format(pair[0], noise[pair[0]])) + + ax = self.axes[4 * n + 1] + if ax.firsttime: + ax.plt = ax.pcolormesh(x , y, nspc[pair[1]].T, + vmin=self.zmin, + vmax=self.zmax, + cmap=plt.get_cmap(self.colormap) + ) + else: + ax.plt.set_array(nspc[pair[1]].T.ravel()) + self.titles.append('CH {}: {:3.2f}dB'.format(pair[1], noise[pair[1]])) + + out = cspc[n] / numpy.sqrt(spc[pair[0]] * spc[pair[1]]) + coh = numpy.abs(out) + phase = numpy.arctan2(out.imag, out.real) * 180 / numpy.pi + + ax = self.axes[4 * n + 2] + if ax.firsttime: + ax.plt = ax.pcolormesh(x, y, coh.T, + vmin=0, + vmax=1, + cmap=plt.get_cmap(self.colormap_coh) + ) + else: + ax.plt.set_array(coh.T.ravel()) + self.titles.append( + 'Coherence Ch{} * Ch{}'.format(pair[0], pair[1])) + + ax = self.axes[4 * n + 3] + if ax.firsttime: + ax.plt = ax.pcolormesh(x, y, phase.T, + vmin=-180, + vmax=180, + cmap=plt.get_cmap(self.colormap_phase) + ) + else: + ax.plt.set_array(phase.T.ravel()) + self.titles.append('Phase CH{} * CH{}'.format(pair[0], pair[1])) + + +class RTIPlot(Plot): + ''' + Plot for RTI data + ''' + + CODE = 'rti' + colormap = 'jet' + plot_name = 'RTI' + plot_type = 'pcolorbuffer' + + def setup(self): + self.xaxis = 'time' + self.ncols = 1 + self.nrows = len(self.data.channels) + self.nplots = len(self.data.channels) + self.ylabel = 'Range [km]' + self.xlabel = 'Time' + self.cb_label = 'dB' + self.plots_adjust.update({'hspace':0.8, 'left': 0.1, 'bottom': 0.08, 'right':0.95}) + self.titles = ['{} Channel {}'.format( + self.CODE.upper(), x) for x in range(self.nrows)] + + def plot(self): + self.x = self.data.times + self.y = self.data.heights + self.z = self.data[self.CODE] + self.z = numpy.ma.masked_invalid(self.z) + + if self.decimation is None: + x, y, z = self.fill_gaps(self.x, self.y, self.z) + else: + x, y, z = self.fill_gaps(*self.decimate()) + + for n, ax in enumerate(self.axes): + self.zmin = self.zmin if self.zmin else numpy.min(self.z) + self.zmax = self.zmax if self.zmax else numpy.max(self.z) + if ax.firsttime: + ax.plt = ax.pcolormesh(x, y, z[n].T, + vmin=self.zmin, + vmax=self.zmax, + cmap=plt.get_cmap(self.colormap) + ) + if self.showprofile: + ax.plot_profile = self.pf_axes[n].plot( + self.data['rti'][n][-1], self.y)[0] + ax.plot_noise = self.pf_axes[n].plot(numpy.repeat(self.data['noise'][n][-1], len(self.y)), self.y, + color="k", linestyle="dashed", lw=1)[0] + else: + ax.collections.remove(ax.collections[0]) + ax.plt = ax.pcolormesh(x, y, z[n].T, + vmin=self.zmin, + vmax=self.zmax, + cmap=plt.get_cmap(self.colormap) + ) + if self.showprofile: + ax.plot_profile.set_data(self.data['rti'][n][-1], self.y) + ax.plot_noise.set_data(numpy.repeat( + self.data['noise'][n][-1], len(self.y)), self.y) + + +class CoherencePlot(RTIPlot): + ''' + Plot for Coherence data + ''' + + CODE = 'coh' + plot_name = 'Coherence' + + def setup(self): + self.xaxis = 'time' + self.ncols = 1 + self.nrows = len(self.data.pairs) + self.nplots = len(self.data.pairs) + self.ylabel = 'Range [km]' + self.xlabel = 'Time' + self.plots_adjust.update({'hspace':0.6, 'left': 0.1, 'bottom': 0.1,'right':0.95}) + if self.CODE == 'coh': + self.cb_label = '' + self.titles = [ + 'Coherence Map Ch{} * Ch{}'.format(x[0], x[1]) for x in self.data.pairs] + else: + self.cb_label = 'Degrees' + self.titles = [ + 'Phase Map Ch{} * Ch{}'.format(x[0], x[1]) for x in self.data.pairs] - def setup(self, id, nplots, wintitle, showprofile=True, show=True): +class PhasePlot(CoherencePlot): + ''' + Plot for Phase map data + ''' - self.__showprofile = showprofile - self.nplots = nplots + CODE = 'phase' + colormap = 'seismic' + plot_name = 'Phase' - ncolspan = 7 - colspan = 6 - self.__nsubplots = 2 - self.createFigure(id = id, - wintitle = wintitle, - widthplot = self.WIDTH+self.WIDTHPROF, - heightplot = self.HEIGHT+self.HEIGHTPROF, - show=show) +class NoisePlot(Plot): + ''' + Plot for noise + ''' - nrow, ncol = self.getSubplots() + CODE = 'noise' + plot_name = 'Noise' + plot_type = 'scatterbuffer' - self.addAxes(nrow, ncol*ncolspan, 0, 0, colspan, 1) + def setup(self): + self.xaxis = 'time' + self.ncols = 1 + self.nrows = 1 + self.nplots = 1 + self.ylabel = 'Intensity [dB]' + self.xlabel = 'Time' + self.titles = ['Noise'] + self.colorbar = False - def run(self, dataOut, id, wintitle="", channelList=None, showprofile='True', - xmin=None, xmax=None, ymin=None, ymax=None, - timerange=None, - save=False, figpath='./', figfile=None, show=True, ftp=False, wr_period=1, - server=None, folder=None, username=None, password=None, - ftp_wei=0, exp_code=0, sub_exp_code=0, plot_pos=0): + def plot(self): - if not isTimeInHourRange(dataOut.datatime, xmin, xmax): - return + x = self.data.times + xmin = self.data.min_time + xmax = xmin + self.xrange * 60 * 60 + Y = self.data[self.CODE] - if channelList == None: - channelIndexList = dataOut.channelIndexList - channelList = dataOut.channelList + if self.axes[0].firsttime: + for ch in self.data.channels: + y = Y[ch] + self.axes[0].plot(x, y, lw=1, label='Ch{}'.format(ch)) + plt.legend() else: - channelIndexList = [] - for channel in channelList: - if channel not in dataOut.channelList: - raise ValueError, "Channel %d is not in dataOut.channelList" - channelIndexList.append(dataOut.channelList.index(channel)) - - x = dataOut.getTimeRange() - #y = dataOut.getHeiRange() - factor = dataOut.normFactor - noise = dataOut.noise[channelIndexList]/factor - noisedB = 10*numpy.log10(noise) - - thisDatetime = dataOut.datatime - - title = wintitle + " Noise" # : %s" %(thisDatetime.strftime("%d-%b-%Y")) - xlabel = "" - ylabel = "Intensity (dB)" - update_figfile = False - - if not self.isConfig: - - nplots = 1 - - self.setup(id=id, - nplots=nplots, - wintitle=wintitle, - showprofile=showprofile, - show=show) - - if timerange != None: - self.timerange = timerange - - self.xmin, self.xmax = self.getTimeLim(x, xmin, xmax, timerange) - - if ymin == None: ymin = numpy.floor(numpy.nanmin(noisedB)) - 10.0 - if ymax == None: ymax = numpy.nanmax(noisedB) + 10.0 - - self.FTP_WEI = ftp_wei - self.EXP_CODE = exp_code - self.SUB_EXP_CODE = sub_exp_code - self.PLOT_POS = plot_pos + for ch in self.data.channels: + y = Y[ch] + self.axes[0].lines[ch].set_data(x, y) + self.ymin = numpy.nanmin(Y) - 5 + self.ymax = numpy.nanmax(Y) + 5 - self.name = thisDatetime.strftime("%Y%m%d_%H%M%S") - self.isConfig = True - self.figfile = figfile - self.xdata = numpy.array([]) - self.ydata = numpy.array([]) - update_figfile = True +class PowerProfilePlot(Plot): - #open file beacon phase - path = '%s%03d' %(self.PREFIX, self.id) - noise_file = os.path.join(path,'%s.txt'%self.name) - self.filename_noise = os.path.join(figpath,noise_file) + CODE = 'spcprofile' + plot_name = 'Power Profile' + plot_type = 'scatter' + buffering = False - self.setWinTitle(title) + def setup(self): - title = "Noise %s" %(thisDatetime.strftime("%Y/%m/%d %H:%M:%S")) + self.ncols = 1 + self.nrows = 1 + self.nplots = 1 + self.height = 4 + self.width = 3 + self.ylabel = 'Range [km]' + self.xlabel = 'Intensity [dB]' + self.titles = ['Power Profile'] + self.colorbar = False - legendlabels = ["channel %d"%(idchannel) for idchannel in channelList] - axes = self.axesList[0] + def plot(self): - self.xdata = numpy.hstack((self.xdata, x[0:1])) + y = self.data.heights + self.y = y - if len(self.ydata)==0: - self.ydata = noisedB.reshape(-1,1) + x = self.data['spcprofile'] + + if self.xmin is None: self.xmin = numpy.nanmin(x)*0.9 + if self.xmax is None: self.xmax = numpy.nanmax(x)*1.1 + + if self.axes[0].firsttime: + for ch in self.data.channels: + self.axes[0].plot(x[ch], y, lw=1, label='Ch{}'.format(ch)) + plt.legend() else: - self.ydata = numpy.hstack((self.ydata, noisedB.reshape(-1,1))) - - - axes.pmultilineyaxis(x=self.xdata, y=self.ydata, - xmin=self.xmin, xmax=self.xmax, ymin=ymin, ymax=ymax, - xlabel=xlabel, ylabel=ylabel, title=title, legendlabels=legendlabels, marker='x', markersize=8, linestyle="solid", - XAxisAsTime=True, grid='both' - ) + for ch in self.data.channels: + self.axes[0].lines[ch].set_data(x[ch], y) + + +class SpectraCutPlot(Plot): + + CODE = 'spc_cut' + plot_name = 'Spectra Cut' + plot_type = 'scatter' + buffering = False + + def setup(self): + + self.nplots = len(self.data.channels) + self.ncols = int(numpy.sqrt(self.nplots) + 0.9) + self.nrows = int((1.0 * self.nplots / self.ncols) + 0.9) + self.width = 3.4 * self.ncols + 1.5 + self.height = 3 * self.nrows + self.ylabel = 'Power [dB]' + self.colorbar = False + self.plots_adjust.update({'left':0.1, 'hspace':0.3, 'right': 0.75, 'bottom':0.08}) + + def plot(self): + if self.xaxis == "frequency": + x = self.data.xrange[0][1:] + self.xlabel = "Frequency (kHz)" + elif self.xaxis == "time": + x = self.data.xrange[1] + self.xlabel = "Time (ms)" + else: + x = self.data.xrange[2] + self.xlabel = "Velocity (m/s)" - self.draw() + self.titles = [] - if dataOut.ltctime >= self.xmax: - self.counter_imagwr = wr_period - self.isConfig = False - update_figfile = True + y = self.data.heights + #self.y = y + z = self.data['spc_cut'] - self.save(figpath=figpath, - figfile=figfile, - save=save, - ftp=ftp, - wr_period=wr_period, - thisDatetime=thisDatetime, - update_figfile=update_figfile) + if self.height_index: + index = numpy.array(self.height_index) + else: + index = numpy.arange(0, len(y), int((len(y))/9)) + + for n, ax in enumerate(self.axes): + if ax.firsttime: + self.xmax = self.xmax if self.xmax else numpy.nanmax(x) + self.xmin = self.xmin if self.xmin else -self.xmax + self.ymin = self.ymin if self.ymin else numpy.nanmin(z) + self.ymax = self.ymax if self.ymax else numpy.nanmax(z) + ax.plt = ax.plot(x, z[n, :, index].T) + labels = ['Range = {:2.1f}km'.format(y[i]) for i in index] + self.figures[0].legend(ax.plt, labels, loc='center right') + else: + for i, line in enumerate(ax.plt): + line.set_data(x, z[n, :, i]) + self.titles.append('CH {}'.format(n)) - #store data beacon phase - if save: - self.save_data(self.filename_noise, noisedB, thisDatetime) -class BeaconPhase(Figure): +class BeaconPhase(Plot): __isConfig = None __nsubplots = None PREFIX = 'beacon_phase' - def __init__(self, **kwargs): - Figure.__init__(self, **kwargs) + def __init__(self): + Plot.__init__(self) self.timerange = 24*60*60 self.isConfig = False self.__nsubplots = 1 @@ -1391,6 +496,8 @@ class BeaconPhase(Figure): f.write(day+' '+month+' '+year+' '+hour+' '+minute+' '+second+' '+str(data[0])+' '+str(data[1])+' '+str(data[2])+' '+str(data[3])+'\n') f.close() + def plot(self): + log.warning('TODO: Not yet implemented...') def run(self, dataOut, id, wintitle="", pairsList=None, showprofile='True', xmin=None, xmax=None, ymin=None, ymax=None, hmin=None, hmax=None, @@ -1399,6 +506,9 @@ class BeaconPhase(Figure): server=None, folder=None, username=None, password=None, ftp_wei=0, exp_code=0, sub_exp_code=0, plot_pos=0): + if dataOut.flagNoData: + return dataOut + if not isTimeInHourRange(dataOut.datatime, xmin, xmax): return @@ -1408,14 +518,14 @@ class BeaconPhase(Figure): pairsIndexList = [] for pair in pairsList: if pair not in dataOut.pairsList: - raise ValueError, "Pair %s is not in dataOut.pairsList" %(pair) + raise ValueError("Pair %s is not in dataOut.pairsList" %(pair)) pairsIndexList.append(dataOut.pairsList.index(pair)) if pairsIndexList == []: return -# if len(pairsIndexList) > 4: -# pairsIndexList = pairsIndexList[0:4] + # if len(pairsIndexList) > 4: + # pairsIndexList = pairsIndexList[0:4] hmin_index = None hmax_index = None @@ -1454,9 +564,6 @@ class BeaconPhase(Figure): avgcoherenceComplex = ccf/numpy.sqrt(powa*powb) phase = numpy.arctan2(avgcoherenceComplex.imag, avgcoherenceComplex.real)*180/numpy.pi - #print "Phase %d%d" %(pair[0], pair[1]) - #print phase[dataOut.beacon_heiIndexList] - if dataOut.beacon_heiIndexList: phase_beacon[i] = numpy.average(phase[dataOut.beacon_heiIndexList]) else: @@ -1540,3 +647,5 @@ class BeaconPhase(Figure): wr_period=wr_period, thisDatetime=thisDatetime, update_figfile=update_figfile) + + return dataOut \ No newline at end of file diff --git a/schainpy/model/graphics/jroplot_voltage.py b/schainpy/model/graphics/jroplot_voltage.py index ed4dfc5..7b98340 100644 --- a/schainpy/model/graphics/jroplot_voltage.py +++ b/schainpy/model/graphics/jroplot_voltage.py @@ -7,219 +7,296 @@ import os import datetime import numpy -from figure import Figure - -class Scope(Figure): - - isConfig = None - - def __init__(self, **kwargs): - Figure.__init__(self, **kwargs) - self.isConfig = False - self.WIDTH = 300 - self.HEIGHT = 200 - self.counter_imagwr = 0 - - def getSubplots(self): - - nrow = self.nplots - ncol = 3 - return nrow, ncol - - def setup(self, id, nplots, wintitle, show): - - self.nplots = nplots - - self.createFigure(id=id, - wintitle=wintitle, - show=show) - - nrow,ncol = self.getSubplots() - colspan = 3 - rowspan = 1 - - for i in range(nplots): - self.addAxes(nrow, ncol, i, 0, colspan, rowspan) - - def plot_iq(self, x, y, id, channelIndexList, thisDatetime, wintitle, show, xmin, xmax, ymin, ymax): +from schainpy.model.graphics.jroplot_base import Plot, plt + + +class ScopePlot(Plot): + + ''' + Plot for Scope + ''' + + CODE = 'scope' + plot_name = 'Scope' + plot_type = 'scatter' + + def setup(self): + + self.xaxis = 'Range (Km)' + self.ncols = 1 + self.nrows = 1 + self.nplots = 1 + self.ylabel = 'Intensity [dB]' + self.titles = ['Scope'] + self.colorbar = False + self.width = 6 + self.height = 4 + + def plot_iq(self, x, y, channelIndexList, thisDatetime, wintitle): + yreal = y[channelIndexList,:].real yimag = y[channelIndexList,:].imag - - title = wintitle + " Scope: %s" %(thisDatetime.strftime("%d-%b-%Y %H:%M:%S")) - xlabel = "Range (Km)" - ylabel = "Intensity - IQ" - - if not self.isConfig: - nplots = len(channelIndexList) - - self.setup(id=id, - nplots=nplots, - wintitle='', - show=show) - - if xmin == None: xmin = numpy.nanmin(x) - if xmax == None: xmax = numpy.nanmax(x) - if ymin == None: ymin = min(numpy.nanmin(yreal),numpy.nanmin(yimag)) - if ymax == None: ymax = max(numpy.nanmax(yreal),numpy.nanmax(yimag)) - - self.isConfig = True - - self.setWinTitle(title) - - for i in range(len(self.axesList)): - title = "Channel %d" %(i) - axes = self.axesList[i] + title = wintitle + " Scope: %s" %(thisDatetime.strftime("%d-%b-%Y")) + self.xlabel = "Range (Km)" + self.ylabel = "Intensity - IQ" + + self.y = yreal + self.x = x + self.xmin = min(x) + self.xmax = max(x) - axes.pline(x, yreal[i,:], - xmin=xmin, xmax=xmax, ymin=ymin, ymax=ymax, - xlabel=xlabel, ylabel=ylabel, title=title) - axes.addpline(x, yimag[i,:], idline=1, color="red", linestyle="solid", lw=2) - - def plot_power(self, x, y, id, channelIndexList, thisDatetime, wintitle, show, xmin, xmax, ymin, ymax): + self.titles[0] = title + + for i,ax in enumerate(self.axes): + title = "Channel %d" %(i) + if ax.firsttime: + ax.plt_r = ax.plot(x, yreal[i,:], color='b')[0] + ax.plt_i = ax.plot(x, yimag[i,:], color='r')[0] + else: + ax.plt_r.set_data(x, yreal[i,:]) + ax.plt_i.set_data(x, yimag[i,:]) + + def plot_power(self, x, y, channelIndexList, thisDatetime, wintitle): y = y[channelIndexList,:] * numpy.conjugate(y[channelIndexList,:]) yreal = y.real - - title = wintitle + " Scope: %s" %(thisDatetime.strftime("%d-%b-%Y %H:%M:%S")) - xlabel = "Range (Km)" - ylabel = "Intensity" - - if not self.isConfig: - nplots = len(channelIndexList) - - self.setup(id=id, - nplots=nplots, - wintitle='', - show=show) - - if xmin == None: xmin = numpy.nanmin(x) - if xmax == None: xmax = numpy.nanmax(x) - if ymin == None: ymin = numpy.nanmin(yreal) - if ymax == None: ymax = numpy.nanmax(yreal) - - self.isConfig = True - - self.setWinTitle(title) - - for i in range(len(self.axesList)): + yreal = 10*numpy.log10(yreal) + self.y = yreal + title = wintitle + " Scope: %s" %(thisDatetime.strftime("%d-%b-%Y")) + self.xlabel = "Range (Km)" + self.ylabel = "Intensity" + self.xmin = min(x) + self.xmax = max(x) + + + self.titles[0] = title + + for i,ax in enumerate(self.axes): title = "Channel %d" %(i) - axes = self.axesList[i] + ychannel = yreal[i,:] - axes.pline(x, ychannel, - xmin=xmin, xmax=xmax, ymin=ymin, ymax=ymax, - xlabel=xlabel, ylabel=ylabel, title=title) - - - def run(self, dataOut, id, wintitle="", channelList=None, - xmin=None, xmax=None, ymin=None, ymax=None, save=False, - figpath='./', figfile=None, show=True, wr_period=1, - ftp=False, server=None, folder=None, username=None, password=None, type='power', **kwargs): - - """ - - Input: - dataOut : - id : - wintitle : - channelList : - xmin : None, - xmax : None, - ymin : None, - ymax : None, - """ - - if channelList == None: - channelIndexList = dataOut.channelIndexList + + if ax.firsttime: + ax.plt_r = ax.plot(x, ychannel)[0] + else: + #pass + ax.plt_r.set_data(x, ychannel) + + def plot_weatherpower(self, x, y, channelIndexList, thisDatetime, wintitle): + + + y = y[channelIndexList,:] + yreal = y.real + yreal = 10*numpy.log10(yreal) + self.y = yreal + title = wintitle + " Scope: %s" %(thisDatetime.strftime("%d-%b-%Y %H:%M:%S")) + self.xlabel = "Range (Km)" + self.ylabel = "Intensity" + self.xmin = min(x) + self.xmax = max(x) + + self.titles[0] =title + for i,ax in enumerate(self.axes): + title = "Channel %d" %(i) + + ychannel = yreal[i,:] + + if ax.firsttime: + ax.plt_r = ax.plot(x, ychannel)[0] + else: + #pass + ax.plt_r.set_data(x, ychannel) + + def plot_weathervelocity(self, x, y, channelIndexList, thisDatetime, wintitle): + + x = x[channelIndexList,:] + yreal = y + self.y = yreal + title = wintitle + " Scope: %s" %(thisDatetime.strftime("%d-%b-%Y %H:%M:%S")) + self.xlabel = "Velocity (m/s)" + self.ylabel = "Range (Km)" + self.xmin = numpy.min(x) + self.xmax = numpy.max(x) + self.titles[0] =title + for i,ax in enumerate(self.axes): + title = "Channel %d" %(i) + xchannel = x[i,:] + if ax.firsttime: + ax.plt_r = ax.plot(xchannel, yreal)[0] + else: + #pass + ax.plt_r.set_data(xchannel, yreal) + + def plot_weatherspecwidth(self, x, y, channelIndexList, thisDatetime, wintitle): + + x = x[channelIndexList,:] + yreal = y + self.y = yreal + title = wintitle + " Scope: %s" %(thisDatetime.strftime("%d-%b-%Y %H:%M:%S")) + self.xlabel = "width " + self.ylabel = "Range (Km)" + self.xmin = numpy.min(x) + self.xmax = numpy.max(x) + self.titles[0] =title + for i,ax in enumerate(self.axes): + title = "Channel %d" %(i) + xchannel = x[i,:] + if ax.firsttime: + ax.plt_r = ax.plot(xchannel, yreal)[0] + else: + #pass + ax.plt_r.set_data(xchannel, yreal) + + def plot(self): + if self.channels: + channels = self.channels else: - channelIndexList = [] - for channel in channelList: - if channel not in dataOut.channelList: - raise ValueError, "Channel %d is not in dataOut.channelList" - channelIndexList.append(dataOut.channelList.index(channel)) - - thisDatetime = datetime.datetime.utcfromtimestamp(dataOut.getTimeRange()[0]) - - if dataOut.flagDataAsBlock: - - for i in range(dataOut.nProfiles): - - wintitle1 = wintitle + " [Profile = %d] " %i - - if type == "power": - self.plot_power(dataOut.heightList, - dataOut.data[:,i,:], - id, - channelIndexList, - thisDatetime, - wintitle1, - show, - xmin, - xmax, - ymin, - ymax) - - if type == "iq": - self.plot_iq(dataOut.heightList, - dataOut.data[:,i,:], - id, - channelIndexList, - thisDatetime, - wintitle1, - show, - xmin, - xmax, - ymin, - ymax) - - self.draw() - - str_datetime = thisDatetime.strftime("%Y%m%d_%H%M%S") - figfile = self.getFilename(name = str_datetime) + "_" + str(i) - - self.save(figpath=figpath, - figfile=figfile, - save=save, - ftp=ftp, - wr_period=wr_period, - thisDatetime=thisDatetime) - + channels = self.data.channels + + thisDatetime = datetime.datetime.utcfromtimestamp(self.data.times[-1]) + if self.CODE == "pp_power": + scope = self.data['pp_power'] + elif self.CODE == "pp_signal": + scope = self.data["pp_signal"] + elif self.CODE == "pp_velocity": + scope = self.data["pp_velocity"] + elif self.CODE == "pp_specwidth": + scope = self.data["pp_specwidth"] + else: + scope =self.data["scope"] + + if self.data.flagDataAsBlock: + + for i in range(self.data.nProfiles): + + wintitle1 = " [Profile = %d] " %i + if self.CODE =="scope": + if self.type == "power": + self.plot_power(self.data.heights, + scope[:,i,:], + channels, + thisDatetime, + wintitle1 + ) + + if self.type == "iq": + self.plot_iq(self.data.heights, + scope[:,i,:], + channels, + thisDatetime, + wintitle1 + ) + if self.CODE=="pp_power": + self.plot_weatherpower(self.data.heights, + scope[:,i,:], + channels, + thisDatetime, + wintitle + ) + if self.CODE=="pp_signal": + self.plot_weatherpower(self.data.heights, + scope[:,i,:], + channels, + thisDatetime, + wintitle + ) + if self.CODE=="pp_velocity": + self.plot_weathervelocity(scope[:,i,:], + self.data.heights, + channels, + thisDatetime, + wintitle + ) + if self.CODE=="pp_spcwidth": + self.plot_weatherspecwidth(scope[:,i,:], + self.data.heights, + channels, + thisDatetime, + wintitle + ) else: - wintitle += " [Profile = %d] " %dataOut.profileIndex - - if type == "power": - self.plot_power(dataOut.heightList, - dataOut.data, - id, - channelIndexList, + wintitle = " [Profile = %d] " %self.data.profileIndex + if self.CODE== "scope": + if self.type == "power": + self.plot_power(self.data.heights, + scope, + channels, + thisDatetime, + wintitle + ) + + if self.type == "iq": + self.plot_iq(self.data.heights, + scope, + channels, thisDatetime, - wintitle, - show, - xmin, - xmax, - ymin, - ymax) - - if type == "iq": - self.plot_iq(dataOut.heightList, - dataOut.data, - id, - channelIndexList, - thisDatetime, - wintitle, - show, - xmin, - xmax, - ymin, - ymax) - - self.draw() - - str_datetime = thisDatetime.strftime("%Y%m%d_%H%M%S") + "_" + str(dataOut.profileIndex) - figfile = self.getFilename(name = str_datetime) - - self.save(figpath=figpath, - figfile=figfile, - save=save, - ftp=ftp, - wr_period=wr_period, - thisDatetime=thisDatetime) + wintitle + ) + if self.CODE=="pp_power": + self.plot_weatherpower(self.data.heights, + scope, + channels, + thisDatetime, + wintitle + ) + if self.CODE=="pp_signal": + self.plot_weatherpower(self.data.heights, + scope, + channels, + thisDatetime, + wintitle + ) + if self.CODE=="pp_velocity": + self.plot_weathervelocity(scope, + self.data.heights, + channels, + thisDatetime, + wintitle + ) + if self.CODE=="pp_specwidth": + self.plot_weatherspecwidth(scope, + self.data.heights, + channels, + thisDatetime, + wintitle + ) + + + +class PulsepairPowerPlot(ScopePlot): + ''' + Plot for P= S+N + ''' + + CODE = 'pp_power' + plot_name = 'PulsepairPower' + plot_type = 'scatter' + buffering = False + +class PulsepairVelocityPlot(ScopePlot): + ''' + Plot for VELOCITY + ''' + CODE = 'pp_velocity' + plot_name = 'PulsepairVelocity' + plot_type = 'scatter' + buffering = False + +class PulsepairSpecwidthPlot(ScopePlot): + ''' + Plot for WIDTH + ''' + CODE = 'pp_specwidth' + plot_name = 'PulsepairSpecwidth' + plot_type = 'scatter' + buffering = False + +class PulsepairSignalPlot(ScopePlot): + ''' + Plot for S + ''' + + CODE = 'pp_signal' + plot_name = 'PulsepairSignal' + plot_type = 'scatter' + buffering = False diff --git a/schainpy/model/graphics/jroplotter.py b/schainpy/model/graphics/jroplotter.py deleted file mode 100644 index 118ca9b..0000000 --- a/schainpy/model/graphics/jroplotter.py +++ /dev/null @@ -1,240 +0,0 @@ -''' -Created on Jul 9, 2014 - -@author: roj-idl71 -''' -import os, sys -import datetime -import numpy -import traceback - -from time import sleep -from threading import Lock -# from threading import Thread - -import schainpy -import schainpy.admin - -from schainpy.model.proc.jroproc_base import Operation -from schainpy.model.serializer.data import obj2Dict, dict2Obj -from jroplot_correlation import * -from jroplot_heispectra import * -from jroplot_parameters import * -from jroplot_spectra import * -from jroplot_voltage import * - - -class Plotter(Operation): - - isConfig = None - name = None - __queue = None - - def __init__(self, plotter_name, plotter_queue=None, **kwargs): - - Operation.__init__(self, **kwargs) - - self.isConfig = False - self.name = plotter_name - self.__queue = plotter_queue - - def getSubplots(self): - - nrow = self.nplots - ncol = 1 - return nrow, ncol - - def setup(self, **kwargs): - - print "Initializing ..." - - - def run(self, dataOut, id=None, **kwargs): - - """ - - Input: - dataOut : - id : - """ - - packDict = {} - - packDict['id'] = id - packDict['name'] = self.name - packDict['kwargs'] = kwargs - -# packDict['data'] = obj2Dict(dataOut) - packDict['data'] = dataOut - - self.__queue.put(packDict) - -# class PlotManager(Thread): -class PlotManager(): - - __err = False - __stop = False - __realtime = False - - controllerThreadObj = None - - plotterList = ['Scope', - 'SpectraPlot', 'RTIPlot', - 'SpectraCutPlot', - 'CrossSpectraPlot', 'CoherenceMap', - 'PowerProfilePlot', 'Noise', 'BeaconPhase', - 'CorrelationPlot', - 'SpectraHeisScope', 'RTIfromSpectraHeis'] - - def __init__(self, plotter_queue): - -# Thread.__init__(self) -# self.setDaemon(True) - - self.__queue = plotter_queue - self.__lock = Lock() - - self.plotInstanceDict = {} - - self.__err = False - self.__stop = False - self.__realtime = False - - def __handleError(self, name="", send_email=False): - - err = traceback.format_exception(sys.exc_info()[0], - sys.exc_info()[1], - sys.exc_info()[2]) - - print "***** Error occurred in PlotManager *****" - print "***** [%s]: %s" %(name, err[-1]) - - message = "\nError ocurred in %s:\n" %name - message += "".join(err) - - sys.stderr.write(message) - - if not send_email: - return - - import socket - - subject = "SChain v%s: Error running %s\n" %(schainpy.__version__, name) - - subtitle = "%s:\n" %(name) - subtitle += "Hostname: %s\n" %socket.gethostbyname(socket.gethostname()) - subtitle += "Working directory: %s\n" %os.path.abspath("./") - # subtitle += "Configuration file: %s\n" %self.filename - subtitle += "Time: %s\n" %str(datetime.datetime.now()) - - adminObj = schainpy.admin.SchainNotify() - adminObj.sendAlert(message=message, - subject=subject, - subtitle=subtitle) - - def run(self): - - if self.__queue.empty(): - return - - if self.__err: - serial_data = self.__queue.get() - self.__queue.task_done() - return - - self.__lock.acquire() - -# if self.__queue.full(): -# for i in range(int(self.__queue.qsize()/2)): -# serial_data = self.__queue.get() -# self.__queue.task_done() - - n = int(self.__queue.qsize()/3 + 1) - - for i in range(n): - - if self.__queue.empty(): - break - - serial_data = self.__queue.get() - self.__queue.task_done() - - plot_id = serial_data['id'] - plot_name = serial_data['name'] - kwargs = serial_data['kwargs'] -# dataDict = serial_data['data'] -# -# dataPlot = dict2Obj(dataDict) - - dataPlot = serial_data['data'] - - if plot_id not in self.plotInstanceDict.keys(): - className = eval(plot_name) - self.plotInstanceDict[plot_id] = className(**kwargs) - - plotter = self.plotInstanceDict[plot_id] - try: - plotter.run(dataPlot, plot_id, **kwargs) - except: - self.__err = True - self.__handleError(plot_name, send_email=True) - break - - self.__lock.release() - - def isEmpty(self): - - return self.__queue.empty() - - def stop(self): - - self.__lock.acquire() - - self.__stop = True - - self.__lock.release() - - def close(self): - - self.__lock.acquire() - - for plot_id in self.plotInstanceDict.keys(): - plotter = self.plotInstanceDict[plot_id] - plotter.close() - - self.__lock.release() - - def setController(self, controllerThreadObj): - - self.controllerThreadObj = controllerThreadObj - - def start(self): - - if not self.controllerThreadObj.isRunning(): - raise RuntimeError, "controllerThreadObj has not been initialized. Use controllerThreadObj.start() before call this method" - - self.join() - - def join(self): - - #Execute plotter while controller is running - while self.controllerThreadObj.isRunning(): - self.run() - - self.controllerThreadObj.stop() - - #Wait until plotter queue is empty - while not self.isEmpty(): - self.run() - - self.close() - - def isErrorDetected(self): - - self.__lock.acquire() - - err = self.__err - - self.__lock.release() - - return err diff --git a/schainpy/model/graphics/mpldriver.py b/schainpy/model/graphics/mpldriver.py deleted file mode 100644 index 6173cd6..0000000 --- a/schainpy/model/graphics/mpldriver.py +++ /dev/null @@ -1,501 +0,0 @@ -import os -import sys -import datetime -import numpy -import matplotlib - -if 'BACKEND' in os.environ: - matplotlib.use(os.environ['BACKEND']) -elif 'linux' in sys.platform: - matplotlib.use("TkAgg") -elif 'darwin' in sys.platform: - matplotlib.use('TkAgg') -else: - from schainpy.utils import log - log.warning('Using default Backend="Agg"', 'INFO') - matplotlib.use('Agg') -# Qt4Agg', 'GTK', 'GTKAgg', 'ps', 'agg', 'cairo', 'MacOSX', 'GTKCairo', 'WXAgg', 'template', 'TkAgg', 'GTK3Cairo', 'GTK3Agg', 'svg', 'WebAgg', 'CocoaAgg', 'emf', 'gdk', 'WX' -import matplotlib.pyplot - -from mpl_toolkits.axes_grid1 import make_axes_locatable -from matplotlib.ticker import FuncFormatter, LinearLocator - -########################################### -# Actualizacion de las funciones del driver -########################################### - -# create jro colormap - -jet_values = matplotlib.pyplot.get_cmap("jet", 100)(numpy.arange(100))[10:90] -blu_values = matplotlib.pyplot.get_cmap( - "seismic_r", 20)(numpy.arange(20))[10:15] -ncmap = matplotlib.colors.LinearSegmentedColormap.from_list( - "jro", numpy.vstack((blu_values, jet_values))) -matplotlib.pyplot.register_cmap(cmap=ncmap) - - -def createFigure(id, wintitle, width, height, facecolor="w", show=True, dpi=80): - - matplotlib.pyplot.ioff() - - fig = matplotlib.pyplot.figure(num=id, facecolor=facecolor, figsize=( - 1.0 * width / dpi, 1.0 * height / dpi)) - fig.canvas.manager.set_window_title(wintitle) -# fig.canvas.manager.resize(width, height) - matplotlib.pyplot.ion() - - if show: - matplotlib.pyplot.show() - - return fig - - -def closeFigure(show=False, fig=None): - - # matplotlib.pyplot.ioff() - # matplotlib.pyplot.pause(0) - - if show: - matplotlib.pyplot.show() - - if fig != None: - matplotlib.pyplot.close(fig) -# matplotlib.pyplot.pause(0) -# matplotlib.pyplot.ion() - - return - - matplotlib.pyplot.close("all") -# matplotlib.pyplot.pause(0) -# matplotlib.pyplot.ion() - - return - - -def saveFigure(fig, filename): - - # matplotlib.pyplot.ioff() - fig.savefig(filename, dpi=matplotlib.pyplot.gcf().dpi) -# matplotlib.pyplot.ion() - - -def clearFigure(fig): - - fig.clf() - - -def setWinTitle(fig, title): - - fig.canvas.manager.set_window_title(title) - - -def setTitle(fig, title): - - fig.suptitle(title) - - -def createAxes(fig, nrow, ncol, xpos, ypos, colspan, rowspan, polar=False): - - matplotlib.pyplot.ioff() - matplotlib.pyplot.figure(fig.number) - axes = matplotlib.pyplot.subplot2grid((nrow, ncol), - (xpos, ypos), - colspan=colspan, - rowspan=rowspan, - polar=polar) - - matplotlib.pyplot.ion() - return axes - - -def setAxesText(ax, text): - - ax.annotate(text, - xy=(.1, .99), - xycoords='figure fraction', - horizontalalignment='left', - verticalalignment='top', - fontsize=10) - - -def printLabels(ax, xlabel, ylabel, title): - - ax.set_xlabel(xlabel, size=11) - ax.set_ylabel(ylabel, size=11) - ax.set_title(title, size=8) - - -def createPline(ax, x, y, xmin, xmax, ymin, ymax, xlabel='', ylabel='', title='', - ticksize=9, xtick_visible=True, ytick_visible=True, - nxticks=4, nyticks=10, - grid=None, color='blue'): - """ - - Input: - grid : None, 'both', 'x', 'y' - """ - - matplotlib.pyplot.ioff() - - ax.set_xlim([xmin, xmax]) - ax.set_ylim([ymin, ymax]) - - printLabels(ax, xlabel, ylabel, title) - - ###################################################### - if (xmax - xmin) <= 1: - xtickspos = numpy.linspace(xmin, xmax, nxticks) - xtickspos = numpy.array([float("%.1f" % i) for i in xtickspos]) - ax.set_xticks(xtickspos) - else: - xtickspos = numpy.arange(nxticks) * \ - int((xmax - xmin) / (nxticks)) + int(xmin) -# xtickspos = numpy.arange(nxticks)*float(xmax-xmin)/float(nxticks) + int(xmin) - ax.set_xticks(xtickspos) - - for tick in ax.get_xticklabels(): - tick.set_visible(xtick_visible) - - for tick in ax.xaxis.get_major_ticks(): - tick.label.set_fontsize(ticksize) - - ###################################################### - for tick in ax.get_yticklabels(): - tick.set_visible(ytick_visible) - - for tick in ax.yaxis.get_major_ticks(): - tick.label.set_fontsize(ticksize) - - ax.plot(x, y, color=color) - iplot = ax.lines[-1] - - ###################################################### - if '0.' in matplotlib.__version__[0:2]: - print "The matplotlib version has to be updated to 1.1 or newer" - return iplot - - if '1.0.' in matplotlib.__version__[0:4]: - print "The matplotlib version has to be updated to 1.1 or newer" - return iplot - - if grid != None: - ax.grid(b=True, which='major', axis=grid) - - matplotlib.pyplot.tight_layout() - - matplotlib.pyplot.ion() - - return iplot - - -def set_linedata(ax, x, y, idline): - - ax.lines[idline].set_data(x, y) - - -def pline(iplot, x, y, xlabel='', ylabel='', title=''): - - ax = iplot.axes - - printLabels(ax, xlabel, ylabel, title) - - set_linedata(ax, x, y, idline=0) - - -def addpline(ax, x, y, color, linestyle, lw): - - ax.plot(x, y, color=color, linestyle=linestyle, lw=lw) - - -def createPcolor(ax, x, y, z, xmin, xmax, ymin, ymax, zmin, zmax, - xlabel='', ylabel='', title='', ticksize=9, - colormap='jet', cblabel='', cbsize="5%", - XAxisAsTime=False): - - matplotlib.pyplot.ioff() - - divider = make_axes_locatable(ax) - ax_cb = divider.new_horizontal(size=cbsize, pad=0.05) - fig = ax.get_figure() - fig.add_axes(ax_cb) - - ax.set_xlim([xmin, xmax]) - ax.set_ylim([ymin, ymax]) - - printLabels(ax, xlabel, ylabel, title) - - z = numpy.ma.masked_invalid(z) - cmap = matplotlib.pyplot.get_cmap(colormap) - cmap.set_bad('black', 1.) - imesh = ax.pcolormesh(x, y, z.T, vmin=zmin, vmax=zmax, cmap=cmap) - cb = matplotlib.pyplot.colorbar(imesh, cax=ax_cb) - cb.set_label(cblabel) - -# for tl in ax_cb.get_yticklabels(): -# tl.set_visible(True) - - for tick in ax.yaxis.get_major_ticks(): - tick.label.set_fontsize(ticksize) - - for tick in ax.xaxis.get_major_ticks(): - tick.label.set_fontsize(ticksize) - - for tick in cb.ax.get_yticklabels(): - tick.set_fontsize(ticksize) - - ax_cb.yaxis.tick_right() - - if '0.' in matplotlib.__version__[0:2]: - print "The matplotlib version has to be updated to 1.1 or newer" - return imesh - - if '1.0.' in matplotlib.__version__[0:4]: - print "The matplotlib version has to be updated to 1.1 or newer" - return imesh - - matplotlib.pyplot.tight_layout() - - if XAxisAsTime: - - def func(x, pos): return ('%s') % ( - datetime.datetime.utcfromtimestamp(x).strftime("%H:%M:%S")) - ax.xaxis.set_major_formatter(FuncFormatter(func)) - ax.xaxis.set_major_locator(LinearLocator(7)) - - matplotlib.pyplot.ion() - return imesh - - -def pcolor(imesh, z, xlabel='', ylabel='', title=''): - - z = z.T - ax = imesh.axes - printLabels(ax, xlabel, ylabel, title) - imesh.set_array(z.ravel()) - - -def addpcolor(ax, x, y, z, zmin, zmax, xlabel='', ylabel='', title='', colormap='jet'): - - printLabels(ax, xlabel, ylabel, title) - - ax.pcolormesh(x, y, z.T, vmin=zmin, vmax=zmax, - cmap=matplotlib.pyplot.get_cmap(colormap)) - - -def addpcolorbuffer(ax, x, y, z, zmin, zmax, xlabel='', ylabel='', title='', colormap='jet'): - - printLabels(ax, xlabel, ylabel, title) - - ax.collections.remove(ax.collections[0]) - - z = numpy.ma.masked_invalid(z) - - cmap = matplotlib.pyplot.get_cmap(colormap) - cmap.set_bad('black', 1.) - - ax.pcolormesh(x, y, z.T, vmin=zmin, vmax=zmax, cmap=cmap) - - -def createPmultiline(ax, x, y, xmin, xmax, ymin, ymax, xlabel='', ylabel='', title='', legendlabels=None, - ticksize=9, xtick_visible=True, ytick_visible=True, - nxticks=4, nyticks=10, - grid=None): - """ - - Input: - grid : None, 'both', 'x', 'y' - """ - - matplotlib.pyplot.ioff() - - lines = ax.plot(x.T, y) - leg = ax.legend(lines, legendlabels, loc='upper right') - leg.get_frame().set_alpha(0.5) - ax.set_xlim([xmin, xmax]) - ax.set_ylim([ymin, ymax]) - printLabels(ax, xlabel, ylabel, title) - - xtickspos = numpy.arange(nxticks) * \ - int((xmax - xmin) / (nxticks)) + int(xmin) - ax.set_xticks(xtickspos) - - for tick in ax.get_xticklabels(): - tick.set_visible(xtick_visible) - - for tick in ax.xaxis.get_major_ticks(): - tick.label.set_fontsize(ticksize) - - for tick in ax.get_yticklabels(): - tick.set_visible(ytick_visible) - - for tick in ax.yaxis.get_major_ticks(): - tick.label.set_fontsize(ticksize) - - iplot = ax.lines[-1] - - if '0.' in matplotlib.__version__[0:2]: - print "The matplotlib version has to be updated to 1.1 or newer" - return iplot - - if '1.0.' in matplotlib.__version__[0:4]: - print "The matplotlib version has to be updated to 1.1 or newer" - return iplot - - if grid != None: - ax.grid(b=True, which='major', axis=grid) - - matplotlib.pyplot.tight_layout() - - matplotlib.pyplot.ion() - - return iplot - - -def pmultiline(iplot, x, y, xlabel='', ylabel='', title=''): - - ax = iplot.axes - - printLabels(ax, xlabel, ylabel, title) - - for i in range(len(ax.lines)): - line = ax.lines[i] - line.set_data(x[i, :], y) - - -def createPmultilineYAxis(ax, x, y, xmin, xmax, ymin, ymax, xlabel='', ylabel='', title='', legendlabels=None, - ticksize=9, xtick_visible=True, ytick_visible=True, - nxticks=4, nyticks=10, marker='.', markersize=10, linestyle="None", - grid=None, XAxisAsTime=False): - """ - - Input: - grid : None, 'both', 'x', 'y' - """ - - matplotlib.pyplot.ioff() - -# lines = ax.plot(x, y.T, marker=marker,markersize=markersize,linestyle=linestyle) - lines = ax.plot(x, y.T) -# leg = ax.legend(lines, legendlabels, loc=2, bbox_to_anchor=(1.01, 1.00), numpoints=1, handlelength=1.5, \ -# handletextpad=0.5, borderpad=0.5, labelspacing=0.5, borderaxespad=0.) - - leg = ax.legend(lines, legendlabels, - loc='upper right', bbox_to_anchor=(1.16, 1), borderaxespad=0) - - for label in leg.get_texts(): - label.set_fontsize(9) - - ax.set_xlim([xmin, xmax]) - ax.set_ylim([ymin, ymax]) - printLabels(ax, xlabel, ylabel, title) - -# xtickspos = numpy.arange(nxticks)*int((xmax-xmin)/(nxticks)) + int(xmin) -# ax.set_xticks(xtickspos) - - for tick in ax.get_xticklabels(): - tick.set_visible(xtick_visible) - - for tick in ax.xaxis.get_major_ticks(): - tick.label.set_fontsize(ticksize) - - for tick in ax.get_yticklabels(): - tick.set_visible(ytick_visible) - - for tick in ax.yaxis.get_major_ticks(): - tick.label.set_fontsize(ticksize) - - iplot = ax.lines[-1] - - if '0.' in matplotlib.__version__[0:2]: - print "The matplotlib version has to be updated to 1.1 or newer" - return iplot - - if '1.0.' in matplotlib.__version__[0:4]: - print "The matplotlib version has to be updated to 1.1 or newer" - return iplot - - if grid != None: - ax.grid(b=True, which='major', axis=grid) - - matplotlib.pyplot.tight_layout() - - if XAxisAsTime: - - def func(x, pos): return ('%s') % ( - datetime.datetime.utcfromtimestamp(x).strftime("%H:%M:%S")) - ax.xaxis.set_major_formatter(FuncFormatter(func)) - ax.xaxis.set_major_locator(LinearLocator(7)) - - matplotlib.pyplot.ion() - - return iplot - - -def pmultilineyaxis(iplot, x, y, xlabel='', ylabel='', title=''): - - ax = iplot.axes - - printLabels(ax, xlabel, ylabel, title) - - for i in range(len(ax.lines)): - line = ax.lines[i] - line.set_data(x, y[i, :]) - - -def createPolar(ax, x, y, - xlabel='', ylabel='', title='', ticksize=9, - colormap='jet', cblabel='', cbsize="5%", - XAxisAsTime=False): - - matplotlib.pyplot.ioff() - - ax.plot(x, y, 'bo', markersize=5) -# ax.set_rmax(90) - ax.set_ylim(0, 90) - ax.set_yticks(numpy.arange(0, 90, 20)) -# ax.text(0, -110, ylabel, rotation='vertical', va ='center', ha = 'center' ,size='11') -# ax.text(0, 50, ylabel, rotation='vertical', va ='center', ha = 'left' ,size='11') -# ax.text(100, 100, 'example', ha='left', va='center', rotation='vertical') - ax.yaxis.labelpad = 40 - printLabels(ax, xlabel, ylabel, title) - iplot = ax.lines[-1] - - if '0.' in matplotlib.__version__[0:2]: - print "The matplotlib version has to be updated to 1.1 or newer" - return iplot - - if '1.0.' in matplotlib.__version__[0:4]: - print "The matplotlib version has to be updated to 1.1 or newer" - return iplot - -# if grid != None: -# ax.grid(b=True, which='major', axis=grid) - - matplotlib.pyplot.tight_layout() - - matplotlib.pyplot.ion() - - return iplot - - -def polar(iplot, x, y, xlabel='', ylabel='', title=''): - - ax = iplot.axes - -# ax.text(0, -110, ylabel, rotation='vertical', va ='center', ha = 'center',size='11') - printLabels(ax, xlabel, ylabel, title) - - set_linedata(ax, x, y, idline=0) - - -def draw(fig): - - if type(fig) == 'int': - raise ValueError, "Error drawing: Fig parameter should be a matplotlib figure object figure" - - fig.canvas.draw() - - -def pause(interval=0.000001): - - matplotlib.pyplot.pause(interval) diff --git a/schainpy/model/io/MIRAtest.py b/schainpy/model/io/MIRAtest.py index ea8e94f..a4e8e20 100644 --- a/schainpy/model/io/MIRAtest.py +++ b/schainpy/model/io/MIRAtest.py @@ -291,8 +291,8 @@ RadarConst5 = RadarConst # print 'OffsetStartHeader ',self.OffsetStartHeader,'RecCounter ', self.RecCounter, 'Off2StartNxtRec ' , self.Off2StartNxtRec #OffRHeader= self.OffsetStartHeader + self.RecCounter*self.Off2StartNxtRec #startFp.seek(OffRHeader, os.SEEK_SET) -print 'debe ser 48, RecCounter*811248', self.OffsetStartHeader, self.RecCounter, self.Off2StartNxtRec -print 'Posicion del bloque: ', OffRHeader +print('debe ser 48, RecCounter*811248', self.OffsetStartHeader, self.RecCounter, self.Off2StartNxtRec) +print('Posicion del bloque: ', OffRHeader) header = numpy.fromfile(startFp, SRVI_STRUCTURE, 1) @@ -326,6 +326,6 @@ self.Datasize = self.nProfiles * self.nChannels * self.nHeights * 2 * 4 # print 'Datasize',self.Datasize endFp = self.OffsetStartHeader + self.RecCounter * self.Off2StartNxtRec -print '==============================================' +print('==============================================') -print '==============================================' +print('==============================================') \ No newline at end of file diff --git a/schainpy/model/io/__init__.py b/schainpy/model/io/__init__.py index e35b986..f6c6768 100644 --- a/schainpy/model/io/__init__.py +++ b/schainpy/model/io/__init__.py @@ -4,20 +4,21 @@ $Author: murco $ $Id: JRODataIO.py 169 2012-11-19 21:57:03Z murco $ ''' -from jroIO_voltage import * -from jroIO_spectra import * -from jroIO_heispectra import * -from jroIO_usrp import * -from jroIO_digitalRF import * -from jroIO_kamisr import * -from jroIO_param import * -from jroIO_hf import * +from .jroIO_voltage import * +from .jroIO_spectra import * +from .jroIO_heispectra import * +from .jroIO_usrp import * +from .jroIO_digitalRF import * +from .jroIO_kamisr import * +from .jroIO_param import * +from .jroIO_hf import * -from jroIO_madrigal import * +from .jroIO_madrigal import * -from bltrIO_param import * -from jroIO_bltr import * -from jroIO_mira35c import * -from julIO_param import * +from .bltrIO_param import * +from .bltrIO_spectra import * +from .jroIO_mira35c import * +from .julIO_param import * -from pxIO_param import * \ No newline at end of file +from .pxIO_param import * +from .jroIO_simulator import * \ No newline at end of file diff --git a/schainpy/model/io/bltrIO_param.py b/schainpy/model/io/bltrIO_param.py index 1d286f7..e253d7d 100644 --- a/schainpy/model/io/bltrIO_param.py +++ b/schainpy/model/io/bltrIO_param.py @@ -13,9 +13,10 @@ import datetime import numpy -from schainpy.model.proc.jroproc_base import ProcessingUnit +import schainpy.admin +from schainpy.model.proc.jroproc_base import ProcessingUnit, MPDecorator from schainpy.model.data.jrodata import Parameters -from schainpy.model.io.jroIO_base import JRODataReader, isNumber +from schainpy.model.io.jroIO_base import Reader from schainpy.utils import log FILE_HEADER_STRUCTURE = numpy.dtype([ @@ -84,139 +85,113 @@ DATA_STRUCTURE = numpy.dtype([ ]) -class BLTRParamReader(JRODataReader, ProcessingUnit): +class BLTRParamReader(Reader, ProcessingUnit): ''' - Boundary Layer and Tropospheric Radar (BLTR) reader, Wind velocities and SNR from *.sswma files + Boundary Layer and Tropospheric Radar (BLTR) reader, Wind velocities and SNR + from *.sswma files ''' ext = '.sswma' - def __init__(self, **kwargs): + def __init__(self): - ProcessingUnit.__init__(self, **kwargs) + ProcessingUnit.__init__(self) self.dataOut = Parameters() + self.dataOut.timezone = 300 self.counter_records = 0 self.flagNoMoreFiles = 0 self.isConfig = False self.filename = None - - def setup(self, - path=None, - startDate=None, - endDate=None, - ext=None, - startTime=datetime.time(0, 0, 0), - endTime=datetime.time(23, 59, 59), - timezone=0, - status_value=0, - **kwargs): - - self.path = path - self.startDate = startDate - self.endDate = endDate - self.startTime = startTime - self.endTime = endTime - self.status_value = status_value + self.status_value = 0 self.datatime = datetime.datetime(1900,1,1) + self.filefmt = "*********%Y%m%d******" + + def setup(self, **kwargs): + + self.set_kwargs(**kwargs) if self.path is None: - raise ValueError, "The path is not valid" - - if ext is None: - ext = self.ext - - self.search_files(self.path, startDate, endDate, ext) - self.timezone = timezone - self.fileIndex = 0 + raise ValueError("The path is not valid") + + if self.online: + log.log("Searching files in online mode...", self.name) + + for nTries in range(self.nTries): + fullpath = self.searchFilesOnLine(self.path, self.startDate, + self.endDate, self.expLabel, self.ext, self.walk, + self.filefmt, self.folderfmt) + try: + fullpath = next(fullpath) + except: + fullpath = None + + if fullpath: + self.fileSize = os.path.getsize(fullpath) + self.filename = fullpath + self.flagIsNewFile = 1 + if self.fp != None: + self.fp.close() + self.fp = self.open_file(fullpath, self.open_mode) + self.flagNoMoreFiles = 0 + break - if not self.fileList: - raise Warning, "There is no files matching these date in the folder: %s. \n Check 'startDate' and 'endDate' " % ( - path) - - self.setNextFile() - - def search_files(self, path, startDate, endDate, ext): + log.warning( + 'Waiting {} sec for a valid file in {}: try {} ...'.format( + self.delay, self.path, nTries + 1), + self.name) + time.sleep(self.delay) + + if not(fullpath): + raise schainpy.admin.SchainError( + 'There isn\'t any valid file in {}'.format(self.path)) + self.readFirstHeader() + else: + log.log("Searching files in {}".format(self.path), self.name) + self.filenameList = self.searchFilesOffLine(self.path, self.startDate, + self.endDate, self.expLabel, self.ext, self.walk, self.filefmt, self.folderfmt) + self.setNextFile() + + def checkForRealPath(self, nextFile, nextDay): ''' - Searching for BLTR rawdata file in path - Creating a list of file to proces included in [startDate,endDate] - - Input: - path - Path to find BLTR rawdata files - startDate - Select file from this date - enDate - Select file until this date - ext - Extension of the file to read ''' - - log.success('Searching files in {} '.format(path), 'BLTRParamReader') - foldercounter = 0 - fileList0 = glob.glob1(path, "*%s" % ext) - fileList0.sort() - - self.fileList = [] - self.dateFileList = [] - - for thisFile in fileList0: - year = thisFile[-14:-10] - if not isNumber(year): - continue - - month = thisFile[-10:-8] - if not isNumber(month): - continue - - day = thisFile[-8:-6] - if not isNumber(day): - continue - - year, month, day = int(year), int(month), int(day) - dateFile = datetime.date(year, month, day) - - if (startDate > dateFile) or (endDate < dateFile): - continue - - self.fileList.append(thisFile) - self.dateFileList.append(dateFile) - - return - - def setNextFile(self): - file_id = self.fileIndex - - if file_id == len(self.fileList): - log.success('No more files in the folder', 'BLTRParamReader') - self.flagNoMoreFiles = 1 - return 0 + dt = self.datatime + datetime.timedelta(1) + filename = '{}.{}{}'.format(self.siteFile, dt.strftime('%Y%m%d'), self.ext) + fullfilename = os.path.join(self.path, filename) + if os.path.exists(fullfilename): + return fullfilename, filename + return None, filename - log.success('Opening {}'.format(self.fileList[file_id]), 'BLTRParamReader') - filename = os.path.join(self.path, self.fileList[file_id]) + + def readFirstHeader(self): + ''' + ''' - dirname, name = os.path.split(filename) # 'peru2' ---> Piura - 'peru1' ---> Huancayo or Porcuya - self.siteFile = name.split('.')[0] - if self.filename is not None: - self.fp.close() - self.filename = filename - self.fp = open(self.filename, 'rb') + self.siteFile = self.filename.split('/')[-1].split('.')[0] self.header_file = numpy.fromfile(self.fp, FILE_HEADER_STRUCTURE, 1) self.nrecords = self.header_file['nrec'][0] - self.sizeOfFile = os.path.getsize(self.filename) self.counter_records = 0 self.flagIsNewFile = 0 - self.fileIndex += 1 - - return 1 + self.fileIndex += 1 def readNextBlock(self): while True: - if self.counter_records == self.nrecords: + if not self.online and self.counter_records == self.nrecords: self.flagIsNewFile = 1 if not self.setNextFile(): return 0 - - self.readBlock() + try: + pointer = self.fp.tell() + self.readBlock() + except: + if self.online and self.waitDataBlock(pointer, 38512) == 1: + continue + else: + if not self.setNextFile(): + return 0 if (self.datatime < datetime.datetime.combine(self.startDate, self.startTime)) or \ (self.datatime > datetime.datetime.combine(self.endDate, self.endTime)): @@ -229,9 +204,8 @@ class BLTRParamReader(JRODataReader, ProcessingUnit): continue break - log.log('Reading Record No. {}/{} -> {}'.format( + log.log('Reading Record No. {} -> {}'.format( self.counter_records, - self.nrecords, self.datatime.ctime()), 'BLTRParamReader') return 1 @@ -240,13 +214,13 @@ class BLTRParamReader(JRODataReader, ProcessingUnit): pointer = self.fp.tell() header_rec = numpy.fromfile(self.fp, REC_HEADER_STRUCTURE, 1) - self.nchannels = header_rec['nchan'][0] / 2 + self.nchannels = int(header_rec['nchan'][0] / 2) self.kchan = header_rec['nrxs'][0] self.nmodes = header_rec['nmodes'][0] self.nranges = header_rec['nranges'][0] self.fp.seek(pointer) self.height = numpy.empty((self.nmodes, self.nranges)) - self.snr = numpy.empty((self.nmodes, self.nchannels, self.nranges)) + self.snr = numpy.empty((self.nmodes, int(self.nchannels), self.nranges)) self.buffer = numpy.empty((self.nmodes, 3, self.nranges)) self.flagDiscontinuousBlock = 0 @@ -268,9 +242,9 @@ class BLTRParamReader(JRODataReader, ProcessingUnit): header_structure = numpy.dtype( REC_HEADER_STRUCTURE.descr + [ - ('antenna_coord', 'f4', (2, self.nchannels)), - ('rx_gains', 'u4', (self.nchannels,)), - ('rx_analysis', 'u4', (self.nchannels,)) + ('antenna_coord', 'f4', (2, int(self.nchannels))), + ('rx_gains', 'u4', (int(self.nchannels),)), + ('rx_analysis', 'u4', (int(self.nchannels),)) ] ) @@ -290,12 +264,15 @@ class BLTRParamReader(JRODataReader, ProcessingUnit): def readData(self): ''' - Reading and filtering data block record of BLTR rawdata file, filtering is according to status_value. + Reading and filtering data block record of BLTR rawdata file, + filtering is according to status_value. Input: - status_value - Array data is set to NAN for values that are not equal to status_value + status_value - Array data is set to NAN for values that are not + equal to status_value ''' + self.nchannels = int(self.nchannels) data_structure = numpy.dtype( DATA_STRUCTURE.descr + [ @@ -334,13 +311,11 @@ class BLTRParamReader(JRODataReader, ProcessingUnit): self.dataOut.utctime = self.dataOut.utctimeInit self.dataOut.useLocalTime = False self.dataOut.paramInterval = 157 - self.dataOut.timezone = self.timezone self.dataOut.site = self.siteFile self.dataOut.nrecords = self.nrecords / self.nmodes - self.dataOut.sizeOfFile = self.sizeOfFile self.dataOut.lat = self.lat self.dataOut.lon = self.lon - self.dataOut.channelList = range(self.nchannels) + self.dataOut.channelList = list(range(self.nchannels)) self.dataOut.kchan = self.kchan self.dataOut.delta = self.delta self.dataOut.correction = self.correction @@ -357,7 +332,6 @@ class BLTRParamReader(JRODataReader, ProcessingUnit): ''' if self.flagNoMoreFiles: self.dataOut.flagNoData = True - log.success('No file left to process', 'BLTRParamReader') return 0 if not self.readNextBlock(): @@ -367,3 +341,15 @@ class BLTRParamReader(JRODataReader, ProcessingUnit): self.set_output() return 1 + + def run(self, **kwargs): + ''' + ''' + + if not(self.isConfig): + self.setup(**kwargs) + self.isConfig = True + + self.getData() + + return \ No newline at end of file diff --git a/schainpy/model/io/bltrIO_spectra.py b/schainpy/model/io/bltrIO_spectra.py new file mode 100644 index 0000000..4045f34 --- /dev/null +++ b/schainpy/model/io/bltrIO_spectra.py @@ -0,0 +1,462 @@ +import os +import sys +import glob +import fnmatch +import datetime +import time +import re +import h5py +import numpy + +import pylab as plb +from scipy.optimize import curve_fit +from scipy import asarray as ar, exp + +SPEED_OF_LIGHT = 299792458 +SPEED_OF_LIGHT = 3e8 + +from .utils import folder_in_range + +import schainpy.admin +from schainpy.model.data.jrodata import Spectra +from schainpy.model.proc.jroproc_base import ProcessingUnit, Operation, MPDecorator +from schainpy.utils import log +from schainpy.model.io.jroIO_base import JRODataReader + +def pol2cart(rho, phi): + x = rho * numpy.cos(phi) + y = rho * numpy.sin(phi) + return(x, y) + +FILE_STRUCTURE = numpy.dtype([ # HEADER 48bytes + ('FileMgcNumber', ' vertical) + ('BeamAngleZen', ' endFp: + sys.stderr.write( + "Warning %s: Size value read from System Header is lower than it has to be\n" % fp) + return 0 + + if OffRHeader < endFp: + sys.stderr.write( + "Warning %s: Size value read from System Header size is greater than it has to be\n" % fp) + return 0 + + return 1 + + +class BLTRSpectraReader (ProcessingUnit): + + def __init__(self): + + ProcessingUnit.__init__(self) + + self.ext = ".fdt" + self.optchar = "P" + self.fpFile = None + self.fp = None + self.BlockCounter = 0 + self.fileSizeByHeader = None + self.filenameList = [] + self.fileSelector = 0 + self.Off2StartNxtRec = 0 + self.RecCounter = 0 + self.flagNoMoreFiles = 0 + self.data_spc = None + self.data_cspc = None + self.path = None + self.OffsetStartHeader = 0 + self.Off2StartData = 0 + self.ipp = 0 + self.nFDTdataRecors = 0 + self.blocksize = 0 + self.dataOut = Spectra() + self.dataOut.flagNoData = False + + def search_files(self): + ''' + Function that indicates the number of .fdt files that exist in the folder to be read. + It also creates an organized list with the names of the files to read. + ''' + + files = glob.glob(os.path.join(self.path, '*{}'.format(self.ext))) + files = sorted(files) + for f in files: + filename = f.split('/')[-1] + if folder_in_range(filename.split('.')[1], self.startDate, self.endDate, '%Y%m%d'): + self.filenameList.append(f) + + def run(self, **kwargs): + ''' + This method will be the one that will initiate the data entry, will be called constantly. + You should first verify that your Setup () is set up and then continue to acquire + the data to be processed with getData (). + ''' + if not self.isConfig: + self.setup(**kwargs) + self.isConfig = True + + self.getData() + + def setup(self, + path=None, + startDate=None, + endDate=None, + startTime=None, + endTime=None, + walk=True, + code=None, + online=False, + mode=None, + **kwargs): + + self.isConfig = True + + self.path = path + self.startDate = startDate + self.endDate = endDate + self.startTime = startTime + self.endTime = endTime + self.walk = walk + self.mode = int(mode) + self.search_files() + if self.filenameList: + self.readFile() + + def getData(self): + ''' + Before starting this function, you should check that there is still an unread file, + If there are still blocks to read or if the data block is empty. + + You should call the file "read". + + ''' + + if self.flagNoMoreFiles: + self.dataOut.flagNoData = True + raise schainpy.admin.SchainError('No more files') + + self.readBlock() + + def readFile(self): + ''' + You must indicate if you are reading in Online or Offline mode and load the + The parameters for this file reading mode. + + Then you must do 2 actions: + + 1. Get the BLTR FileHeader. + 2. Start reading the first block. + ''' + + if self.fileSelector < len(self.filenameList): + log.success('Opening file: {}'.format(self.filenameList[self.fileSelector]), self.name) + self.fp = open(self.filenameList[self.fileSelector]) + self.fheader = FileHeaderBLTR(self.fp) + self.rheader = RecordHeaderBLTR(self.fp) + self.nFDTdataRecors = self.fheader.nFDTdataRecors + self.fileSelector += 1 + self.BlockCounter = 0 + return 1 + else: + self.flagNoMoreFiles = True + self.dataOut.flagNoData = True + return 0 + + def readBlock(self): + ''' + It should be checked if the block has data, if it is not passed to the next file. + + Then the following is done: + + 1. Read the RecordHeader + 2. Fill the buffer with the current block number. + + ''' + + if self.BlockCounter == self.nFDTdataRecors: + if not self.readFile(): + return + + if self.mode == 1: + self.rheader.read(self.BlockCounter+1) + elif self.mode == 0: + self.rheader.read(self.BlockCounter) + + self.RecCounter = self.rheader.RecCounter + self.OffsetStartHeader = self.rheader.OffsetStartHeader + self.Off2StartNxtRec = self.rheader.Off2StartNxtRec + self.Off2StartData = self.rheader.Off2StartData + self.nProfiles = self.rheader.nProfiles + self.nChannels = self.rheader.nChannels + self.nHeights = self.rheader.nHeights + self.frequency = self.rheader.TransmitFrec + self.DualModeIndex = self.rheader.DualModeIndex + self.pairsList = [(0, 1), (0, 2), (1, 2)] + self.dataOut.pairsList = self.pairsList + self.nRdPairs = len(self.dataOut.pairsList) + self.dataOut.nRdPairs = self.nRdPairs + self.dataOut.heightList = (self.rheader.StartRangeSamp + numpy.arange(self.nHeights) * self.rheader.SampResolution) / 1000. + self.dataOut.channelList = range(self.nChannels) + self.dataOut.nProfiles=self.rheader.nProfiles + self.dataOut.nIncohInt=self.rheader.nIncohInt + self.dataOut.nCohInt=self.rheader.nCohInt + self.dataOut.ippSeconds= 1/float(self.rheader.PRFhz) + self.dataOut.PRF=self.rheader.PRFhz + self.dataOut.nFFTPoints=self.rheader.nProfiles + self.dataOut.utctime = self.rheader.nUtime + self.rheader.nMilisec/1000. + self.dataOut.timeZone = 0 + self.dataOut.useLocalTime = False + self.dataOut.nmodes = 2 + log.log('Reading block {} - {}'.format(self.BlockCounter, self.dataOut.datatime), self.name) + OffDATA = self.OffsetStartHeader + self.RecCounter * \ + self.Off2StartNxtRec + self.Off2StartData + self.fp.seek(OffDATA, os.SEEK_SET) + + self.data_fft = numpy.fromfile(self.fp, [('complex','0: self.status = 1 self.dirnameList = dirnameList @@ -186,8 +186,8 @@ class AMISRReader(ProcessingUnit): startDateTime_Reader = datetime.datetime.combine(self.startDate,self.startTime) endDateTime_Reader = datetime.datetime.combine(self.endDate,self.endTime) - print 'Filtering Files from %s to %s'%(startDateTime_Reader, endDateTime_Reader) - print '........................................' + print('Filtering Files from %s to %s'%(startDateTime_Reader, endDateTime_Reader)) + print('........................................') filter_filenameList = [] self.filenameList.sort() for i in range(len(self.filenameList)-1): @@ -226,7 +226,7 @@ class AMISRReader(ProcessingUnit): def __getFilenameList(self, fileListInKeys, dirList): for value in fileListInKeys: - dirName = value.keys()[0] + dirName = list(value.keys())[0] for file in value[dirName]: filename = os.path.join(dirName, file) self.filenameList.append(filename) @@ -304,7 +304,7 @@ class AMISRReader(ProcessingUnit): self.__selectDataForTimes() for i in range(len(self.filenameList)): - print "%s" %(self.filenameList[i]) + print("%s" %(self.filenameList[i])) return @@ -315,7 +315,7 @@ class AMISRReader(ProcessingUnit): idFile += 1 if not(idFile < len(self.filenameList)): self.flagNoMoreFiles = 1 - print "No more Files" + print("No more Files") return 0 filename = self.filenameList[idFile] @@ -330,7 +330,7 @@ class AMISRReader(ProcessingUnit): self.amisrFilePointer = amisrFilePointer - print "Setting the file: %s"%self.filename + print("Setting the file: %s"%self.filename) return 1 @@ -341,7 +341,7 @@ class AMISRReader(ProcessingUnit): self.__selectDataForTimes(online=True) filename = self.filenameList[0] while self.__filename_online == filename: - print 'waiting %d seconds to get a new file...'%(self.__waitForNewFile) + print('waiting %d seconds to get a new file...'%(self.__waitForNewFile)) sleep(self.__waitForNewFile) self.__selectDataForTimes(online=True) filename = self.filenameList[0] @@ -351,7 +351,7 @@ class AMISRReader(ProcessingUnit): self.amisrFilePointer = h5py.File(filename,'r') self.flagIsNewFile = 1 self.filename = filename - print "Setting the file: %s"%self.filename + print("Setting the file: %s"%self.filename) return 1 @@ -368,12 +368,12 @@ class AMISRReader(ProcessingUnit): #looking index list for data start_index = self.radacHeaderObj.pulseCount[0,:][0] end_index = self.radacHeaderObj.npulses - range4data = range(start_index, end_index) + range4data = list(range(start_index, end_index)) self.index4_schain_datablock = numpy.array(range4data) buffer_start_index = 0 buffer_end_index = self.radacHeaderObj.pulseCount[0,:][0] - range4buffer = range(buffer_start_index, buffer_end_index) + range4buffer = list(range(buffer_start_index, buffer_end_index)) self.index4_buffer = numpy.array(range4buffer) self.linear_pulseCount = numpy.array(range4data + range4buffer) @@ -403,8 +403,8 @@ class AMISRReader(ProcessingUnit): just4record0 = self.radacHeaderObj.beamCodeByPulse[0,:] - for i in range(len(self.beamCodeDict.values())): - xx = numpy.where(just4record0==self.beamCodeDict.values()[i][0]) + for i in range(len(list(self.beamCodeDict.values()))): + xx = numpy.where(just4record0==list(self.beamCodeDict.values())[i][0]) indexPulseByBeam = self.linear_pulseCount[xx[0]] self.beamRangeDict[i] = indexPulseByBeam @@ -499,7 +499,7 @@ class AMISRReader(ProcessingUnit): self.searchFilesOnLine(path, walk) if not(self.filenameList): - print "There is no files into the folder: %s"%(path) + print("There is no files into the folder: %s"%(path)) sys.exit(-1) @@ -632,8 +632,8 @@ class AMISRReader(ProcessingUnit): return 0 def printUTC(self): - print self.dataOut.utctime - print '' + print(self.dataOut.utctime) + print('') def setObjProperties(self): @@ -661,7 +661,6 @@ class AMISRReader(ProcessingUnit): if self.flagNoMoreFiles: self.dataOut.flagNoData = True - print 'Process finished' return 0 if self.__hasNotDataInBuffer(): @@ -689,4 +688,4 @@ class AMISRReader(ProcessingUnit): self.setObjProperties() self.isConfig = True - self.getData() + self.getData() \ No newline at end of file diff --git a/schainpy/model/io/jroIO_base.py b/schainpy/model/io/jroIO_base.py index 790e5ac..729bd05 100644 --- a/schainpy/model/io/jroIO_base.py +++ b/schainpy/model/io/jroIO_base.py @@ -1,8 +1,8 @@ -''' +""" Created on Jul 2, 2014 @author: roj-idl71 -''' +""" import os import sys import glob @@ -12,21 +12,26 @@ import fnmatch import inspect import time import datetime -import traceback import zmq -try: - from gevent import sleep -except: - from time import sleep - -import schainpy.admin +from schainpy.model.proc.jroproc_base import Operation, MPDecorator from schainpy.model.data.jroheaderIO import PROCFLAG, BasicHeader, SystemHeader, RadarControllerHeader, ProcessingHeader from schainpy.model.data.jroheaderIO import get_dtype_index, get_numpy_dtype, get_procflag_dtype, get_dtype_width from schainpy.utils import log import schainpy.admin LOCALTIME = True +DT_DIRECTIVES = { + '%Y': 4, + '%y': 2, + '%m': 2, + '%d': 2, + '%j': 3, + '%H': 2, + '%M': 2, + '%S': 2, + '%f': 6 +} def isNumber(cad): @@ -75,14 +80,14 @@ def isFileInEpoch(filename, startUTSeconds, endUTSeconds): try: fp = open(filename, 'rb') except IOError: - print "The file %s can't be opened" % (filename) + print("The file %s can't be opened" % (filename)) return 0 sts = basicHeaderObj.read(fp) fp.close() if not(sts): - print "Skipping the file %s because it has not a valid header" % (filename) + print("Skipping the file %s because it has not a valid header" % (filename)) return 0 if not ((startUTSeconds <= basicHeaderObj.utc) and (endUTSeconds > basicHeaderObj.utc)): @@ -130,7 +135,7 @@ def isFileInTimeRange(filename, startDate, endDate, startTime, endTime): try: fp = open(filename, 'rb') except IOError: - print "The file %s can't be opened" % (filename) + print("The file %s can't be opened" % (filename)) return None firstBasicHeaderObj = BasicHeader(LOCALTIME) @@ -143,7 +148,7 @@ def isFileInTimeRange(filename, startDate, endDate, startTime, endTime): sts = firstBasicHeaderObj.read(fp) if not(sts): - print "[Reading] Skipping the file %s because it has not a valid header" % (filename) + print("[Reading] Skipping the file %s because it has not a valid header" % (filename)) return None if not systemHeaderObj.read(fp): @@ -160,7 +165,7 @@ def isFileInTimeRange(filename, startDate, endDate, startTime, endTime): offset = processingHeaderObj.blockSize + 24 # header size if filesize <= offset: - print "[Reading] %s: This file has not enough data" % filename + print("[Reading] %s: This file has not enough data" % filename) return None fp.seek(-offset, 2) @@ -231,7 +236,7 @@ def isFolderInDateRange(folder, startDate=None, endDate=None): basename = os.path.basename(folder) if not isRadarFolder(basename): - print "The folder %s has not the rigth format" % folder + print("The folder %s has not the rigth format" % folder) return 0 if startDate and endDate: @@ -274,7 +279,7 @@ def isFileInDateRange(filename, startDate=None, endDate=None): basename = os.path.basename(filename) if not isRadarFile(basename): - print "The filename %s has not the rigth format" % filename + print("The filename %s has not the rigth format" % filename) return 0 if startDate and endDate: @@ -315,8 +320,8 @@ def getFileFromSet(path, ext, set): return myfile[0] else: filename = '*%4.4d%3.3d%3.3d%s' % (year, doy, set, ext.lower()) - print 'the filename %s does not exist' % filename - print '...going to the last file: ' + print('the filename %s does not exist' % filename) + print('...going to the last file: ') if validFilelist: validFilelist = sorted(validFilelist, key=str.lower) @@ -368,70 +373,6 @@ def getlastFileFromPath(path, ext): return None -def checkForRealPath(path, foldercounter, year, doy, set, ext): - """ - Por ser Linux Case Sensitive entonces checkForRealPath encuentra el nombre correcto de un path, - Prueba por varias combinaciones de nombres entre mayusculas y minusculas para determinar - el path exacto de un determinado file. - - Example : - nombre correcto del file es .../.../D2009307/P2009307367.ext - - Entonces la funcion prueba con las siguientes combinaciones - .../.../y2009307367.ext - .../.../Y2009307367.ext - .../.../x2009307/y2009307367.ext - .../.../x2009307/Y2009307367.ext - .../.../X2009307/y2009307367.ext - .../.../X2009307/Y2009307367.ext - siendo para este caso, la ultima combinacion de letras, identica al file buscado - - Return: - Si encuentra la cobinacion adecuada devuelve el path completo y el nombre del file - caso contrario devuelve None como path y el la ultima combinacion de nombre en mayusculas - para el filename - """ - fullfilename = None - find_flag = False - filename = None - - prefixDirList = [None, 'd', 'D'] - if ext.lower() == ".r": # voltage - prefixFileList = ['d', 'D'] - elif ext.lower() == ".pdata": # spectra - prefixFileList = ['p', 'P'] - else: - return None, filename - - # barrido por las combinaciones posibles - for prefixDir in prefixDirList: - thispath = path - if prefixDir != None: - # formo el nombre del directorio xYYYYDDD (x=d o x=D) - if foldercounter == 0: - thispath = os.path.join(path, "%s%04d%03d" % - (prefixDir, year, doy)) - else: - thispath = os.path.join(path, "%s%04d%03d_%02d" % ( - prefixDir, year, doy, foldercounter)) - for prefixFile in prefixFileList: # barrido por las dos combinaciones posibles de "D" - # formo el nombre del file xYYYYDDDSSS.ext - filename = "%s%04d%03d%03d%s" % (prefixFile, year, doy, set, ext) - fullfilename = os.path.join( - thispath, filename) # formo el path completo - - if os.path.exists(fullfilename): # verifico que exista - find_flag = True - break - if find_flag: - break - - if not(find_flag): - return None, filename - - return fullfilename, filename - - def isRadarFolder(folder): try: year = int(folder[1:5]) @@ -475,85 +416,75 @@ def getDateFromRadarFolder(folder): thisDate = datetime.date(year, 1, 1) + datetime.timedelta(doy - 1) return thisDate +def parse_format(s, fmt): + + for i in range(fmt.count('%')): + x = fmt.index('%') + d = DT_DIRECTIVES[fmt[x:x+2]] + fmt = fmt.replace(fmt[x:x+2], s[x:x+d]) + return fmt -class JRODataIO: +class Reader(object): c = 3E8 - isConfig = False - - basicHeaderObj = None - - systemHeaderObj = None - - radarControllerHeaderObj = None - - processingHeaderObj = None - dtype = None - pathList = [] - filenameList = [] - + datetimeList = [] filename = None - ext = None - flagIsNewFile = 1 - flagDiscontinuousBlock = 0 - flagIsNewBlock = 0 - + flagNoMoreFiles = 0 fp = None - firstHeaderSize = 0 - basicHeaderSize = 24 - versionFile = 1103 - fileSize = None - -# ippSeconds = None - fileSizeByHeader = None - - fileIndex = None - + fileIndex = -1 profileIndex = None - - blockIndex = None - - nTotalBlocks = None - + blockIndex = 0 + nTotalBlocks = 0 maxTimeStep = 30 - lastUTTime = None - datablock = None - dataOut = None - - blocksize = None - getByBlock = False - - def __init__(self): - - raise NotImplementedError + path = None + startDate = None + endDate = None + startTime = datetime.time(0, 0, 0) + endTime = datetime.time(23, 59, 59) + set = None + expLabel = "" + online = False + delay = 60 + nTries = 3 # quantity tries + nFiles = 3 # number of files for searching + walk = True + getblock = False + nTxs = 1 + realtime = False + blocksize = 0 + blocktime = None + warnings = True + verbose = True + server = None + format = None + oneDDict = None + twoDDict = None + independentParam = None + filefmt = None + folderfmt = None + open_file = open + open_mode = 'rb' def run(self): - raise NotImplementedError - - def getDtypeWidth(self): - - dtype_index = get_dtype_index(self.dtype) - dtype_width = get_dtype_width(dtype_index) - - return dtype_width + raise NotImplementedError def getAllowedArgs(self): if hasattr(self, '__attrs__'): @@ -561,353 +492,333 @@ class JRODataIO: else: return inspect.getargspec(self.run).args + def set_kwargs(self, **kwargs): -class JRODataReader(JRODataIO): + for key, value in kwargs.items(): + setattr(self, key, value) + + def find_folders(self, path, startDate, endDate, folderfmt, last=False): - online = 0 + folders = [x for f in path.split(',') + for x in os.listdir(f) if os.path.isdir(os.path.join(f, x))] + folders.sort() - realtime = 0 + if last: + folders = [folders[-1]] - nReadBlocks = 0 - - delay = 10 # number of seconds waiting a new file - - nTries = 3 # quantity tries - - nFiles = 3 # number of files for searching - - path = None - - foldercounter = 0 - - flagNoMoreFiles = 0 - - datetimeList = [] - - __isFirstTimeOnline = 1 - - __printInfo = True - - profileIndex = None - - nTxs = 1 - - txIndex = None - - # Added-------------------- - - selBlocksize = None - - selBlocktime = None + for folder in folders: + try: + dt = datetime.datetime.strptime(parse_format(folder, folderfmt), folderfmt).date() + if dt >= startDate and dt <= endDate: + yield os.path.join(path, folder) + else: + log.log('Skiping folder {}'.format(folder), self.name) + except Exception as e: + log.log('Skiping folder {}'.format(folder), self.name) + continue + return + + def find_files(self, folders, ext, filefmt, startDate=None, endDate=None, + expLabel='', last=False): + + for path in folders: + files = glob.glob1(path, '*{}'.format(ext)) + files.sort() + if last: + if files: + fo = files[-1] + try: + dt = datetime.datetime.strptime(parse_format(fo, filefmt), filefmt).date() + yield os.path.join(path, expLabel, fo) + except Exception as e: + pass + return + else: + return - def __init__(self): - """ - This class is used to find data files + for fo in files: + try: + dt = datetime.datetime.strptime(parse_format(fo, filefmt), filefmt).date() + if dt >= startDate and dt <= endDate: + yield os.path.join(path, expLabel, fo) + else: + log.log('Skiping file {}'.format(fo), self.name) + except Exception as e: + log.log('Skiping file {}'.format(fo), self.name) + continue - Example: - reader = JRODataReader() - fileList = reader.findDataFiles() + def searchFilesOffLine(self, path, startDate, endDate, + expLabel, ext, walk, + filefmt, folderfmt): + """Search files in offline mode for the given arguments + Return: + Generator of files """ - pass - def createObjByDefault(self): - """ + if walk: + folders = self.find_folders( + path, startDate, endDate, folderfmt) + else: + folders = path.split(',') + + return self.find_files( + folders, ext, filefmt, startDate, endDate, expLabel) + + def searchFilesOnLine(self, path, startDate, endDate, + expLabel, ext, walk, + filefmt, folderfmt): + """Search for the last file of the last folder + + Arguments: + path : carpeta donde estan contenidos los files que contiene data + expLabel : Nombre del subexperimento (subfolder) + ext : extension de los files + walk : Si es habilitado no realiza busquedas dentro de los ubdirectorios (doypath) + Return: + generator with the full path of last filename """ - raise NotImplementedError - - def getBlockDimension(self): - - raise NotImplementedError - - def searchFilesOffLine(self, - path, - startDate=None, - endDate=None, - startTime=datetime.time(0, 0, 0), - endTime=datetime.time(23, 59, 59), - set=None, - expLabel='', - ext='.r', - cursor=None, - skip=None, - walk=True): - - self.filenameList = [] - self.datetimeList = [] - - pathList = [] - - dateList, pathList = self.findDatafiles( - path, startDate, endDate, expLabel, ext, walk, include_path=True) - - if dateList == []: - return [], [] - - if len(dateList) > 1: - print "[Reading] Data found for date range [%s - %s]: total days = %d" % (startDate, endDate, len(dateList)) + + if walk: + folders = self.find_folders( + path, startDate, endDate, folderfmt, last=True) else: - print "[Reading] Data found for date range [%s - %s]: date = %s" % (startDate, endDate, dateList[0]) - - filenameList = [] - datetimeList = [] + folders = path.split(',') - for thisPath in pathList: - - fileList = glob.glob1(thisPath, "*%s" % ext) - fileList.sort() - - for file in fileList: - - filename = os.path.join(thisPath, file) - - if not isFileInDateRange(filename, startDate, endDate): - continue - - thisDatetime = isFileInTimeRange( - filename, startDate, endDate, startTime, endTime) - - if not(thisDatetime): - continue - - filenameList.append(filename) - datetimeList.append(thisDatetime) + return self.find_files( + folders, ext, filefmt, startDate, endDate, expLabel, last=True) - if cursor is not None and skip is not None: - filenameList = filenameList[cursor * skip:cursor * skip + skip] - datetimeList = datetimeList[cursor * skip:cursor * skip + skip] + def setNextFile(self): + """Set the next file to be readed open it and parse de file header""" - if not(filenameList): - print "[Reading] Time range selected invalid [%s - %s]: No *%s files in %s)" % (startTime, endTime, ext, path) - return [], [] + while True: + if self.fp != None: + self.fp.close() - print "[Reading] %d file(s) was(were) found in time range: %s - %s" % (len(filenameList), startTime, endTime) + if self.online: + newFile = self.setNextFileOnline() + else: + newFile = self.setNextFileOffline() + + if not(newFile): + if self.online: + raise schainpy.admin.SchainError('Time to wait for new files reach') + else: + if self.fileIndex == -1: + raise schainpy.admin.SchainWarning('No files found in the given path') + else: + raise schainpy.admin.SchainWarning('No more files to read') + + if self.verifyFile(self.filename): + break + + log.log('Opening file: %s' % self.filename, self.name) - # for i in range(len(filenameList)): - # print "[Reading] %s -> [%s]" %(filenameList[i], datetimeList[i].ctime()) + self.readFirstHeader() + self.nReadBlocks = 0 - self.filenameList = filenameList - self.datetimeList = datetimeList + def setNextFileOnline(self): + """Check for the next file to be readed in online mode. - return pathList, filenameList + Set: + self.filename + self.fp + self.filesize + + Return: + boolean - def __searchFilesOnLine(self, path, expLabel="", ext=None, walk=True, set=None): """ - Busca el ultimo archivo de la ultima carpeta (determinada o no por startDateTime) y - devuelve el archivo encontrado ademas de otros datos. + nextFile = True + nextDay = False - Input: - path : carpeta donde estan contenidos los files que contiene data + for nFiles in range(self.nFiles+1): + for nTries in range(self.nTries): + fullfilename, filename = self.checkForRealPath(nextFile, nextDay) + if fullfilename is not None: + break + log.warning( + "Waiting %0.2f sec for the next file: \"%s\" , try %02d ..." % (self.delay, filename, nTries + 1), + self.name) + time.sleep(self.delay) + nextFile = False + continue + + if fullfilename is not None: + break + + self.nTries = 1 + nextFile = True - expLabel : Nombre del subexperimento (subfolder) + if nFiles == (self.nFiles - 1): + log.log('Trying with next day...', self.name) + nextDay = True + self.nTries = 3 - ext : extension de los files + if fullfilename: + self.fileSize = os.path.getsize(fullfilename) + self.filename = fullfilename + self.flagIsNewFile = 1 + if self.fp != None: + self.fp.close() + self.fp = self.open_file(fullfilename, self.open_mode) + self.flagNoMoreFiles = 0 + self.fileIndex += 1 + return 1 + else: + return 0 + + def setNextFileOffline(self): + """Open the next file to be readed in offline mode""" + + try: + filename = next(self.filenameList) + self.fileIndex +=1 + except StopIteration: + self.flagNoMoreFiles = 1 + return 0 - walk : Si es habilitado no realiza busquedas dentro de los ubdirectorios (doypath) + self.filename = filename + self.fileSize = os.path.getsize(filename) + self.fp = self.open_file(filename, self.open_mode) + self.flagIsNewFile = 1 + return 1 + + @staticmethod + def isDateTimeInRange(dt, startDate, endDate, startTime, endTime): + """Check if the given datetime is in range""" + + if startDate <= dt.date() <= endDate: + if startTime <= dt.time() <= endTime: + return True + return False + + def verifyFile(self, filename): + """Check for a valid file + + Arguments: + filename -- full path filename + Return: - directory : eL directorio donde esta el file encontrado - filename : el ultimo file de una determinada carpeta - year : el anho - doy : el numero de dia del anho - set : el set del archivo - - + boolean """ - if not os.path.isdir(path): - return None, None, None, None, None, None - dirList = [] - - if not walk: - fullpath = path - foldercounter = 0 - else: - # Filtra solo los directorios - for thisPath in os.listdir(path): - if not os.path.isdir(os.path.join(path, thisPath)): - continue - if not isRadarFolder(thisPath): - continue - - dirList.append(thisPath) + return True - if not(dirList): - return None, None, None, None, None, None + def checkForRealPath(self, nextFile, nextDay): + """Check if the next file to be readed exists""" - dirList = sorted(dirList, key=str.lower) + raise NotImplementedError + + def readFirstHeader(self): + """Parse the file header""" - doypath = dirList[-1] - foldercounter = int(doypath.split('_')[1]) if len( - doypath.split('_')) > 1 else 0 - fullpath = os.path.join(path, doypath, expLabel) + pass - print "[Reading] %s folder was found: " % (fullpath) + def waitDataBlock(self, pointer_location, blocksize=None): + """ + """ - if set == None: - filename = getlastFileFromPath(fullpath, ext) + currentPointer = pointer_location + if blocksize is None: + neededSize = self.processingHeaderObj.blockSize # + self.basicHeaderSize else: - filename = getFileFromSet(fullpath, ext, set) + neededSize = blocksize - if not(filename): - return None, None, None, None, None, None - - print "[Reading] %s file was found" % (filename) - - if not(self.__verifyFile(os.path.join(fullpath, filename))): - return None, None, None, None, None, None - - year = int(filename[1:5]) - doy = int(filename[5:8]) - set = int(filename[8:11]) + for nTries in range(self.nTries): + self.fp.close() + self.fp = open(self.filename, 'rb') + self.fp.seek(currentPointer) - return fullpath, foldercounter, filename, year, doy, set + self.fileSize = os.path.getsize(self.filename) + currentSize = self.fileSize - currentPointer - def __setNextFileOffline(self): + if (currentSize >= neededSize): + return 1 - idFile = self.fileIndex + log.warning( + "Waiting %0.2f seconds for the next block, try %03d ..." % (self.delay, nTries + 1), + self.name + ) + time.sleep(self.delay) - while (True): - idFile += 1 - if not(idFile < len(self.filenameList)): - self.flagNoMoreFiles = 1 -# print "[Reading] No more Files" - return 0 + return 0 - filename = self.filenameList[idFile] +class JRODataReader(Reader): - if not(self.__verifyFile(filename)): - continue + utc = 0 + nReadBlocks = 0 + foldercounter = 0 + firstHeaderSize = 0 + basicHeaderSize = 24 + __isFirstTimeOnline = 1 + filefmt = "*%Y%j***" + folderfmt = "*%Y%j" + __attrs__ = ['path', 'startDate', 'endDate', 'startTime', 'endTime', 'online', 'delay', 'walk'] - fileSize = os.path.getsize(filename) - fp = open(filename, 'rb') - break + def getDtypeWidth(self): - self.flagIsNewFile = 1 - self.fileIndex = idFile - self.filename = filename - self.fileSize = fileSize - self.fp = fp + dtype_index = get_dtype_index(self.dtype) + dtype_width = get_dtype_width(dtype_index) -# print "[Reading] Setting the file: %s"%self.filename + return dtype_width - return 1 + def checkForRealPath(self, nextFile, nextDay): + """Check if the next file to be readed exists. - def __setNextFileOnline(self): - """ - Busca el siguiente file que tenga suficiente data para ser leida, dentro de un folder especifico, si - no encuentra un file valido espera un tiempo determinado y luego busca en los posibles n files - siguientes. + Example : + nombre correcto del file es .../.../D2009307/P2009307367.ext - Affected: - self.flagIsNewFile - self.filename - self.fileSize - self.fp - self.set - self.flagNoMoreFiles + Entonces la funcion prueba con las siguientes combinaciones + .../.../y2009307367.ext + .../.../Y2009307367.ext + .../.../x2009307/y2009307367.ext + .../.../x2009307/Y2009307367.ext + .../.../X2009307/y2009307367.ext + .../.../X2009307/Y2009307367.ext + siendo para este caso, la ultima combinacion de letras, identica al file buscado Return: - 0 : si luego de una busqueda del siguiente file valido este no pudo ser encontrado - 1 : si el file fue abierto con exito y esta listo a ser leido - - Excepciones: - Si un determinado file no puede ser abierto + str -- fullpath of the file """ - nFiles = 0 - fileOk_flag = False - firstTime_flag = True - - self.set += 1 - - if self.set > 999: + + + if nextFile: + self.set += 1 + if nextDay: self.set = 0 - self.foldercounter += 1 - - # busca el 1er file disponible - fullfilename, filename = checkForRealPath( - self.path, self.foldercounter, self.year, self.doy, self.set, self.ext) - if fullfilename: - if self.__verifyFile(fullfilename, False): - fileOk_flag = True - - # si no encuentra un file entonces espera y vuelve a buscar - if not(fileOk_flag): - # busco en los siguientes self.nFiles+1 files posibles - for nFiles in range(self.nFiles + 1): - - if firstTime_flag: # si es la 1era vez entonces hace el for self.nTries veces - tries = self.nTries + self.doy += 1 + foldercounter = 0 + prefixDirList = [None, 'd', 'D'] + if self.ext.lower() == ".r": # voltage + prefixFileList = ['d', 'D'] + elif self.ext.lower() == ".pdata": # spectra + prefixFileList = ['p', 'P'] + + # barrido por las combinaciones posibles + for prefixDir in prefixDirList: + thispath = self.path + if prefixDir != None: + # formo el nombre del directorio xYYYYDDD (x=d o x=D) + if foldercounter == 0: + thispath = os.path.join(self.path, "%s%04d%03d" % + (prefixDir, self.year, self.doy)) else: - tries = 1 # si no es la 1era vez entonces solo lo hace una vez - - for nTries in range(tries): - if firstTime_flag: - print "\t[Reading] Waiting %0.2f sec for the next file: \"%s\" , try %03d ..." % (self.delay, filename, nTries + 1) - sleep(self.delay) - else: - print "\t[Reading] Searching the next \"%s%04d%03d%03d%s\" file ..." % (self.optchar, self.year, self.doy, self.set, self.ext) - - fullfilename, filename = checkForRealPath( - self.path, self.foldercounter, self.year, self.doy, self.set, self.ext) - if fullfilename: - if self.__verifyFile(fullfilename): - fileOk_flag = True - break - - if fileOk_flag: - break - - firstTime_flag = False - - log.warning('Skipping the file {} due to this file doesn\'t exist'.format(filename)) - self.set += 1 - - # si no encuentro el file buscado cambio de carpeta y busco en la siguiente carpeta - if nFiles == (self.nFiles - 1): - self.set = 0 - self.doy += 1 - self.foldercounter = 0 - - if fileOk_flag: - self.fileSize = os.path.getsize(fullfilename) - self.filename = fullfilename - self.flagIsNewFile = 1 - if self.fp != None: - self.fp.close() - self.fp = open(fullfilename, 'rb') - self.flagNoMoreFiles = 0 -# print '[Reading] Setting the file: %s' % fullfilename - else: - self.fileSize = 0 - self.filename = None - self.flagIsNewFile = 0 - self.fp = None - self.flagNoMoreFiles = 1 - - return fileOk_flag - - def setNextFile(self): - if self.fp != None: - self.fp.close() - - if self.online: - newFile = self.__setNextFileOnline() - else: - newFile = self.__setNextFileOffline() - - if not(newFile): - raise schainpy.admin.SchainWarning('No more files to read') - return 0 - - if self.verbose: - print '[Reading] Setting the file: %s' % self.filename - - self.__readFirstHeader() - self.nReadBlocks = 0 - return 1 - + thispath = os.path.join(self.path, "%s%04d%03d_%02d" % ( + prefixDir, self.year, self.doy, foldercounter)) + for prefixFile in prefixFileList: # barrido por las dos combinaciones posibles de "D" + # formo el nombre del file xYYYYDDDSSS.ext + filename = "%s%04d%03d%03d%s" % (prefixFile, self.year, self.doy, self.set, self.ext) + fullfilename = os.path.join( + thispath, filename) + + if os.path.exists(fullfilename): + return fullfilename, filename + + return None, filename + def __waitNewBlock(self): """ Return 1 si se encontro un nuevo bloque de datos, 0 de otra forma. @@ -941,75 +852,17 @@ class JRODataReader(JRODataIO): # self.flagEoF = True return 0 - print "[Reading] Waiting %0.2f seconds for the next block, try %03d ..." % (self.delay, nTries + 1) - sleep(self.delay) + print("[Reading] Waiting %0.2f seconds for the next block, try %03d ..." % (self.delay, nTries + 1)) + time.sleep(self.delay) return 0 - def waitDataBlock(self, pointer_location): - - currentPointer = pointer_location - - neededSize = self.processingHeaderObj.blockSize # + self.basicHeaderSize - - for nTries in range(self.nTries): - self.fp.close() - self.fp = open(self.filename, 'rb') - self.fp.seek(currentPointer) - - self.fileSize = os.path.getsize(self.filename) - currentSize = self.fileSize - currentPointer - - if (currentSize >= neededSize): - return 1 - - print "[Reading] Waiting %0.2f seconds for the next block, try %03d ..." % (self.delay, nTries + 1) - sleep(self.delay) - - return 0 - - def __jumpToLastBlock(self): - - if not(self.__isFirstTimeOnline): - return - - csize = self.fileSize - self.fp.tell() - blocksize = self.processingHeaderObj.blockSize - - # salta el primer bloque de datos - if csize > self.processingHeaderObj.blockSize: - self.fp.seek(self.fp.tell() + blocksize) - else: - return - - csize = self.fileSize - self.fp.tell() - neededsize = self.processingHeaderObj.blockSize + self.basicHeaderSize - while True: - - if self.fp.tell() < self.fileSize: - self.fp.seek(self.fp.tell() + neededsize) - else: - self.fp.seek(self.fp.tell() - neededsize) - break - -# csize = self.fileSize - self.fp.tell() -# neededsize = self.processingHeaderObj.blockSize + self.basicHeaderSize -# factor = int(csize/neededsize) -# if factor > 0: -# self.fp.seek(self.fp.tell() + factor*neededsize) - - self.flagIsNewFile = 0 - self.__isFirstTimeOnline = 0 - def __setNewBlock(self): - # if self.server is None: - if self.fp == None: - return 0 - -# if self.online: -# self.__jumpToLastBlock() - if self.flagIsNewFile: + if self.fp == None: + return 0 + + if self.flagIsNewFile: self.lastUTTime = self.basicHeaderObj.utc return 1 @@ -1019,21 +872,19 @@ class JRODataReader(JRODataIO): return 0 else: return 1 - # if self.server is None: + currentSize = self.fileSize - self.fp.tell() neededSize = self.processingHeaderObj.blockSize + self.basicHeaderSize + if (currentSize >= neededSize): self.basicHeaderObj.read(self.fp) self.lastUTTime = self.basicHeaderObj.utc return 1 - # else: - # self.basicHeaderObj.read(self.zHeader) - # self.lastUTTime = self.basicHeaderObj.utc - # return 1 + if self.__waitNewBlock(): self.lastUTTime = self.basicHeaderObj.utc return 1 - # if self.server is None: + if not(self.setNextFile()): return 0 @@ -1049,37 +900,35 @@ class JRODataReader(JRODataIO): def readNextBlock(self): - # Skip block out of startTime and endTime while True: if not(self.__setNewBlock()): - raise(schainpy.admin.SchainWarning('No more files')) - return 0 + continue if not(self.readBlock()): return 0 self.getBasicHeader() - if (self.dataOut.datatime < datetime.datetime.combine(self.startDate, self.startTime)) or (self.dataOut.datatime > datetime.datetime.combine(self.endDate, self.endTime)): - print "[Reading] Block No. %d/%d -> %s [Skipping]" % (self.nReadBlocks, + + if not self.isDateTimeInRange(self.dataOut.datatime, self.startDate, self.endDate, self.startTime, self.endTime): + print("[Reading] Block No. %d/%d -> %s [Skipping]" % (self.nReadBlocks, self.processingHeaderObj.dataBlocksPerFile, - self.dataOut.datatime.ctime()) + self.dataOut.datatime.ctime())) continue break if self.verbose: - print "[Reading] Block No. %d/%d -> %s" % (self.nReadBlocks, + print("[Reading] Block No. %d/%d -> %s" % (self.nReadBlocks, self.processingHeaderObj.dataBlocksPerFile, - self.dataOut.datatime.ctime()) + self.dataOut.datatime.ctime())) return 1 - def __readFirstHeader(self): + def readFirstHeader(self): self.basicHeaderObj.read(self.fp) self.systemHeaderObj.read(self.fp) self.radarControllerHeaderObj.read(self.fp) self.processingHeaderObj.read(self.fp) - self.firstHeaderSize = self.basicHeaderObj.size datatype = int(numpy.log2((self.processingHeaderObj.processFlags & @@ -1097,7 +946,7 @@ class JRODataReader(JRODataIO): elif datatype == 5: datatype_str = numpy.dtype([('real', ' %s" %(self.nReadBlocks, -# self.processingHeaderObj.dataBlocksPerFile, -# self.dataOut.datatime.ctime()) - - def printInfo(self): - - if self.__printInfo == False: - return - - self.basicHeaderObj.printInfo() - self.systemHeaderObj.printInfo() - self.radarControllerHeaderObj.printInfo() - self.processingHeaderObj.printInfo() + print("[Reading] Number of read blocks %04d" % self.nTotalBlocks) - self.__printInfo = False + def run(self, **kwargs): + """ - def run(self, - path=None, - startDate=None, - endDate=None, - startTime=datetime.time(0, 0, 0), - endTime=datetime.time(23, 59, 59), - set=None, - expLabel="", - ext=None, - online=False, - delay=60, - walk=True, - getblock=False, - nTxs=1, - realtime=False, - blocksize=None, - blocktime=None, - skip=None, - cursor=None, - warnings=True, - server=None, - verbose=True, - format=None, - oneDDict=None, - twoDDict=None, - ind2DList=None, **kwargs): + Arguments: + path : + startDate : + endDate : + startTime : + endTime : + set : + expLabel : + ext : + online : + delay : + walk : + getblock : + nTxs : + realtime : + blocksize : + blocktime : + skip : + cursor : + warnings : + server : + verbose : + format : + oneDDict : + twoDDict : + independentParam : + """ if not(self.isConfig): - self.setup(path=path, - startDate=startDate, - endDate=endDate, - startTime=startTime, - endTime=endTime, - set=set, - expLabel=expLabel, - ext=ext, - online=online, - delay=delay, - walk=walk, - getblock=getblock, - nTxs=nTxs, - realtime=realtime, - blocksize=blocksize, - blocktime=blocktime, - skip=skip, - cursor=cursor, - warnings=warnings, - server=server, - verbose=verbose, - format=format, - oneDDict=oneDDict, - twoDDict=twoDDict, - ind2DList=ind2DList) + self.setup(**kwargs) self.isConfig = True - if server is None: + if self.server is None: self.getData() else: self.getFromServer() -class JRODataWriter(JRODataIO): +class JRODataWriter(Reader): """ Esta clase permite escribir datos a archivos procesados (.r o ,pdata). La escritura de los datos siempre se realiza por bloques. """ - blockIndex = 0 - - path = None - setFile = None - profilesPerBlock = None - blocksPerFile = None - nWriteBlocks = 0 - fileDate = None def __init__(self, dataOut=None): @@ -1556,6 +1280,13 @@ class JRODataWriter(JRODataIO): def putData(self): raise NotImplementedError + def getDtypeWidth(self): + + dtype_index = get_dtype_index(self.dtype) + dtype_width = get_dtype_width(dtype_index) + + return dtype_width + def getProcessFlags(self): processFlags = 0 @@ -1593,11 +1324,9 @@ class JRODataWriter(JRODataIO): self.basicHeaderObj.size = self.basicHeaderSize # bytes self.basicHeaderObj.version = self.versionFile - self.basicHeaderObj.dataBlock = self.nTotalBlocks - + self.basicHeaderObj.dataBlock = self.nTotalBlocks utc = numpy.floor(self.dataOut.utctime) - milisecond = (self.dataOut.utctime - utc) * 1000.0 - + milisecond = (self.dataOut.utctime - utc) * 1000.0 self.basicHeaderObj.utc = utc self.basicHeaderObj.miliSecond = milisecond self.basicHeaderObj.timeZone = self.dataOut.timeZone @@ -1679,14 +1408,13 @@ class JRODataWriter(JRODataIO): self.writeBlock() - print "[Writing] Block No. %d/%d" % (self.blockIndex, - self.processingHeaderObj.dataBlocksPerFile) + print("[Writing] Block No. %d/%d" % (self.blockIndex, + self.processingHeaderObj.dataBlocksPerFile)) return 1 def setNextFile(self): - """ - Determina el siguiente file que sera escrito + """Determina el siguiente file que sera escrito Affected: self.filename @@ -1705,6 +1433,9 @@ class JRODataWriter(JRODataIO): if self.fp != None: self.fp.close() + if not os.path.exists(path): + os.mkdir(path) + timeTuple = time.localtime(self.dataOut.utctime) subfolder = 'd%4.4d%3.3d' % (timeTuple.tm_year, timeTuple.tm_yday) @@ -1745,18 +1476,15 @@ class JRODataWriter(JRODataIO): fp = open(filename, 'wb') self.blockIndex = 0 - - # guardando atributos self.filename = filename self.subfolder = subfolder self.fp = fp self.setFile = setFile self.flagIsNewFile = 1 self.fileDate = self.dataOut.datatime.date() - self.setFirstHeader() - print '[Writing] Opening file: %s' % self.filename + print('[Writing] Opening file: %s' % self.filename) self.__writeFirstHeader() @@ -1796,26 +1524,23 @@ class JRODataWriter(JRODataIO): self.setFile = set - 1 self.blocksPerFile = blocksPerFile - self.profilesPerBlock = profilesPerBlock - self.dataOut = dataOut self.fileDate = self.dataOut.datatime.date() - # By default self.dtype = self.dataOut.dtype if datatype is not None: self.dtype = get_numpy_dtype(datatype) if not(self.setNextFile()): - print "[Writing] There isn't a next file" + print("[Writing] There isn't a next file") return 0 self.setBlockDimension() return 1 - def run(self, dataOut, path, blocksPerFile, profilesPerBlock=64, set=None, ext=None, datatype=4, **kwargs): + def run(self, dataOut, path, blocksPerFile=100, profilesPerBlock=64, set=None, ext=None, datatype=4, **kwargs): if not(self.isConfig): @@ -1823,4 +1548,30 @@ class JRODataWriter(JRODataIO): set=set, ext=ext, datatype=datatype, **kwargs) self.isConfig = True + self.dataOut = dataOut self.putData() + return self.dataOut + +@MPDecorator +class printInfo(Operation): + + def __init__(self): + + Operation.__init__(self) + self.__printInfo = True + + def run(self, dataOut, headers = ['systemHeaderObj', 'radarControllerHeaderObj', 'processingHeaderObj']): + if self.__printInfo == False: + return + + for header in headers: + if hasattr(dataOut, header): + obj = getattr(dataOut, header) + if hasattr(obj, 'printInfo'): + obj.printInfo() + else: + print(obj) + else: + log.warning('Header {} Not found in object'.format(header)) + + self.__printInfo = False diff --git a/schainpy/model/io/jroIO_bltr.py b/schainpy/model/io/jroIO_bltr.py deleted file mode 100644 index 6a1c6a5..0000000 --- a/schainpy/model/io/jroIO_bltr.py +++ /dev/null @@ -1,1180 +0,0 @@ -import os -import sys -import glob -import fnmatch -import datetime -import time -import re -import h5py -import numpy - -import pylab as plb -from scipy.optimize import curve_fit -from scipy import asarray as ar, exp -from scipy import stats - -from numpy.ma.core import getdata - -SPEED_OF_LIGHT = 299792458 -SPEED_OF_LIGHT = 3e8 - -try: - from gevent import sleep -except: - from time import sleep - -from schainpy.model.data.jrodata import Spectra -#from schainpy.model.data.BLTRheaderIO import FileHeader, RecordHeader -from schainpy.model.proc.jroproc_base import ProcessingUnit, Operation -#from schainpy.model.io.jroIO_bltr import BLTRReader -from numpy import imag, shape, NaN - -from jroIO_base import JRODataReader - - -class Header(object): - - def __init__(self): - raise NotImplementedError - - def read(self): - - raise NotImplementedError - - def write(self): - - raise NotImplementedError - - def printInfo(self): - - message = "#" * 50 + "\n" - message += self.__class__.__name__.upper() + "\n" - message += "#" * 50 + "\n" - - keyList = self.__dict__.keys() - keyList.sort() - - for key in keyList: - message += "%s = %s" % (key, self.__dict__[key]) + "\n" - - if "size" not in keyList: - attr = getattr(self, "size") - - if attr: - message += "%s = %s" % ("size", attr) + "\n" - - # print message - - -FILE_STRUCTURE = numpy.dtype([ # HEADER 48bytes - ('FileMgcNumber', ' vertical) - ('BeamAngleZen', ' endFp: - sys.stderr.write( - "Warning %s: Size value read from System Header is lower than it has to be\n" % fp) - return 0 - - if OffRHeader < endFp: - sys.stderr.write( - "Warning %s: Size value read from System Header size is greater than it has to be\n" % fp) - return 0 - - return 1 - - -class BLTRSpectraReader (ProcessingUnit, FileHeaderBLTR, RecordHeaderBLTR, JRODataReader): - - path = None - startDate = None - endDate = None - startTime = None - endTime = None - walk = None - isConfig = False - - fileList = None - - # metadata - TimeZone = None - Interval = None - heightList = None - - # data - data = None - utctime = None - - def __init__(self, **kwargs): - - # Eliminar de la base la herencia - ProcessingUnit.__init__(self, **kwargs) - - #self.isConfig = False - - #self.pts2read_SelfSpectra = 0 - #self.pts2read_CrossSpectra = 0 - #self.pts2read_DCchannels = 0 - #self.datablock = None - self.utc = None - self.ext = ".fdt" - self.optchar = "P" - self.fpFile = None - self.fp = None - self.BlockCounter = 0 - self.dtype = None - self.fileSizeByHeader = None - self.filenameList = [] - self.fileSelector = 0 - self.Off2StartNxtRec = 0 - self.RecCounter = 0 - self.flagNoMoreFiles = 0 - self.data_spc = None - self.data_cspc = None - self.data_output = None - self.path = None - self.OffsetStartHeader = 0 - self.Off2StartData = 0 - self.ipp = 0 - self.nFDTdataRecors = 0 - self.blocksize = 0 - self.dataOut = Spectra() - self.profileIndex = 1 # Always - self.dataOut.flagNoData = False - self.dataOut.nRdPairs = 0 - self.dataOut.data_spc = None - self.dataOut.velocityX = [] - self.dataOut.velocityY = [] - self.dataOut.velocityV = [] - - def Files2Read(self, fp): - ''' - Function that indicates the number of .fdt files that exist in the folder to be read. - It also creates an organized list with the names of the files to read. - ''' - # self.__checkPath() - - # Gets the list of files within the fp address - ListaData = os.listdir(fp) - # Sort the list of files from least to largest by names - ListaData = sorted(ListaData) - nFiles = 0 # File Counter - FileList = [] # A list is created that will contain the .fdt files - for IndexFile in ListaData: - if '.fdt' in IndexFile: - FileList.append(IndexFile) - nFiles += 1 - - # print 'Files2Read' - # print 'Existen '+str(nFiles)+' archivos .fdt' - - self.filenameList = FileList # List of files from least to largest by names - - def run(self, **kwargs): - ''' - This method will be the one that will initiate the data entry, will be called constantly. - You should first verify that your Setup () is set up and then continue to acquire - the data to be processed with getData (). - ''' - if not self.isConfig: - self.setup(**kwargs) - self.isConfig = True - - self.getData() - # print 'running' - - def setup(self, path=None, - startDate=None, - endDate=None, - startTime=None, - endTime=None, - walk=True, - timezone='utc', - code=None, - online=False, - ReadMode=None, - **kwargs): - - self.isConfig = True - - self.path = path - self.startDate = startDate - self.endDate = endDate - self.startTime = startTime - self.endTime = endTime - self.walk = walk - self.ReadMode = int(ReadMode) - - pass - - def getData(self): - ''' - Before starting this function, you should check that there is still an unread file, - If there are still blocks to read or if the data block is empty. - - You should call the file "read". - - ''' - - if self.flagNoMoreFiles: - self.dataOut.flagNoData = True - print 'NoData se vuelve true' - return 0 - - self.fp = self.path - self.Files2Read(self.fp) - self.readFile(self.fp) - self.dataOut.data_spc = self.data_spc - self.dataOut.data_cspc = self.data_cspc - self.dataOut.data_output = self.data_output - - print 'self.dataOut.data_output', shape(self.dataOut.data_output) - - # self.removeDC() - return self.dataOut.data_spc - - def readFile(self, fp): - ''' - You must indicate if you are reading in Online or Offline mode and load the - The parameters for this file reading mode. - - Then you must do 2 actions: - - 1. Get the BLTR FileHeader. - 2. Start reading the first block. - ''' - - # The address of the folder is generated the name of the .fdt file that will be read - print "File: ", self.fileSelector + 1 - - if self.fileSelector < len(self.filenameList): - - self.fpFile = str(fp) + '/' + \ - str(self.filenameList[self.fileSelector]) - # print self.fpFile - fheader = FileHeaderBLTR() - fheader.FHread(self.fpFile) # Bltr FileHeader Reading - self.nFDTdataRecors = fheader.nFDTdataRecors - - self.readBlock() # Block reading - else: - print 'readFile FlagNoData becomes true' - self.flagNoMoreFiles = True - self.dataOut.flagNoData = True - return 0 - - def getVelRange(self, extrapoints=0): - Lambda = SPEED_OF_LIGHT / 50000000 - # 1./(self.dataOut.ippSeconds * self.dataOut.nCohInt) - PRF = self.dataOut.PRF - Vmax = -Lambda / (4. * (1. / PRF) * self.dataOut.nCohInt * 2.) - deltafreq = PRF / (self.nProfiles) - deltavel = (Vmax * 2) / (self.nProfiles) - freqrange = deltafreq * \ - (numpy.arange(self.nProfiles) - self.nProfiles / 2.) - deltafreq / 2 - velrange = deltavel * \ - (numpy.arange(self.nProfiles) - self.nProfiles / 2.) - return velrange - - def readBlock(self): - ''' - It should be checked if the block has data, if it is not passed to the next file. - - Then the following is done: - - 1. Read the RecordHeader - 2. Fill the buffer with the current block number. - - ''' - - if self.BlockCounter < self.nFDTdataRecors - 2: - print self.nFDTdataRecors, 'CONDICION!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!' - if self.ReadMode == 1: - rheader = RecordHeaderBLTR(RecCounter=self.BlockCounter + 1) - elif self.ReadMode == 0: - rheader = RecordHeaderBLTR(RecCounter=self.BlockCounter) - - rheader.RHread(self.fpFile) # Bltr FileHeader Reading - - self.OffsetStartHeader = rheader.OffsetStartHeader - self.RecCounter = rheader.RecCounter - self.Off2StartNxtRec = rheader.Off2StartNxtRec - self.Off2StartData = rheader.Off2StartData - self.nProfiles = rheader.nProfiles - self.nChannels = rheader.nChannels - self.nHeights = rheader.nHeights - self.frequency = rheader.TransmitFrec - self.DualModeIndex = rheader.DualModeIndex - - self.pairsList = [(0, 1), (0, 2), (1, 2)] - self.dataOut.pairsList = self.pairsList - - self.nRdPairs = len(self.dataOut.pairsList) - self.dataOut.nRdPairs = self.nRdPairs - - self.__firstHeigth = rheader.StartRangeSamp - self.__deltaHeigth = rheader.SampResolution - self.dataOut.heightList = self.__firstHeigth + \ - numpy.array(range(self.nHeights)) * self.__deltaHeigth - self.dataOut.channelList = range(self.nChannels) - self.dataOut.nProfiles = rheader.nProfiles - self.dataOut.nIncohInt = rheader.nIncohInt - self.dataOut.nCohInt = rheader.nCohInt - self.dataOut.ippSeconds = 1 / float(rheader.PRFhz) - self.dataOut.PRF = rheader.PRFhz - self.dataOut.nFFTPoints = rheader.nProfiles - self.dataOut.utctime = rheader.nUtime - self.dataOut.timeZone = 0 - self.dataOut.normFactor = self.dataOut.nProfiles * \ - self.dataOut.nIncohInt * self.dataOut.nCohInt - self.dataOut.outputInterval = self.dataOut.ippSeconds * \ - self.dataOut.nCohInt * self.dataOut.nIncohInt * self.nProfiles - - self.data_output = numpy.ones([3, rheader.nHeights]) * numpy.NaN - print 'self.data_output', shape(self.data_output) - self.dataOut.velocityX = [] - self.dataOut.velocityY = [] - self.dataOut.velocityV = [] - - '''Block Reading, the Block Data is received and Reshape is used to give it - shape. - ''' - - # Procedure to take the pointer to where the date block starts - startDATA = open(self.fpFile, "rb") - OffDATA = self.OffsetStartHeader + self.RecCounter * \ - self.Off2StartNxtRec + self.Off2StartData - startDATA.seek(OffDATA, os.SEEK_SET) - - def moving_average(x, N=2): - return numpy.convolve(x, numpy.ones((N,)) / N)[(N - 1):] - - def gaus(xSamples, a, x0, sigma): - return a * exp(-(xSamples - x0)**2 / (2 * sigma**2)) - - def Find(x, value): - for index in range(len(x)): - if x[index] == value: - return index - - def pol2cart(rho, phi): - x = rho * numpy.cos(phi) - y = rho * numpy.sin(phi) - return(x, y) - - if self.DualModeIndex == self.ReadMode: - - self.data_fft = numpy.fromfile( - startDATA, [('complex', ' 0.0001) : -# -# try: -# popt,pcov = curve_fit(gaus,xSamples,yMean,p0=[1,meanGauss,sigma]) -# -# if numpy.amax(popt)>numpy.amax(yMean)*0.3: -# FitGauss=gaus(xSamples,*popt) -# -# else: -# FitGauss=numpy.ones(len(xSamples))*numpy.mean(yMean) -# print 'Verificador: Dentro', Height -# except RuntimeError: -# -# try: -# for j in range(len(ySamples[1])): -# yMean2=numpy.append(yMean2,numpy.average([ySamples[1,j],ySamples[2,j]])) -# popt,pcov = curve_fit(gaus,xSamples,yMean2,p0=[1,meanGauss,sigma]) -# FitGauss=gaus(xSamples,*popt) -# print 'Verificador: Exepcion1', Height -# except RuntimeError: -# -# try: -# popt,pcov = curve_fit(gaus,xSamples,ySamples[1],p0=[1,meanGauss,sigma]) -# FitGauss=gaus(xSamples,*popt) -# print 'Verificador: Exepcion2', Height -# except RuntimeError: -# FitGauss=numpy.ones(len(xSamples))*numpy.mean(yMean) -# print 'Verificador: Exepcion3', Height -# else: -# FitGauss=numpy.ones(len(xSamples))*numpy.mean(yMean) -# #print 'Verificador: Fuera', Height -# -# -# -# Maximun=numpy.amax(yMean) -# eMinus1=Maximun*numpy.exp(-1) -# -# HWpos=Find(FitGauss,min(FitGauss, key=lambda value:abs(value-eMinus1))) -# HalfWidth= xFrec[HWpos] -# GCpos=Find(FitGauss, numpy.amax(FitGauss)) -# Vpos=Find(FactNorm, numpy.amax(FactNorm)) -# #Vpos=numpy.sum(FactNorm)/len(FactNorm) -# #Vpos=Find(FactNorm, min(FactNorm, key=lambda value:abs(value- numpy.mean(FactNorm) ))) -# #print 'GCpos',GCpos, numpy.amax(FitGauss), 'HWpos',HWpos -# '''****** Getting Fij ******''' -# -# GaussCenter=xFrec[GCpos] -# if (GaussCenter<0 and HalfWidth>0) or (GaussCenter>0 and HalfWidth<0): -# Fij=abs(GaussCenter)+abs(HalfWidth)+0.0000001 -# else: -# Fij=abs(GaussCenter-HalfWidth)+0.0000001 -# -# '''****** Getting Frecuency range of significant data ******''' -# -# Rangpos=Find(FitGauss,min(FitGauss, key=lambda value:abs(value-Maximun*0.10))) -# -# if Rangpos5 and len(FrecRange) 0.: -# self.dataOut.velocityX=numpy.append(self.dataOut.velocityX, Vzon) #Vmag -# #print 'Vmag',Vmag -# else: -# self.dataOut.velocityX=numpy.append(self.dataOut.velocityX, NaN) -# -# if abs(Vx)<100 and abs(Vx) > 0.: -# self.dataOut.velocityY=numpy.append(self.dataOut.velocityY, Vmer) #Vang -# #print 'Vang',Vang -# else: -# self.dataOut.velocityY=numpy.append(self.dataOut.velocityY, NaN) -# -# if abs(GaussCenter)<2: -# self.dataOut.velocityV=numpy.append(self.dataOut.velocityV, xFrec[Vpos]) -# -# else: -# self.dataOut.velocityV=numpy.append(self.dataOut.velocityV, NaN) -# -# -# # print '********************************************' -# # print 'HalfWidth ', HalfWidth -# # print 'Maximun ', Maximun -# # print 'eMinus1 ', eMinus1 -# # print 'Rangpos ', Rangpos -# # print 'GaussCenter ',GaussCenter -# # print 'E01 ',E01 -# # print 'N01 ',N01 -# # print 'E02 ',E02 -# # print 'N02 ',N02 -# # print 'E12 ',E12 -# # print 'N12 ',N12 -# #print 'self.dataOut.velocityX ', self.dataOut.velocityX -# # print 'Fij ', Fij -# # print 'cC ', cC -# # print 'cF ', cF -# # print 'cG ', cG -# # print 'cA ', cA -# # print 'cB ', cB -# # print 'cH ', cH -# # print 'Vx ', Vx -# # print 'Vy ', Vy -# # print 'Vmag ', Vmag -# # print 'Vang ', Vang*180/numpy.pi -# # print 'PhaseSlope ',PhaseSlope[0] -# # print 'PhaseSlope ',PhaseSlope[1] -# # print 'PhaseSlope ',PhaseSlope[2] -# # print '********************************************' -# #print 'data_output',shape(self.dataOut.velocityX), shape(self.dataOut.velocityY) -# -# #print 'self.dataOut.velocityX', len(self.dataOut.velocityX) -# #print 'self.dataOut.velocityY', len(self.dataOut.velocityY) -# #print 'self.dataOut.velocityV', self.dataOut.velocityV -# -# self.data_output[0]=numpy.array(self.dataOut.velocityX) -# self.data_output[1]=numpy.array(self.dataOut.velocityY) -# self.data_output[2]=numpy.array(self.dataOut.velocityV) -# -# prin= self.data_output[0][~numpy.isnan(self.data_output[0])] -# print ' ' -# print 'VmagAverage',numpy.mean(prin) -# print ' ' -# # plt.figure(5) -# # plt.subplot(211) -# # plt.plot(self.dataOut.velocityX,'yo:') -# # plt.subplot(212) -# # plt.plot(self.dataOut.velocityY,'yo:') -# -# # plt.figure(1) -# # # plt.subplot(121) -# # # plt.plot(xFrec,ySamples[0],'k',label='Ch0') -# # # plt.plot(xFrec,ySamples[1],'g',label='Ch1') -# # # plt.plot(xFrec,ySamples[2],'r',label='Ch2') -# # # plt.plot(xFrec,FitGauss,'yo:',label='fit') -# # # plt.legend() -# # plt.title('DATOS A ALTURA DE 2850 METROS') -# # -# # plt.xlabel('Frecuencia (KHz)') -# # plt.ylabel('Magnitud') -# # # plt.subplot(122) -# # # plt.title('Fit for Time Constant') -# # #plt.plot(xFrec,zline) -# # #plt.plot(xFrec,SmoothSPC,'g') -# # plt.plot(xFrec,FactNorm) -# # plt.axis([-4, 4, 0, 0.15]) -# # # plt.xlabel('SelfSpectra KHz') -# # -# # plt.figure(10) -# # # plt.subplot(121) -# # plt.plot(xFrec,ySamples[0],'b',label='Ch0') -# # plt.plot(xFrec,ySamples[1],'y',label='Ch1') -# # plt.plot(xFrec,ySamples[2],'r',label='Ch2') -# # # plt.plot(xFrec,FitGauss,'yo:',label='fit') -# # plt.legend() -# # plt.title('SELFSPECTRA EN CANALES') -# # -# # plt.xlabel('Frecuencia (KHz)') -# # plt.ylabel('Magnitud') -# # # plt.subplot(122) -# # # plt.title('Fit for Time Constant') -# # #plt.plot(xFrec,zline) -# # #plt.plot(xFrec,SmoothSPC,'g') -# # # plt.plot(xFrec,FactNorm) -# # # plt.axis([-4, 4, 0, 0.15]) -# # # plt.xlabel('SelfSpectra KHz') -# # -# # plt.figure(9) -# # -# # -# # plt.title('DATOS SUAVIZADOS') -# # plt.xlabel('Frecuencia (KHz)') -# # plt.ylabel('Magnitud') -# # plt.plot(xFrec,SmoothSPC,'g') -# # -# # #plt.plot(xFrec,FactNorm) -# # plt.axis([-4, 4, 0, 0.15]) -# # # plt.xlabel('SelfSpectra KHz') -# # # -# # plt.figure(2) -# # # #plt.subplot(121) -# # plt.plot(xFrec,yMean,'r',label='Mean SelfSpectra') -# # plt.plot(xFrec,FitGauss,'yo:',label='Ajuste Gaussiano') -# # # plt.plot(xFrec[Rangpos],FitGauss[Find(FitGauss,min(FitGauss, key=lambda value:abs(value-Maximun*0.1)))],'bo') -# # # #plt.plot(xFrec,phase) -# # # plt.xlabel('Suavizado, promediado KHz') -# # plt.title('SELFSPECTRA PROMEDIADO') -# # # #plt.subplot(122) -# # # #plt.plot(xSamples,zline) -# # plt.xlabel('Frecuencia (KHz)') -# # plt.ylabel('Magnitud') -# # plt.legend() -# # # -# # # plt.figure(3) -# # # plt.subplot(311) -# # # #plt.plot(xFrec,phase[0]) -# # # plt.plot(xFrec,phase[0],'g') -# # # plt.subplot(312) -# # # plt.plot(xFrec,phase[1],'g') -# # # plt.subplot(313) -# # # plt.plot(xFrec,phase[2],'g') -# # # #plt.plot(xFrec,phase[2]) -# # # -# # # plt.figure(4) -# # # -# # # plt.plot(xSamples,coherence[0],'b') -# # # plt.plot(xSamples,coherence[1],'r') -# # # plt.plot(xSamples,coherence[2],'g') -# # plt.show() -# # # -# # # plt.clf() -# # # plt.cla() -# # # plt.close() -# -# print ' ' - - self.BlockCounter += 2 - - else: - self.fileSelector += 1 - self.BlockCounter = 0 - print "Next File" diff --git a/schainpy/model/io/jroIO_digitalRF.py b/schainpy/model/io/jroIO_digitalRF.py index 45576b6..cb6be51 100644 --- a/schainpy/model/io/jroIO_digitalRF.py +++ b/schainpy/model/io/jroIO_digitalRF.py @@ -11,26 +11,24 @@ Created on Jul 3, 2014 # METADATA import os +import time import datetime import numpy import timeit from fractions import Fraction +from time import time +from time import sleep -try: - from gevent import sleep -except: - from time import sleep - +import schainpy.admin from schainpy.model.data.jroheaderIO import RadarControllerHeader, SystemHeader from schainpy.model.data.jrodata import Voltage -from schainpy.model.proc.jroproc_base import ProcessingUnit, Operation -from time import time +from schainpy.model.proc.jroproc_base import ProcessingUnit, Operation, MPDecorator -import cPickle +import pickle try: import digital_rf except: - print 'You should install "digital_rf" module if you want to read Digital RF data' + pass class DigitalRFReader(ProcessingUnit): @@ -38,28 +36,28 @@ class DigitalRFReader(ProcessingUnit): classdocs ''' - def __init__(self, **kwargs): + def __init__(self): ''' Constructor ''' - ProcessingUnit.__init__(self, **kwargs) + ProcessingUnit.__init__(self) - self.dataOut = Voltage() - self.__printInfo = True + self.dataOut = Voltage() + self.__printInfo = True self.__flagDiscontinuousBlock = False self.__bufferIndex = 9999999 - self.__ippKm = None - self.__codeType = 0 - self.__nCode = None - self.__nBaud = None - self.__code = None - self.dtype = None - self.oldAverage = None - self.path = None + self.__codeType = 0 + self.__ippKm = None + self.__nCode = None + self.__nBaud = None + self.__code = None + self.dtype = None + self.oldAverage = None + self.path = None def close(self): - print 'Average of writing to digital rf format is ', self.oldAverage * 1000 + print('Average of writing to digital rf format is ', self.oldAverage * 1000) return def __getCurrentSecond(self): @@ -99,9 +97,9 @@ class DigitalRFReader(ProcessingUnit): nChannels=len( self.__channelList), adcResolution=14) - self.dataOut.type = "Voltage" + self.dataOut.type = "Voltage" - self.dataOut.data = None + self.dataOut.data = None self.dataOut.dtype = self.dtype @@ -109,32 +107,32 @@ class DigitalRFReader(ProcessingUnit): # self.dataOut.nHeights = 0 - self.dataOut.nProfiles = int(nProfiles) + self.dataOut.nProfiles = int(nProfiles) - self.dataOut.heightList = self.__firstHeigth + \ + self.dataOut.heightList = self.__firstHeigth + \ numpy.arange(self.__nSamples, dtype=numpy.float) * \ self.__deltaHeigth - self.dataOut.channelList = range(self.__num_subchannels) + self.dataOut.channelList = list(range(self.__num_subchannels)) - self.dataOut.blocksize = self.dataOut.getNChannels() * self.dataOut.getNHeights() + self.dataOut.blocksize = self.dataOut.getNChannels() * self.dataOut.getNHeights() # self.dataOut.channelIndexList = None - self.dataOut.flagNoData = True + self.dataOut.flagNoData = True self.dataOut.flagDataAsBlock = False # Set to TRUE if the data is discontinuous self.dataOut.flagDiscontinuousBlock = False - self.dataOut.utctime = None + self.dataOut.utctime = None # timezone like jroheader, difference in minutes between UTC and localtime - self.dataOut.timeZone = self.__timezone / 60 + self.dataOut.timeZone = self.__timezone / 60 - self.dataOut.dstFlag = 0 + self.dataOut.dstFlag = 0 - self.dataOut.errorCount = 0 + self.dataOut.errorCount = 0 try: self.dataOut.nCohInt = self.fixed_metadata_dict.get( @@ -147,9 +145,9 @@ class DigitalRFReader(ProcessingUnit): # asumo que la data esta sin flip self.dataOut.flagDeflipData = self.fixed_metadata_dict['flagDeflipData'] - self.dataOut.flagShiftFFT = self.fixed_metadata_dict['flagShiftFFT'] + self.dataOut.flagShiftFFT = self.fixed_metadata_dict['flagShiftFFT'] - self.dataOut.useLocalTime = self.fixed_metadata_dict['useLocalTime'] + self.dataOut.useLocalTime = self.fixed_metadata_dict['useLocalTime'] except: pass @@ -158,9 +156,9 @@ class DigitalRFReader(ProcessingUnit): # Time interval between profiles # self.dataOut.timeInterval = self.dataOut.ippSeconds * self.dataOut.nCohInt - self.dataOut.frequency = self.__frequency + self.dataOut.frequency = self.__frequency - self.dataOut.realtime = self.__online + self.dataOut.realtime = self.__online def findDatafiles(self, path, startDate=None, endDate=None): @@ -173,46 +171,46 @@ class DigitalRFReader(ProcessingUnit): except: digitalReadObj = digital_rf.DigitalRFReader(path) - channelNameList = digitalReadObj.get_channels() + channelNameList = digitalReadObj.get_channels() if not channelNameList: return [] - metadata_dict = digitalReadObj.get_rf_file_metadata(channelNameList[0]) + metadata_dict = digitalReadObj.get_rf_file_metadata(channelNameList[0]) - sample_rate = metadata_dict['sample_rate'][0] + sample_rate = metadata_dict['sample_rate'][0] this_metadata_file = digitalReadObj.get_metadata(channelNameList[0]) try: - timezone = this_metadata_file['timezone'].value + timezone = this_metadata_file['timezone'].value except: - timezone = 0 + timezone = 0 startUTCSecond, endUTCSecond = digitalReadObj.get_bounds( channelNameList[0]) / sample_rate - timezone - startDatetime = datetime.datetime.utcfromtimestamp(startUTCSecond) - endDatatime = datetime.datetime.utcfromtimestamp(endUTCSecond) + startDatetime = datetime.datetime.utcfromtimestamp(startUTCSecond) + endDatatime = datetime.datetime.utcfromtimestamp(endUTCSecond) if not startDate: - startDate = startDatetime.date() + startDate = startDatetime.date() if not endDate: - endDate = endDatatime.date() + endDate = endDatatime.date() - dateList = [] + dateList = [] - thisDatetime = startDatetime + thisDatetime = startDatetime while(thisDatetime <= endDatatime): - thisDate = thisDatetime.date() + thisDate = thisDatetime.date() - if thisDate < startDate: + if thisDate < startDate: continue - if thisDate > endDate: + if thisDate > endDate: break dateList.append(thisDate) @@ -245,19 +243,19 @@ class DigitalRFReader(ProcessingUnit): startDate endDate startTime - endTime + endTime set expLabel ext online delay ''' - self.path = path - self.nCohInt = nCohInt + self.path = path + self.nCohInt = nCohInt self.flagDecodeData = flagDecodeData - self.i = 0 + self.i = 0 if not os.path.isdir(path): - raise ValueError, "[Reading] Directory %s does not exist" % path + raise ValueError("[Reading] Directory %s does not exist" % path) try: self.digitalReadObj = digital_rf.DigitalRFReader( @@ -265,37 +263,37 @@ class DigitalRFReader(ProcessingUnit): except: self.digitalReadObj = digital_rf.DigitalRFReader(path) - channelNameList = self.digitalReadObj.get_channels() + channelNameList = self.digitalReadObj.get_channels() if not channelNameList: - raise ValueError, "[Reading] Directory %s does not have any files" % path + raise ValueError("[Reading] Directory %s does not have any files" % path) if not channelList: - channelList = range(len(channelNameList)) + channelList = list(range(len(channelNameList))) ########## Reading metadata ###################### - top_properties = self.digitalReadObj.get_properties( + top_properties = self.digitalReadObj.get_properties( channelNameList[channelList[0]]) - self.__num_subchannels = top_properties['num_subchannels'] - self.__sample_rate = 1.0 * \ + self.__num_subchannels = top_properties['num_subchannels'] + self.__sample_rate = 1.0 * \ top_properties['sample_rate_numerator'] / \ top_properties['sample_rate_denominator'] # self.__samples_per_file = top_properties['samples_per_file'][0] - self.__deltaHeigth = 1e6 * 0.15 / self.__sample_rate # why 0.15? + self.__deltaHeigth = 1e6 * 0.15 / self.__sample_rate # why 0.15? - this_metadata_file = self.digitalReadObj.get_digital_metadata( + this_metadata_file = self.digitalReadObj.get_digital_metadata( channelNameList[channelList[0]]) - metadata_bounds = this_metadata_file.get_bounds() + metadata_bounds = this_metadata_file.get_bounds() self.fixed_metadata_dict = this_metadata_file.read( metadata_bounds[0])[metadata_bounds[0]] # GET FIRST HEADER try: - self.__processingHeader = self.fixed_metadata_dict['processingHeader'] + self.__processingHeader = self.fixed_metadata_dict['processingHeader'] self.__radarControllerHeader = self.fixed_metadata_dict['radarControllerHeader'] - self.__systemHeader = self.fixed_metadata_dict['systemHeader'] - self.dtype = cPickle.loads(self.fixed_metadata_dict['dtype']) + self.__systemHeader = self.fixed_metadata_dict['systemHeader'] + self.dtype = pickle.loads(self.fixed_metadata_dict['dtype']) except: pass @@ -313,15 +311,15 @@ class DigitalRFReader(ProcessingUnit): self.__firstHeigth = 0 try: - codeType = self.__radarControllerHeader['codeType'] + codeType = self.__radarControllerHeader['codeType'] except: - codeType = 0 + codeType = 0 try: if codeType: nCode = self.__radarControllerHeader['nCode'] nBaud = self.__radarControllerHeader['nBaud'] - code = self.__radarControllerHeader['code'] + code = self.__radarControllerHeader['code'] except: pass @@ -332,18 +330,18 @@ class DigitalRFReader(ProcessingUnit): except: ippKm = None #################################################### - self.__ippKm = ippKm + self.__ippKm = ippKm startUTCSecond = None - endUTCSecond = None + endUTCSecond = None if startDate: - startDatetime = datetime.datetime.combine(startDate, startTime) + startDatetime = datetime.datetime.combine(startDate, startTime) startUTCSecond = ( startDatetime - datetime.datetime(1970, 1, 1)).total_seconds() + self.__timezone if endDate: - endDatetime = datetime.datetime.combine(endDate, endTime) - endUTCSecond = (endDatetime - datetime.datetime(1970, + endDatetime = datetime.datetime.combine(endDate, endTime) + endUTCSecond = (endDatetime - datetime.datetime(1970, 1, 1)).total_seconds() + self.__timezone start_index, end_index = self.digitalReadObj.get_bounds( @@ -352,50 +350,50 @@ class DigitalRFReader(ProcessingUnit): if not startUTCSecond: startUTCSecond = start_index / self.__sample_rate - if start_index > startUTCSecond * self.__sample_rate: + if start_index > startUTCSecond * self.__sample_rate: startUTCSecond = start_index / self.__sample_rate if not endUTCSecond: - endUTCSecond = end_index / self.__sample_rate + endUTCSecond = end_index / self.__sample_rate - if end_index < endUTCSecond * self.__sample_rate: - endUTCSecond = end_index / self.__sample_rate + if end_index < endUTCSecond * self.__sample_rate: + endUTCSecond = end_index / self.__sample_rate if not nSamples: if not ippKm: - raise ValueError, "[Reading] nSamples or ippKm should be defined" - nSamples = int(ippKm / (1e6 * 0.15 / self.__sample_rate)) - channelBoundList = [] + raise ValueError("[Reading] nSamples or ippKm should be defined") + nSamples = int(ippKm / (1e6 * 0.15 / self.__sample_rate)) + channelBoundList = [] channelNameListFiltered = [] for thisIndexChannel in channelList: - thisChannelName = channelNameList[thisIndexChannel] + thisChannelName = channelNameList[thisIndexChannel] start_index, end_index = self.digitalReadObj.get_bounds( thisChannelName) channelBoundList.append((start_index, end_index)) channelNameListFiltered.append(thisChannelName) self.profileIndex = 0 - self.i = 0 - self.__delay = delay - - self.__codeType = codeType - self.__nCode = nCode - self.__nBaud = nBaud - self.__code = code - - self.__datapath = path - self.__online = online - self.__channelList = channelList - self.__channelNameList = channelNameListFiltered + self.i = 0 + self.__delay = delay + + self.__codeType = codeType + self.__nCode = nCode + self.__nBaud = nBaud + self.__code = code + + self.__datapath = path + self.__online = online + self.__channelList = channelList + self.__channelNameList = channelNameListFiltered self.__channelBoundList = channelBoundList - self.__nSamples = nSamples - self.__samples_to_read = long(nSamples) # FIJO: AHORA 40 - self.__nChannels = len(self.__channelList) + self.__nSamples = nSamples + self.__samples_to_read = int(nSamples) # FIJO: AHORA 40 + self.__nChannels = len(self.__channelList) - self.__startUTCSecond = startUTCSecond - self.__endUTCSecond = endUTCSecond + self.__startUTCSecond = startUTCSecond + self.__endUTCSecond = endUTCSecond - self.__timeInterval = 1.0 * self.__samples_to_read / \ + self.__timeInterval = 1.0 * self.__samples_to_read / \ self.__sample_rate # Time interval if online: @@ -403,28 +401,27 @@ class DigitalRFReader(ProcessingUnit): startUTCSecond = numpy.floor(endUTCSecond) # por que en el otro metodo lo primero q se hace es sumar samplestoread - self.__thisUnixSample = long( - startUTCSecond * self.__sample_rate) - self.__samples_to_read + self.__thisUnixSample = int(startUTCSecond * self.__sample_rate) - self.__samples_to_read - self.__data_buffer = numpy.zeros( + self.__data_buffer = numpy.zeros( (self.__num_subchannels, self.__samples_to_read), dtype=numpy.complex) self.__setFileHeader() self.isConfig = True - print "[Reading] Digital RF Data was found from %s to %s " % ( + print("[Reading] Digital RF Data was found from %s to %s " % ( datetime.datetime.utcfromtimestamp( self.__startUTCSecond - self.__timezone), datetime.datetime.utcfromtimestamp( self.__endUTCSecond - self.__timezone) - ) + )) - print "[Reading] Starting process from %s to %s" % (datetime.datetime.utcfromtimestamp(startUTCSecond - self.__timezone), + print("[Reading] Starting process from %s to %s" % (datetime.datetime.utcfromtimestamp(startUTCSecond - self.__timezone), datetime.datetime.utcfromtimestamp( endUTCSecond - self.__timezone) - ) - self.oldAverage = None - self.count = 0 + )) + self.oldAverage = None + self.count = 0 self.executionTime = 0 def __reload(self): @@ -434,43 +431,42 @@ class DigitalRFReader(ProcessingUnit): # datetime.datetime.utcfromtimestamp(self.__startUTCSecond - self.__timezone), # datetime.datetime.utcfromtimestamp(self.__endUTCSecond - self.__timezone) # ) - print "[Reading] reloading metadata ..." + print("[Reading] reloading metadata ...") try: self.digitalReadObj.reload(complete_update=True) except: - self.digitalReadObj = digital_rf.DigitalRFReader(self.path) - #self.digitalReadObj.reload() + self.digitalReadObj = digital_rf.DigitalRFReader(self.path) - start_index, end_index = self.digitalReadObj.get_bounds( + start_index, end_index = self.digitalReadObj.get_bounds( self.__channelNameList[self.__channelList[0]]) - if start_index > self.__startUTCSecond * self.__sample_rate: + if start_index > self.__startUTCSecond * self.__sample_rate: self.__startUTCSecond = 1.0 * start_index / self.__sample_rate - if end_index > self.__endUTCSecond * self.__sample_rate: + if end_index > self.__endUTCSecond * self.__sample_rate: self.__endUTCSecond = 1.0 * end_index / self.__sample_rate - print - print "[Reading] New timerange found [%s, %s] " % ( + print() + print("[Reading] New timerange found [%s, %s] " % ( datetime.datetime.utcfromtimestamp( self.__startUTCSecond - self.__timezone), datetime.datetime.utcfromtimestamp( self.__endUTCSecond - self.__timezone) - ) + )) return True return False def timeit(self, toExecute): - t0 = time() + t0 = time.time() toExecute() - self.executionTime = time() - t0 + self.executionTime = time.time() - t0 if self.oldAverage is None: self.oldAverage = self.executionTime - self.oldAverage = (self.executionTime + self.count * + self.oldAverage = (self.executionTime + self.count * self.oldAverage) / (self.count + 1.0) - self.count = self.count + 1.0 + self.count = self.count + 1.0 return def __readNextBlock(self, seconds=30, volt_scale=1): @@ -479,10 +475,10 @@ class DigitalRFReader(ProcessingUnit): # Set the next data self.__flagDiscontinuousBlock = False - self.__thisUnixSample += self.__samples_to_read - + self.__thisUnixSample += self.__samples_to_read + if self.__thisUnixSample + 2 * self.__samples_to_read > self.__endUTCSecond * self.__sample_rate: - print "[Reading] There are no more data into selected time-range" + print ("[Reading] There are no more data into selected time-range") if self.__online: sleep(3) self.__reload() @@ -496,49 +492,49 @@ class DigitalRFReader(ProcessingUnit): indexChannel = 0 dataOk = False + for thisChannelName in self.__channelNameList: # TODO VARIOS CHANNELS? for indexSubchannel in range(self.__num_subchannels): try: - t0 = time() + t0 = time() result = self.digitalReadObj.read_vector_c81d(self.__thisUnixSample, self.__samples_to_read, thisChannelName, sub_channel=indexSubchannel) - self.executionTime = time() - t0 + self.executionTime = time() - t0 if self.oldAverage is None: self.oldAverage = self.executionTime - self.oldAverage = ( + self.oldAverage = ( self.executionTime + self.count * self.oldAverage) / (self.count + 1.0) self.count = self.count + 1.0 - except IOError, e: + except IOError as e: # read next profile self.__flagDiscontinuousBlock = True - print "[Reading] %s" % datetime.datetime.utcfromtimestamp(self.thisSecond - self.__timezone), e + print("[Reading] %s" % datetime.datetime.utcfromtimestamp(self.thisSecond - self.__timezone), e) break if result.shape[0] != self.__samples_to_read: self.__flagDiscontinuousBlock = True - print "[Reading] %s: Too few samples were found, just %d/%d samples" % (datetime.datetime.utcfromtimestamp(self.thisSecond - self.__timezone), + print("[Reading] %s: Too few samples were found, just %d/%d samples" % (datetime.datetime.utcfromtimestamp(self.thisSecond - self.__timezone), result.shape[0], - self.__samples_to_read) + self.__samples_to_read)) break - + self.__data_buffer[indexSubchannel, :] = result * volt_scale + indexChannel+=1 - indexChannel += 1 - - dataOk = True + dataOk = True - self.__utctime = self.__thisUnixSample / self.__sample_rate + self.__utctime = self.__thisUnixSample / self.__sample_rate if not dataOk: return False - print "[Reading] %s: %d samples <> %f sec" % (datetime.datetime.utcfromtimestamp(self.thisSecond - self.__timezone), + print("[Reading] %s: %d samples <> %f sec" % (datetime.datetime.utcfromtimestamp(self.thisSecond - self.__timezone), self.__samples_to_read, - self.__timeInterval) + self.__timeInterval)) - self.__bufferIndex = 0 + self.__bufferIndex = 0 return True @@ -561,43 +557,46 @@ class DigitalRFReader(ProcessingUnit): self.flagDiscontinuousBlock self.flagIsNewBlock ''' - + #print("getdata") err_counter = 0 self.dataOut.flagNoData = True if self.__isBufferEmpty(): + #print("hi") self.__flagDiscontinuousBlock = False while True: + #print ("q ha pasado") if self.__readNextBlock(): break if self.__thisUnixSample > self.__endUTCSecond * self.__sample_rate: - return False + raise schainpy.admin.SchainError('Error') + return if self.__flagDiscontinuousBlock: - print '[Reading] discontinuous block found ... continue with the next block' - continue + raise schainpy.admin.SchainError('discontinuous block found') + return if not self.__online: - return False + raise schainpy.admin.SchainError('Online?') + return err_counter += 1 if err_counter > nTries: - return False + raise schainpy.admin.SchainError('Max retrys reach') + return - print '[Reading] waiting %d seconds to read a new block' % seconds - sleep(seconds) + print('[Reading] waiting %d seconds to read a new block' % seconds) + time.sleep(seconds) - self.dataOut.data = self.__data_buffer[:, - self.__bufferIndex:self.__bufferIndex + self.__nSamples] - self.dataOut.utctime = ( - self.__thisUnixSample + self.__bufferIndex) / self.__sample_rate - self.dataOut.flagNoData = False + self.dataOut.data = self.__data_buffer[:, self.__bufferIndex:self.__bufferIndex + self.__nSamples] + self.dataOut.utctime = ( self.__thisUnixSample + self.__bufferIndex) / self.__sample_rate + self.dataOut.flagNoData = False self.dataOut.flagDiscontinuousBlock = self.__flagDiscontinuousBlock - self.dataOut.profileIndex = self.profileIndex + self.dataOut.profileIndex = self.profileIndex self.__bufferIndex += self.__nSamples - self.profileIndex += 1 + self.profileIndex += 1 if self.profileIndex == self.dataOut.nProfiles: self.profileIndex = 0 @@ -625,15 +624,15 @@ class DigitalRFReader(ProcessingUnit): ''' This method will be called many times so here you should put all your code ''' - + if not self.isConfig: self.setup(**kwargs) #self.i = self.i+1 self.getData(seconds=self.__delay) - + return - +@MPDecorator class DigitalRFWriter(Operation): ''' classdocs @@ -645,25 +644,25 @@ class DigitalRFWriter(Operation): ''' Operation.__init__(self, **kwargs) self.metadata_dict = {} - self.dataOut = None - self.dtype = None - self.oldAverage = 0 + self.dataOut = None + self.dtype = None + self.oldAverage = 0 def setHeader(self): - self.metadata_dict['frequency'] = self.dataOut.frequency - self.metadata_dict['timezone'] = self.dataOut.timeZone - self.metadata_dict['dtype'] = cPickle.dumps(self.dataOut.dtype) - self.metadata_dict['nProfiles'] = self.dataOut.nProfiles - self.metadata_dict['heightList'] = self.dataOut.heightList - self.metadata_dict['channelList'] = self.dataOut.channelList + self.metadata_dict['frequency'] = self.dataOut.frequency + self.metadata_dict['timezone'] = self.dataOut.timeZone + self.metadata_dict['dtype'] = pickle.dumps(self.dataOut.dtype) + self.metadata_dict['nProfiles'] = self.dataOut.nProfiles + self.metadata_dict['heightList'] = self.dataOut.heightList + self.metadata_dict['channelList'] = self.dataOut.channelList self.metadata_dict['flagDecodeData'] = self.dataOut.flagDecodeData self.metadata_dict['flagDeflipData'] = self.dataOut.flagDeflipData - self.metadata_dict['flagShiftFFT'] = self.dataOut.flagShiftFFT - self.metadata_dict['useLocalTime'] = self.dataOut.useLocalTime - self.metadata_dict['nCohInt'] = self.dataOut.nCohInt - self.metadata_dict['type'] = self.dataOut.type - self.metadata_dict['flagDataAsBlock'] = getattr( + self.metadata_dict['flagShiftFFT'] = self.dataOut.flagShiftFFT + self.metadata_dict['useLocalTime'] = self.dataOut.useLocalTime + self.metadata_dict['nCohInt'] = self.dataOut.nCohInt + self.metadata_dict['type'] = self.dataOut.type + self.metadata_dict['flagDataAsBlock']= getattr( self.dataOut, 'flagDataAsBlock', None) # chequear def setup(self, dataOut, path, frequency, fileCadence, dirCadence, metadataCadence, set=0, metadataFile='metadata', ext='.h5'): @@ -673,13 +672,13 @@ class DigitalRFWriter(Operation): dataOut: Input data will also be outputa data ''' self.setHeader() - self.__ippSeconds = dataOut.ippSeconds - self.__deltaH = dataOut.getDeltaH() + self.__ippSeconds = dataOut.ippSeconds + self.__deltaH = dataOut.getDeltaH() self.__sample_rate = 1e6 * 0.15 / self.__deltaH - self.__dtype = dataOut.dtype + self.__dtype = dataOut.dtype if len(dataOut.dtype) == 2: self.__dtype = dataOut.dtype[0] - self.__nSamples = dataOut.systemHeaderObj.nSamples + self.__nSamples = dataOut.systemHeaderObj.nSamples self.__nProfiles = dataOut.nProfiles if self.dataOut.type != 'Voltage': @@ -690,44 +689,44 @@ class DigitalRFWriter(Operation): self.arr_data = numpy.ones((self.__nSamples, len( self.dataOut.channelList)), dtype=[('r', self.__dtype), ('i', self.__dtype)]) - file_cadence_millisecs = 1000 + file_cadence_millisecs = 1000 - sample_rate_fraction = Fraction(self.__sample_rate).limit_denominator() - sample_rate_numerator = long(sample_rate_fraction.numerator) - sample_rate_denominator = long(sample_rate_fraction.denominator) - start_global_index = dataOut.utctime * self.__sample_rate + sample_rate_fraction = Fraction(self.__sample_rate).limit_denominator() + sample_rate_numerator = int(sample_rate_fraction.numerator) + sample_rate_denominator = int(sample_rate_fraction.denominator) + start_global_index = dataOut.utctime * self.__sample_rate - uuid = 'prueba' + uuid = 'prueba' compression_level = 0 - checksum = False - is_complex = True - num_subchannels = len(dataOut.channelList) - is_continuous = True - marching_periods = False + checksum = False + is_complex = True + num_subchannels = len(dataOut.channelList) + is_continuous = True + marching_periods = False self.digitalWriteObj = digital_rf.DigitalRFWriter(path, self.__dtype, dirCadence, fileCadence, start_global_index, sample_rate_numerator, sample_rate_denominator, uuid, compression_level, checksum, is_complex, num_subchannels, is_continuous, marching_periods) - metadata_dir = os.path.join(path, 'metadata') + metadata_dir = os.path.join(path, 'metadata') os.system('mkdir %s' % (metadata_dir)) self.digitalMetadataWriteObj = digital_rf.DigitalMetadataWriter(metadata_dir, dirCadence, 1, # 236, file_cadence_millisecs / 1000 sample_rate_numerator, sample_rate_denominator, metadataFile) - self.isConfig = True + self.isConfig = True self.currentSample = 0 - self.oldAverage = 0 - self.count = 0 + self.oldAverage = 0 + self.count = 0 return def writeMetadata(self): - start_idx = self.__sample_rate * self.dataOut.utctime + start_idx = self.__sample_rate * self.dataOut.utctime - self.metadata_dict['processingHeader'] = self.dataOut.processingHeaderObj.getAsDict( + self.metadata_dict['processingHeader'] = self.dataOut.processingHeaderObj.getAsDict( ) self.metadata_dict['radarControllerHeader'] = self.dataOut.radarControllerHeaderObj.getAsDict( ) - self.metadata_dict['systemHeader'] = self.dataOut.systemHeaderObj.getAsDict( + self.metadata_dict['systemHeader'] = self.dataOut.systemHeaderObj.getAsDict( ) self.digitalMetadataWriteObj.write(start_idx, self.metadata_dict) return @@ -735,12 +734,12 @@ class DigitalRFWriter(Operation): def timeit(self, toExecute): t0 = time() toExecute() - self.executionTime = time() - t0 + self.executionTime = time() - t0 if self.oldAverage is None: self.oldAverage = self.executionTime - self.oldAverage = (self.executionTime + self.count * + self.oldAverage = (self.executionTime + self.count * self.oldAverage) / (self.count + 1.0) - self.count = self.count + 1.0 + self.count = self.count + 1.0 return def writeData(self): @@ -783,21 +782,12 @@ class DigitalRFWriter(Operation): # self.writeMetadata() ## if self.currentSample == self.__nProfiles: self.currentSample = 0 + return dataOut# en la version 2.7 no aparece este return + def close(self): - print '[Writing] - Closing files ' - print 'Average of writing to digital rf format is ', self.oldAverage * 1000 + print('[Writing] - Closing files ') + print('Average of writing to digital rf format is ', self.oldAverage * 1000) try: self.digitalWriteObj.close() except: pass - - - # raise -if __name__ == '__main__': - - readObj = DigitalRFReader() - - while True: - readObj.run(path='/home/jchavez/jicamarca/mocked_data/') - # readObj.printInfo() - # readObj.printNumberOfBlock() diff --git a/schainpy/model/io/jroIO_example.py b/schainpy/model/io/jroIO_example.py deleted file mode 100644 index 59f846a..0000000 --- a/schainpy/model/io/jroIO_example.py +++ /dev/null @@ -1,175 +0,0 @@ -''' -Created on Jul 3, 2014 - -@author: roj-idl71 -''' - -import os - -from schainpy.model.data.jrodata import Voltage -from schainpy.model.proc.jroproc_base import ProcessingUnit, Operation - -class Reader(ProcessingUnit): - ''' - classdocs - ''' - - def __init__(self): - ''' - Constructor - ''' - - ProcessingUnit.__init__(self) - - #Is really necessary create the output object in the initializer - self.dataOut = Voltage() - - def fillJROHeader(*args): - - self.dataOut.radarControllerHeaderObj = RadarControllerHeader(ippKm=10e5, - txA=0, - txB=0, - nWindows=1, - nHeights=self.__nSamples, - firstHeight=self.__firstHeigth, - deltaHeight=self.__deltaHeigth, - codeType=self.__codeType, - nCode=self.__nCode, nBaud=self.__nBaud, - code = self.__code) - - self.dataOut.systemHeaderObj = SystemHeader(nSamples=self.__nSamples, - nProfiles=nProfiles, - nChannels=len(self.__channelList), - adcResolution=14) - - self.dataOut.data = None - - self.dataOut.dtype = numpy.dtype([('real',' [%s]" %(filenameList[i], datetimeList[i].ctime()) + print("%s -> [%s]" %(filenameList[i], datetimeList[i].ctime())) self.filenameList = filenameList self.datetimeList = datetimeList @@ -552,22 +554,22 @@ class FitsReader(ProcessingUnit): walk = True): if path == None: - raise ValueError, "The path is not valid" + raise ValueError("The path is not valid") if ext == None: ext = self.ext if not(online): - print "Searching files in offline mode ..." + print("Searching files in offline mode ...") pathList, filenameList = self.searchFilesOffLine(path, startDate=startDate, endDate=endDate, startTime=startTime, endTime=endTime, set=set, expLabel=expLabel, ext=ext, walk=walk) if not(pathList): - print "No *%s files into the folder %s \nfor the range: %s - %s"%(ext, path, + print("No *%s files into the folder %s \nfor the range: %s - %s"%(ext, path, datetime.datetime.combine(startDate,startTime).ctime(), - datetime.datetime.combine(endDate,endTime).ctime()) + datetime.datetime.combine(endDate,endTime).ctime())) sys.exit(-1) @@ -582,11 +584,11 @@ class FitsReader(ProcessingUnit): if not(self.setNextFile()): if (startDate!=None) and (endDate!=None): - print "No files in range: %s - %s" %(datetime.datetime.combine(startDate,startTime).ctime(), datetime.datetime.combine(endDate,endTime).ctime()) + print("No files in range: %s - %s" %(datetime.datetime.combine(startDate,startTime).ctime(), datetime.datetime.combine(endDate,endTime).ctime())) elif startDate != None: - print "No files in range: %s" %(datetime.datetime.combine(startDate,startTime).ctime()) + print("No files in range: %s" %(datetime.datetime.combine(startDate,startTime).ctime())) else: - print "No files" + print("No files") sys.exit(-1) @@ -638,7 +640,7 @@ class FitsReader(ProcessingUnit): self.__rdBasicHeader() return 1 - print "\tWaiting %0.2f seconds for the next block, try %03d ..." % (self.delay, nTries+1) + print("\tWaiting %0.2f seconds for the next block, try %03d ..." % (self.delay, nTries+1)) sleep( self.delay ) @@ -691,18 +693,17 @@ class FitsReader(ProcessingUnit): if self.flagNoMoreFiles: self.dataOut.flagNoData = True - print 'Process finished' - return 0 + return (0, 'No more files') self.flagDiscontinuousBlock = 0 self.flagIsNewBlock = 0 if not(self.readNextBlock()): - return 0 + return (1, 'Error reading data') if self.data is None: self.dataOut.flagNoData = True - return 0 + return (0, 'No more data') self.dataOut.data = self.data self.dataOut.data_header = self.data_header_dict @@ -718,8 +719,7 @@ class FitsReader(ProcessingUnit): # self.dataOut.channelList = self.channelList # self.dataOut.heightList = self.heightList self.dataOut.flagNoData = False - - return self.dataOut.data + # return self.dataOut.data def run(self, **kwargs): @@ -729,6 +729,7 @@ class FitsReader(ProcessingUnit): self.getData() +@MPDecorator class SpectraHeisWriter(Operation): # set = None setFile = None @@ -736,8 +737,8 @@ class SpectraHeisWriter(Operation): doypath = None subfolder = None - def __init__(self, **kwargs): - Operation.__init__(self, **kwargs) + def __init__(self):#, **kwargs): + Operation.__init__(self)#, **kwargs) self.wrObj = PyFits() # self.dataOut = dataOut self.nTotalBlocks=0 @@ -846,3 +847,4 @@ class SpectraHeisWriter(Operation): self.isConfig = True self.putData() + return dataOut \ No newline at end of file diff --git a/schainpy/model/io/jroIO_hf.py b/schainpy/model/io/jroIO_hf.py index 5ba1787..2a89ab1 100644 --- a/schainpy/model/io/jroIO_hf.py +++ b/schainpy/model/io/jroIO_hf.py @@ -68,10 +68,10 @@ def getFileFromSet(path, ext, set=None): if set == None: return validFilelist[-1] - print "set =" ,set + print("set =" ,set) for thisFile in validFilelist: if set <= int(thisFile[6:16]): - print thisFile,int(thisFile[6:16]) + print(thisFile,int(thisFile[6:16])) return thisFile return validFilelist[-1] @@ -83,8 +83,8 @@ def getFileFromSet(path, ext, set=None): return myfile[0] else: filename = '*%10.10d%s'%(set,ext.lower()) - print 'the filename %s does not exist'%filename - print '...going to the last file: ' + print('the filename %s does not exist'%filename) + print('...going to the last file: ') if validFilelist: validFilelist = sorted( validFilelist, key=str.lower ) @@ -115,7 +115,7 @@ Depura el fileList dejando solo los que cumplan el formato de "res-xxxxxx.ext" try: number= int(thisFile[6:16]) except: - print "There is a file or folder with different format" + print("There is a file or folder with different format") if not isNumber(number): continue @@ -256,7 +256,7 @@ class HFReader(ProcessingUnit): self.status=1 else: self.status=0 - print 'Path %s does not exits'%self.path + print('Path %s does not exits'%self.path) return return @@ -282,12 +282,12 @@ class HFReader(ProcessingUnit): pat = '\d+.\d+' dirnameList = [re.search(pat,x) for x in os.listdir(self.path)] - dirnameList = filter(lambda x:x!=None,dirnameList) + dirnameList = [x for x in dirnameList if x!=None] dirnameList = [x.string for x in dirnameList] if not(online): dirnameList = [self.__selDates(x) for x in dirnameList] - dirnameList = filter(lambda x:x!=None,dirnameList) + dirnameList = [x for x in dirnameList if x!=None] if len(dirnameList)>0: self.status = 1 @@ -301,8 +301,8 @@ class HFReader(ProcessingUnit): def __getTimeFromData(self): startDateTime_Reader = datetime.datetime.combine(self.startDate,self.startTime) endDateTime_Reader = datetime.datetime.combine(self.endDate,self.endTime) - print 'Filtering Files from %s to %s'%(startDateTime_Reader, endDateTime_Reader) - print '........................................' + print('Filtering Files from %s to %s'%(startDateTime_Reader, endDateTime_Reader)) + print('........................................') filter_filenameList=[] self.filenameList.sort() for i in range(len(self.filenameList)-1): @@ -363,24 +363,24 @@ class HFReader(ProcessingUnit): self.flag_nextfile=False else: - print filename - print "PRIMERA CONDICION" + print(filename) + print("PRIMERA CONDICION") #if self.filename_next_set== int(filename[6:16]): - print "TODO BIEN" + print("TODO BIEN") if filename == None: - raise ValueError, "corregir" + raise ValueError("corregir") self.dirnameList=[filename] fullfilename=self.path+"/"+filename self.filenameList=[fullfilename] self.filename_next_set=int(filename[6:16])+10 - print "Setting next file",self.filename_next_set + print("Setting next file",self.filename_next_set) self.set=int(filename[6:16]) if True: pass else: - print "ESTOY AQUI PORQUE NO EXISTE EL SIGUIENTE ARCHIVO" + print("ESTOY AQUI PORQUE NO EXISTE EL SIGUIENTE ARCHIVO") else: filename =getlastFileFromPath(self.path,self.ext) @@ -394,24 +394,24 @@ class HFReader(ProcessingUnit): self.flag_nextfile=False else: filename=getFileFromSet(self.path,self.ext,self.set) - print filename - print "PRIMERA CONDICION" + print(filename) + print("PRIMERA CONDICION") #if self.filename_next_set== int(filename[6:16]): - print "TODO BIEN" + print("TODO BIEN") if filename == None: - raise ValueError, "corregir" + raise ValueError("corregir") self.dirnameList=[filename] fullfilename=self.path+"/"+filename self.filenameList=[fullfilename] self.filename_next_set=int(filename[6:16])+10 - print "Setting next file",self.filename_next_set + print("Setting next file",self.filename_next_set) self.set=int(filename[6:16]) if True: pass else: - print "ESTOY AQUI PORQUE NO EXISTE EL SIGUIENTE ARCHIVO" + print("ESTOY AQUI PORQUE NO EXISTE EL SIGUIENTE ARCHIVO") @@ -434,7 +434,7 @@ class HFReader(ProcessingUnit): self.__selectDataForTimes() for i in range(len(self.filenameList)): - print "%s"% (self.filenameList[i]) + print("%s"% (self.filenameList[i])) return @@ -456,7 +456,7 @@ class HFReader(ProcessingUnit): self.__checkPath() fullpath=path - print "%s folder was found: " %(fullpath ) + print("%s folder was found: " %(fullpath )) if set == None: self.set=None @@ -518,7 +518,7 @@ class HFReader(ProcessingUnit): idFile += 1 if not (idFile < len(self.filenameList)): self.flagNoMoreFiles = 1 - print "No more Files" + print("No more Files") return 0 filename = self.filenameList[idFile] hfFilePointer =h5py.File(filename,'r') @@ -534,14 +534,14 @@ class HFReader(ProcessingUnit): self.hfFilePointer = hfFilePointer hfFilePointer.close() self.__t0=epoc - print "Setting the file: %s"%self.filename + print("Setting the file: %s"%self.filename) return 1 def __setNextFileOnline(self): """ """ - print "SOY NONE",self.set + print("SOY NONE",self.set) if self.set==None: pass else: @@ -552,7 +552,7 @@ class HFReader(ProcessingUnit): self.__selectDataForTimes(online=True) filename = self.filenameList[0] while self.filename_online == filename: - print 'waiting %d seconds to get a new file...'%(self.__waitForNewFile) + print('waiting %d seconds to get a new file...'%(self.__waitForNewFile)) time.sleep(self.__waitForNewFile) #self.__findDataForDates(online=True) self.set=self.filename_next_set @@ -563,27 +563,27 @@ class HFReader(ProcessingUnit): #print filename sizeoffile=os.path.getsize(filename) if sizeoffile<1670240: - print "%s is not the rigth size"%filename + print("%s is not the rigth size"%filename) delay=50 - print 'waiting %d seconds for delay...'%(delay) + print('waiting %d seconds for delay...'%(delay)) time.sleep(delay) sizeoffile=os.path.getsize(filename) if sizeoffile<1670240: delay=50 - print 'waiting %d more seconds for delay...'%(delay) + print('waiting %d more seconds for delay...'%(delay)) time.sleep(delay) sizeoffile=os.path.getsize(filename) if sizeoffile<1670240: delay=50 - print 'waiting %d more seconds for delay...'%(delay) + print('waiting %d more seconds for delay...'%(delay)) time.sleep(delay) try: hfFilePointer=h5py.File(filename,'r') except: - print "Error reading file %s"%filename + print("Error reading file %s"%filename) self.filename_online=filename epoc=hfFilePointer['t'].value @@ -596,7 +596,7 @@ class HFReader(ProcessingUnit): self.flagIsNewFile = 1 self.filename = filename - print "Setting the file: %s"%self.filename + print("Setting the file: %s"%self.filename) return 1 def __getExpParameters(self): @@ -622,7 +622,7 @@ class HFReader(ProcessingUnit): ''' if path==None: - raise ValueError,"The path is not valid" + raise ValueError("The path is not valid") if ext==None: ext = self.ext @@ -634,11 +634,11 @@ class HFReader(ProcessingUnit): #print set if not(online): - print "Searching files in offline mode..." + print("Searching files in offline mode...") self.searchFilesOffLine(path, startDate, endDate, ext, startTime, endTime, walk) else: - print "Searching files in online mode..." + print("Searching files in online mode...") self.searchFilesOnLine(path, walk,ext,set=set) if set==None: pass @@ -659,7 +659,7 @@ class HFReader(ProcessingUnit): if not(self.filenameList): - print "There is no files into the folder: %s"%(path) + print("There is no files into the folder: %s"%(path)) sys.exit(-1) self.__getExpParameters() @@ -745,7 +745,7 @@ class HFReader(ProcessingUnit): self.dataOut.heightList = self.__firstHeigth + numpy.arange(self.__nSamples, dtype = numpy.float)*self.__deltaHeigth - self.dataOut.channelList = range(self.nChannels) + self.dataOut.channelList = list(range(self.nChannels)) #self.dataOut.channelIndexList = None @@ -833,7 +833,6 @@ class HFReader(ProcessingUnit): def getData(self): if self.flagNoMoreFiles: self.dataOut.flagNoData = True - print 'Process finished' return 0 if self.__hasNotDataInBuffer(): @@ -860,4 +859,4 @@ class HFReader(ProcessingUnit): if not self.isConfig: self.setup(**kwargs) self.isConfig = True - self.getData() + self.getData() \ No newline at end of file diff --git a/schainpy/model/io/jroIO_kamisr.py b/schainpy/model/io/jroIO_kamisr.py index 7e4d14a..da7adb1 100644 --- a/schainpy/model/io/jroIO_kamisr.py +++ b/schainpy/model/io/jroIO_kamisr.py @@ -111,7 +111,7 @@ class AMISRReader(ProcessingUnit): self.searchFilesOnLine(path, startDate, endDate, startTime,endTime,walk) if not(self.filenameList): - print "There is no files into the folder: %s"%(path) + print("There is no files into the folder: %s"%(path)) sys.exit(-1) @@ -177,7 +177,7 @@ class AMISRReader(ProcessingUnit): #filling system header parameters self.__nSamples = self.nsa self.newProfiles = self.nprofiles/self.nchannels - self.__channelList = range(self.nchannels) + self.__channelList = list(range(self.nchannels)) self.__frequency = self.frequency[0][0] @@ -200,7 +200,7 @@ class AMISRReader(ProcessingUnit): self.status = 1 else: self.status = 0 - print 'Path:%s does not exists'%self.path + print('Path:%s does not exists'%self.path) return @@ -225,11 +225,11 @@ class AMISRReader(ProcessingUnit): pat = '\d+.\d+' dirnameList = [re.search(pat,x) for x in os.listdir(self.path)] - dirnameList = filter(lambda x:x!=None,dirnameList) + dirnameList = [x for x in dirnameList if x!=None] dirnameList = [x.string for x in dirnameList] if not(online): dirnameList = [self.__selDates(x) for x in dirnameList] - dirnameList = filter(lambda x:x!=None,dirnameList) + dirnameList = [x for x in dirnameList if x!=None] if len(dirnameList)>0: self.status = 1 self.dirnameList = dirnameList @@ -242,8 +242,8 @@ class AMISRReader(ProcessingUnit): startDateTime_Reader = datetime.datetime.combine(self.startDate,self.startTime) endDateTime_Reader = datetime.datetime.combine(self.endDate,self.endTime) - print 'Filtering Files from %s to %s'%(startDateTime_Reader, endDateTime_Reader) - print '........................................' + print('Filtering Files from %s to %s'%(startDateTime_Reader, endDateTime_Reader)) + print('........................................') filter_filenameList = [] self.filenameList.sort() #for i in range(len(self.filenameList)-1): @@ -288,7 +288,7 @@ class AMISRReader(ProcessingUnit): def __getFilenameList(self, fileListInKeys, dirList): for value in fileListInKeys: - dirName = value.keys()[0] + dirName = list(value.keys())[0] for file in value[dirName]: filename = os.path.join(dirName, file) self.filenameList.append(filename) @@ -366,7 +366,7 @@ class AMISRReader(ProcessingUnit): self.__selectDataForTimes() for i in range(len(self.filenameList)): - print "%s" %(self.filenameList[i]) + print("%s" %(self.filenameList[i])) return @@ -377,7 +377,7 @@ class AMISRReader(ProcessingUnit): idFile += 1 if not(idFile < len(self.filenameList)): self.flagNoMoreFiles = 1 - print "No more Files" + print("No more Files") return 0 filename = self.filenameList[idFile] @@ -392,7 +392,7 @@ class AMISRReader(ProcessingUnit): self.amisrFilePointer = amisrFilePointer - print "Setting the file: %s"%self.filename + print("Setting the file: %s"%self.filename) return 1 @@ -404,7 +404,7 @@ class AMISRReader(ProcessingUnit): filename = self.filenameList[0] wait = 0 while self.__filename_online == filename: - print 'waiting %d seconds to get a new file...'%(self.__waitForNewFile) + print('waiting %d seconds to get a new file...'%(self.__waitForNewFile)) if wait == 5: return 0 sleep(self.__waitForNewFile) @@ -417,7 +417,7 @@ class AMISRReader(ProcessingUnit): self.amisrFilePointer = h5py.File(filename,'r') self.flagIsNewFile = 1 self.filename = filename - print "Setting the file: %s"%self.filename + print("Setting the file: %s"%self.filename) return 1 @@ -585,7 +585,6 @@ class AMISRReader(ProcessingUnit): if self.flagNoMoreFiles: self.dataOut.flagNoData = True - print 'Process finished' return 0 if self.__hasNotDataInBuffer(): @@ -628,48 +627,3 @@ class AMISRReader(ProcessingUnit): self.isConfig = True self.getData() - -class Writer(Operation): - ''' - classdocs - ''' - - def __init__(self): - ''' - Constructor - ''' - self.dataOut = None - - self.isConfig = False - - def setup(self, dataIn, path, blocksPerFile, set=0, ext=None): - ''' - In this method we should set all initial parameters. - - Input: - dataIn : Input data will also be outputa data - - ''' - self.dataOut = dataIn - - - - - - self.isConfig = True - - return - - def run(self, dataIn, **kwargs): - ''' - This method will be called many times so here you should put all your code - - Inputs: - - dataIn : object with the data - - ''' - - if not self.isConfig: - self.setup(dataIn, **kwargs) - \ No newline at end of file diff --git a/schainpy/model/io/jroIO_madrigal.py b/schainpy/model/io/jroIO_madrigal.py index eb764d0..485428c 100644 --- a/schainpy/model/io/jroIO_madrigal.py +++ b/schainpy/model/io/jroIO_madrigal.py @@ -14,32 +14,38 @@ import datetime import numpy import h5py -from schainpy.model.io.jroIO_base import JRODataReader -from schainpy.model.proc.jroproc_base import ProcessingUnit, Operation +import schainpy.admin +from schainpy.model.io.jroIO_base import LOCALTIME, Reader +from schainpy.model.proc.jroproc_base import ProcessingUnit, Operation, MPDecorator from schainpy.model.data.jrodata import Parameters from schainpy.utils import log try: import madrigal.cedar except: - log.warning( - 'You should install "madrigal library" module if you want to read/write Madrigal data' - ) + pass + +try: + basestring +except: + basestring = str DEF_CATALOG = { 'principleInvestigator': 'Marco Milla', - 'expPurpose': None, - 'cycleTime': None, - 'correlativeExp': None, - 'sciRemarks': None, - 'instRemarks': None + 'expPurpose': '', + 'cycleTime': '', + 'correlativeExp': '', + 'sciRemarks': '', + 'instRemarks': '' } + DEF_HEADER = { - 'kindatDesc': None, + 'kindatDesc': '', 'analyst': 'Jicamarca User', - 'comments': None, - 'history': None + 'comments': '', + 'history': '' } + MNEMONICS = { 10: 'jro', 11: 'jbr', @@ -48,6 +54,8 @@ MNEMONICS = { 1000: 'pbr', 1001: 'hbr', 1002: 'obr', + 400: 'clr' + } UT1970 = datetime.datetime(1970, 1, 1) - datetime.timedelta(seconds=time.timezone) @@ -63,112 +71,96 @@ def load_json(obj): iterable = obj if isinstance(iterable, dict): - return {str(k): load_json(v) if isinstance(v, dict) else str(v) if isinstance(v, unicode) else v - for k, v in iterable.items()} + return {str(k): load_json(v) if isinstance(v, dict) else str(v) if isinstance(v, basestring) else v + for k, v in list(iterable.items())} elif isinstance(iterable, (list, tuple)): - return [str(v) if isinstance(v, unicode) else v for v in iterable] + return [str(v) if isinstance(v, basestring) else v for v in iterable] return iterable -class MADReader(JRODataReader, ProcessingUnit): +class MADReader(Reader, ProcessingUnit): - def __init__(self, **kwargs): + def __init__(self): - ProcessingUnit.__init__(self, **kwargs) + ProcessingUnit.__init__(self) self.dataOut = Parameters() self.counter_records = 0 self.nrecords = None self.flagNoMoreFiles = 0 - self.isConfig = False self.filename = None self.intervals = set() + self.datatime = datetime.datetime(1900,1,1) + self.format = None + self.filefmt = "***%Y%m%d*******" - def setup(self, - path=None, - startDate=None, - endDate=None, - format=None, - startTime=datetime.time(0, 0, 0), - endTime=datetime.time(23, 59, 59), - **kwargs): + def setup(self, **kwargs): - self.path = path - self.startDate = startDate - self.endDate = endDate - self.startTime = startTime - self.endTime = endTime - self.datatime = datetime.datetime(1900,1,1) - self.oneDDict = load_json(kwargs.get('oneDDict', - "{\"GDLATR\":\"lat\", \"GDLONR\":\"lon\"}")) - self.twoDDict = load_json(kwargs.get('twoDDict', - "{\"GDALT\": \"heightList\"}")) - self.ind2DList = load_json(kwargs.get('ind2DList', - "[\"GDALT\"]")) + self.set_kwargs(**kwargs) + self.oneDDict = load_json(self.oneDDict) + self.twoDDict = load_json(self.twoDDict) + self.ind2DList = load_json(self.ind2DList) + self.independentParam = self.ind2DList[0] + if self.path is None: - raise ValueError, 'The path is not valid' + raise ValueError('The path is not valid') + + self.open_file = open + self.open_mode = 'rb' - if format is None: - raise ValueError, 'The format is not valid choose simple or hdf5' - elif format.lower() in ('simple', 'txt'): + if self.format is None: + raise ValueError('The format is not valid choose simple or hdf5') + elif self.format.lower() in ('simple', 'txt'): self.ext = '.txt' - elif format.lower() in ('cedar',): + elif self.format.lower() in ('cedar',): self.ext = '.001' else: self.ext = '.hdf5' + self.open_file = h5py.File + self.open_mode = 'r' - self.search_files(self.path) - self.fileId = 0 - - if not self.fileList: - raise Warning, 'There is no files matching these date in the folder: {}. \n Check startDate and endDate'.format(path) - - self.setNextFile() - - def search_files(self, path): - ''' - Searching for madrigal files in path - Creating a list of files to procces included in [startDate,endDate] - - Input: - path - Path to find files - ''' - - log.log('Searching files {} in {} '.format(self.ext, path), 'MADReader') - foldercounter = 0 - fileList0 = glob.glob1(path, '*{}'.format(self.ext)) - fileList0.sort() - - self.fileList = [] - self.dateFileList = [] - - startDate = self.startDate - datetime.timedelta(1) - endDate = self.endDate + datetime.timedelta(1) - - for thisFile in fileList0: - year = thisFile[3:7] - if not year.isdigit(): - continue - - month = thisFile[7:9] - if not month.isdigit(): - continue + if self.online: + log.log("Searching files in online mode...", self.name) - day = thisFile[9:11] - if not day.isdigit(): - continue + for nTries in range(self.nTries): + fullpath = self.searchFilesOnLine(self.path, self.startDate, + self.endDate, self.expLabel, self.ext, self.walk, + self.filefmt, self.folderfmt) - year, month, day = int(year), int(month), int(day) - dateFile = datetime.date(year, month, day) + try: + fullpath = next(fullpath) + except: + fullpath = None + + if fullpath: + break - if (startDate > dateFile) or (endDate < dateFile): - continue + log.warning( + 'Waiting {} sec for a valid file in {}: try {} ...'.format( + self.delay, self.path, nTries + 1), + self.name) + time.sleep(self.delay) + + if not(fullpath): + raise schainpy.admin.SchainError( + 'There isn\'t any valid file in {}'.format(self.path)) + + else: + log.log("Searching files in {}".format(self.path), self.name) + self.filenameList = self.searchFilesOffLine(self.path, self.startDate, + self.endDate, self.expLabel, self.ext, self.walk, self.filefmt, self.folderfmt) + + self.setNextFile() - self.fileList.append(thisFile) - self.dateFileList.append(dateFile) + def readFirstHeader(self): + '''Read header and data''' - return + self.parseHeader() + self.parseData() + self.blockIndex = 0 + + return def parseHeader(self): ''' @@ -178,27 +170,21 @@ class MADReader(JRODataReader, ProcessingUnit): self.version = '2' s_parameters = None if self.ext == '.txt': - self.parameters = [s.strip().lower() for s in self.fp.readline().strip().split(' ') if s] + self.parameters = [s.strip().lower() for s in self.fp.readline().decode().strip().split(' ') if s] elif self.ext == '.hdf5': - metadata = self.fp['Metadata'] - data = self.fp['Data']['Array Layout'] - if 'Independent Spatial Parameters' in metadata: - s_parameters = [s[0].lower() for s in metadata['Independent Spatial Parameters']] + self.metadata = self.fp['Metadata'] + if '_record_layout' in self.metadata: + s_parameters = [s[0].lower().decode() for s in self.metadata['Independent Spatial Parameters']] self.version = '3' - one = [s[0].lower() for s in data['1D Parameters']['Data Parameters']] - one_d = [1 for s in one] - two = [s[0].lower() for s in data['2D Parameters']['Data Parameters']] - two_d = [2 for s in two] - self.parameters = one + two - self.parameters_d = one_d + two_d - - log.success('Parameters found: {}'.format(','.join(self.parameters)), + self.parameters = [s[0].lower().decode() for s in self.metadata['Data Parameters']] + + log.success('Parameters found: {}'.format(self.parameters), 'MADReader') if s_parameters: - log.success('Spatial parameters: {}'.format(','.join(s_parameters)), + log.success('Spatial parameters found: {}'.format(s_parameters), 'MADReader') - for param in self.oneDDict.keys(): + for param in list(self.oneDDict.keys()): if param.lower() not in self.parameters: log.warning( 'Parameter {} not found will be ignored'.format( @@ -206,7 +192,7 @@ class MADReader(JRODataReader, ProcessingUnit): 'MADReader') self.oneDDict.pop(param, None) - for param, value in self.twoDDict.items(): + for param, value in list(self.twoDDict.items()): if param.lower() not in self.parameters: log.warning( 'Parameter {} not found, it will be ignored'.format( @@ -217,7 +203,7 @@ class MADReader(JRODataReader, ProcessingUnit): if isinstance(value, list): if value[0] not in self.output: self.output[value[0]] = [] - self.output[value[0]].append(None) + self.output[value[0]].append([]) def parseData(self): ''' @@ -226,57 +212,21 @@ class MADReader(JRODataReader, ProcessingUnit): if self.ext == '.txt': self.data = numpy.genfromtxt(self.fp, missing_values=('missing')) self.nrecords = self.data.shape[0] - self.ranges = numpy.unique(self.data[:,self.parameters.index(self.ind2DList[0].lower())]) + self.ranges = numpy.unique(self.data[:,self.parameters.index(self.independentParam.lower())]) + self.counter_records = 0 elif self.ext == '.hdf5': - self.data = self.fp['Data']['Array Layout'] - self.nrecords = len(self.data['timestamps'].value) - self.ranges = self.data['range'].value - - def setNextFile(self): - ''' - ''' - - file_id = self.fileId - - if file_id == len(self.fileList): - log.success('No more files', 'MADReader') - self.flagNoMoreFiles = 1 - return 0 - - log.success( - 'Opening: {}'.format(self.fileList[file_id]), - 'MADReader' - ) - - filename = os.path.join(self.path, self.fileList[file_id]) - - if self.filename is not None: - self.fp.close() - - self.filename = filename - self.filedate = self.dateFileList[file_id] - - if self.ext=='.hdf5': - self.fp = h5py.File(self.filename, 'r') - else: - self.fp = open(self.filename, 'rb') - - self.parseHeader() - self.parseData() - self.sizeOfFile = os.path.getsize(self.filename) - self.counter_records = 0 - self.flagIsNewFile = 0 - self.fileId += 1 - - return 1 + self.data = self.fp['Data'] + self.ranges = numpy.unique(self.data['Table Layout'][self.independentParam.lower()]) + self.times = numpy.unique(self.data['Table Layout']['ut1_unix']) + self.counter_records = int(self.data['Table Layout']['recno'][0]) + self.nrecords = int(self.data['Table Layout']['recno'][-1]) def readNextBlock(self): while True: self.flagDiscontinuousBlock = 0 - if self.flagIsNewFile: - if not self.setNextFile(): - return 0 + if self.counter_records == self.nrecords: + self.setNextFile() self.readBlock() @@ -316,31 +266,19 @@ class MADReader(JRODataReader, ProcessingUnit): dum.append(self.data[self.counter_records]) self.counter_records += 1 if self.counter_records == self.nrecords: - self.flagIsNewFile = True break continue self.intervals.add((datatime-self.datatime).seconds) break elif self.ext == '.hdf5': datatime = datetime.datetime.utcfromtimestamp( - self.data['timestamps'][self.counter_records]) - nHeights = len(self.ranges) - for n, param in enumerate(self.parameters): - if self.parameters_d[n] == 1: - dum.append(numpy.ones(nHeights)*self.data['1D Parameters'][param][self.counter_records]) - else: - if self.version == '2': - dum.append(self.data['2D Parameters'][param][self.counter_records]) - else: - tmp = self.data['2D Parameters'][param].value.T - dum.append(tmp[self.counter_records]) + self.times[self.counter_records]) + dum = self.data['Table Layout'][self.data['Table Layout']['recno']==self.counter_records] self.intervals.add((datatime-self.datatime).seconds) if datatime.date()>self.datatime.date(): self.flagDiscontinuousBlock = 1 self.datatime = datatime - self.counter_records += 1 - if self.counter_records == self.nrecords: - self.flagIsNewFile = True + self.counter_records += 1 self.buffer = numpy.array(dum) return @@ -352,33 +290,34 @@ class MADReader(JRODataReader, ProcessingUnit): parameters = [None for __ in self.parameters] - for param, attr in self.oneDDict.items(): + for param, attr in list(self.oneDDict.items()): x = self.parameters.index(param.lower()) setattr(self.dataOut, attr, self.buffer[0][x]) - for param, value in self.twoDDict.items(): - x = self.parameters.index(param.lower()) + for param, value in list(self.twoDDict.items()): + dummy = numpy.zeros(self.ranges.shape) + numpy.nan if self.ext == '.txt': - y = self.parameters.index(self.ind2DList[0].lower()) + x = self.parameters.index(param.lower()) + y = self.parameters.index(self.independentParam.lower()) ranges = self.buffer[:,y] - if self.ranges.size == ranges.size: - continue + #if self.ranges.size == ranges.size: + # continue index = numpy.where(numpy.in1d(self.ranges, ranges))[0] - dummy = numpy.zeros(self.ranges.shape) + numpy.nan dummy[index] = self.buffer[:,x] - else: - dummy = self.buffer[x] - + else: + ranges = self.buffer[self.independentParam.lower()] + index = numpy.where(numpy.in1d(self.ranges, ranges))[0] + dummy[index] = self.buffer[param.lower()] + if isinstance(value, str): - if value not in self.ind2DList: + if value not in self.independentParam: setattr(self.dataOut, value, dummy.reshape(1,-1)) elif isinstance(value, list): self.output[value[0]][value[1]] = dummy parameters[value[1]] = param - - for key, value in self.output.items(): + for key, value in list(self.output.items()): setattr(self.dataOut, key, numpy.array(value)) - + self.dataOut.parameters = [s for s in parameters if s] self.dataOut.heightList = self.ranges self.dataOut.utctime = (self.datatime - datetime.datetime(1970, 1, 1)).total_seconds() @@ -393,10 +332,6 @@ class MADReader(JRODataReader, ProcessingUnit): ''' Storing data from databuffer to dataOut object ''' - if self.flagNoMoreFiles: - self.dataOut.flagNoData = True - log.error('No file left to process', 'MADReader') - return 0 if not self.readNextBlock(): self.dataOut.flagNoData = True @@ -406,14 +341,53 @@ class MADReader(JRODataReader, ProcessingUnit): return 1 + def run(self, **kwargs): -class MADWriter(Operation): + if not(self.isConfig): + self.setup(**kwargs) + self.isConfig = True + + self.getData() - missing = -32767 + return + +@MPDecorator +class MADWriter(Operation): + '''Writing module for Madrigal files - def __init__(self, **kwargs): +type: external + +Inputs: + path path where files will be created + oneDDict json of one-dimensional parameters in record where keys + are Madrigal codes (integers or mnemonics) and values the corresponding + dataOut attribute e.g: { + 'gdlatr': 'lat', + 'gdlonr': 'lon', + 'gdlat2':'lat', + 'glon2':'lon'} + ind2DList list of independent spatial two-dimensional parameters e.g: + ['heigthList'] + twoDDict json of two-dimensional parameters in record where keys + are Madrigal codes (integers or mnemonics) and values the corresponding + dataOut attribute if multidimensional array specify as tupple + ('attr', pos) e.g: { + 'gdalt': 'heightList', + 'vn1p2': ('data_output', 0), + 'vn2p2': ('data_output', 1), + 'vn3': ('data_output', 2), + 'snl': ('data_SNR', 'db') + } + metadata json of madrigal metadata (kinst, kindat, catalog and header) + format hdf5, cedar + blocks number of blocks per file''' + + __attrs__ = ['path', 'oneDDict', 'ind2DList', 'twoDDict','metadata', 'format', 'blocks'] + missing = -32767 + + def __init__(self): - Operation.__init__(self, **kwargs) + Operation.__init__(self) self.dataOut = Parameters() self.counter = 0 self.path = None @@ -421,37 +395,14 @@ class MADWriter(Operation): def run(self, dataOut, path, oneDDict, ind2DList='[]', twoDDict='{}', metadata='{}', format='cedar', **kwargs): - ''' - Inputs: - path - path where files will be created - oneDDict - json of one-dimensional parameters in record where keys - are Madrigal codes (integers or mnemonics) and values the corresponding - dataOut attribute e.g: { - 'gdlatr': 'lat', - 'gdlonr': 'lon', - 'gdlat2':'lat', - 'glon2':'lon'} - ind2DList - list of independent spatial two-dimensional parameters e.g: - ['heighList'] - twoDDict - json of two-dimensional parameters in record where keys - are Madrigal codes (integers or mnemonics) and values the corresponding - dataOut attribute if multidimensional array specify as tupple - ('attr', pos) e.g: { - 'gdalt': 'heightList', - 'vn1p2': ('data_output', 0), - 'vn2p2': ('data_output', 1), - 'vn3': ('data_output', 2), - 'snl': ('data_SNR', 'db') - } - metadata - json of madrigal metadata (kinst, kindat, catalog and header) - ''' + if not self.isConfig: self.setup(path, oneDDict, ind2DList, twoDDict, metadata, format, **kwargs) self.isConfig = True self.dataOut = dataOut self.putData() - return + return 1 def setup(self, path, oneDDict, ind2DList, twoDDict, metadata, format, **kwargs): ''' @@ -495,7 +446,7 @@ class MADWriter(Operation): self.ext) self.fullname = os.path.join(self.path, filename) - + if os.path.isfile(self.fullname) : log.warning( 'Destination file {} already exists, previous file deleted.'.format( @@ -507,8 +458,10 @@ class MADWriter(Operation): log.success( 'Creating file: {}'.format(self.fullname), 'MADWriter') + if not os.path.exists(self.path): + os.makedirs(self.path) self.fp = madrigal.cedar.MadrigalCedarFile(self.fullname, True) - except ValueError, e: + except ValueError as e: log.error( 'Impossible to create a cedar object with "madrigal.cedar.MadrigalCedarFile"', 'MADWriter') @@ -528,7 +481,7 @@ class MADWriter(Operation): heights = self.dataOut.heightList if self.ext == '.dat': - for key, value in self.twoDDict.items(): + for key, value in list(self.twoDDict.items()): if isinstance(value, str): data = getattr(self.dataOut, value) invalid = numpy.isnan(data) @@ -540,7 +493,7 @@ class MADWriter(Operation): data[invalid] = self.missing out = {} - for key, value in self.twoDDict.items(): + for key, value in list(self.twoDDict.items()): key = key.lower() if isinstance(value, str): if 'db' in value.lower(): @@ -549,12 +502,12 @@ class MADWriter(Operation): tmp = 10*numpy.log10(SNRavg) else: tmp = getattr(self.dataOut, value) - out[key] = tmp.flatten() + out[key] = tmp.flatten()[:len(heights)] elif isinstance(value, (tuple, list)): attr, x = value - data = getattr(self.dataOut, attr) - out[key] = data[int(x)] - + data = getattr(self.dataOut, attr) + out[key] = data[int(x)][:len(heights)] + a = numpy.array([out[k] for k in self.keys]) nrows = numpy.array([numpy.isnan(a[:, x]).all() for x in range(len(heights))]) index = numpy.where(nrows == False)[0] @@ -576,8 +529,8 @@ class MADWriter(Operation): endTime.minute, endTime.second, endTime.microsecond/10000, - self.oneDDict.keys(), - self.twoDDict.keys(), + list(self.oneDDict.keys()), + list(self.twoDDict.keys()), len(index), **self.extra_args ) @@ -596,7 +549,7 @@ class MADWriter(Operation): self.fp.append(rec) if self.ext == '.hdf5' and self.counter % 500 == 0 and self.counter > 0: self.fp.dump() - if self.counter % 100 == 0 and self.counter > 0: + if self.counter % 20 == 0 and self.counter > 0: log.log( 'Writing {} records'.format( self.counter), @@ -639,4 +592,4 @@ class MADWriter(Operation): def close(self): if self.counter > 0: - self.setHeader() + self.setHeader() \ No newline at end of file diff --git a/schainpy/model/io/jroIO_matlab.py b/schainpy/model/io/jroIO_matlab.py index 4e7512c..175ba9a 100644 --- a/schainpy/model/io/jroIO_matlab.py +++ b/schainpy/model/io/jroIO_matlab.py @@ -29,7 +29,7 @@ class matoffReader(ProcessingUnit): def __setHeader(self, datastuff): self.dataOut.pairsList=[(0,1)] - self.dataOut.channelList = range(np.array(datastuff.get('power')).shape[1]) + self.dataOut.channelList = list(range(np.array(datastuff.get('power')).shape[1])) self.dataOut.nProfiles = len(np.array(datastuff.get('vel')).flatten()) #this! self.dataOut.nIncohInt = 20 self.dataOut.nCohInt = 1 #this! @@ -39,7 +39,7 @@ class matoffReader(ProcessingUnit): self.dataOut.heightList = np.array(datastuff.get('hts')).flatten() def __readFile(self, currentfile): - print "Reading from this file:" + currentfile + print("Reading from this file:" + currentfile) #filesplit=currentfile.split("\\") filesplit=currentfile.split("/") @@ -64,7 +64,7 @@ class matoffReader(ProcessingUnit): # self.utcmatcounter=0 # print self.utcmatcounter - print self.utcfirst + print(self.utcfirst) try: datastuff=sio.loadmat(currentfile) except: @@ -115,7 +115,7 @@ class matoffReader(ProcessingUnit): utclist=[] if not dirList: - print "No directories found" + print("No directories found") return [] #if self.online: @@ -146,7 +146,7 @@ class matoffReader(ProcessingUnit): utclist.append(utctime) if not dirListFiltered: - print "filtro" + print("filtro") return [] for thisDir in dirListFiltered: @@ -188,7 +188,7 @@ class matoffReader(ProcessingUnit): if nTries > 3: break - print "Waiting %d seconds ..." %seconds + print("Waiting %d seconds ..." %seconds) time.sleep(40) if not (len(filelist) > ncurrentfiles): @@ -227,7 +227,7 @@ class matoffReader(ProcessingUnit): self.fileList = fileList - print "fin setup" + print("fin setup") def run(self,path=None,startDate=None, endDate=None, startTime=datetime.time(0,0,0), @@ -251,7 +251,7 @@ class matoffReader(ProcessingUnit): if not self.fileList: self.dataOut.flagNoData = True - print "lista vacia" + print("lista vacia") return currentfile = self.__getNextFile() diff --git a/schainpy/model/io/jroIO_mira35c.py b/schainpy/model/io/jroIO_mira35c.py index 5ecc67a..19acf20 100644 --- a/schainpy/model/io/jroIO_mira35c.py +++ b/schainpy/model/io/jroIO_mira35c.py @@ -48,7 +48,7 @@ class Header(object): message += self.__class__.__name__.upper() + "\n" message += "#" * 50 + "\n" - keyList = self.__dict__.keys() + keyList = list(self.__dict__.keys()) keyList.sort() for key in keyList: @@ -333,7 +333,7 @@ class SRVIHeader(Header): self.DataBlockTitleSRVI1 = str(header['DataBlockTitleSRVI1'][0]) self.SizeOfSRVI1 = header['SizeOfSRVI1'][0] # 16 - print 'Pointer fp SRVIheader', fp.tell() + print('Pointer fp SRVIheader', fp.tell()) SRVI_STRUCTURE = numpy.dtype([ @@ -435,9 +435,9 @@ class RecordHeader(Header): # print 'Datasize',self.Datasize #endFp = self.OffsetStartHeader + self.RecCounter*self.Off2StartNxtRec - print '==============================================' + print('==============================================') - print '==============================================' + print('==============================================') return 1 @@ -572,7 +572,7 @@ class MIRA35CReader (ProcessingUnit, FileHeaderMIRA35c, SRVIHeader, RecordHeader if self.flagNoMoreFiles: self.dataOut.flagNoData = True - print 'NoData se vuelve true' + print('NoData se vuelve true') return 0 self.fp = self.path @@ -602,7 +602,7 @@ class MIRA35CReader (ProcessingUnit, FileHeaderMIRA35c, SRVIHeader, RecordHeader ''' # The address of the folder is generated the name of the .fdt file that will be read - print "File: ", self.fileSelector + 1 + print("File: ", self.fileSelector + 1) if self.fileSelector < len(self.filenameList): @@ -642,7 +642,7 @@ class MIRA35CReader (ProcessingUnit, FileHeaderMIRA35c, SRVIHeader, RecordHeader self.readBlock() # Block reading else: - print 'readFile FlagNoData becomes true' + print('readFile FlagNoData becomes true') self.flagNoMoreFiles = True self.dataOut.flagNoData = True self.FileHeaderFlag == True @@ -673,7 +673,7 @@ class MIRA35CReader (ProcessingUnit, FileHeaderMIRA35c, SRVIHeader, RecordHeader self.blocksize = self.srviHeader.SizeOfDataBlock1 # Se obtiene el tamao del bloque if self.blocksize == 148: - print 'blocksize == 148 bug' + print('blocksize == 148 bug') jump = numpy.fromfile(self.fp, [('jump', numpy.str_, 140)], 1) # Se obtiene la cabecera del SRVI @@ -691,7 +691,7 @@ class MIRA35CReader (ProcessingUnit, FileHeaderMIRA35c, SRVIHeader, RecordHeader npw1 = self.recordheader.npw1 npw2 = self.recordheader.npw2 - self.dataOut.channelList = range(1) + self.dataOut.channelList = list(range(1)) self.dataOut.nIncohInt = self.Num_inCoh self.dataOut.nProfiles = self.Num_Bins self.dataOut.nCohInt = 1 @@ -701,7 +701,7 @@ class MIRA35CReader (ProcessingUnit, FileHeaderMIRA35c, SRVIHeader, RecordHeader self.dataOut.outputInterval = self.dataOut.getTimeInterval() self.dataOut.heightList = self.SPARrawGate1 * self.__deltaHeigth + \ - numpy.array(range(self.Num_Hei)) * self.__deltaHeigth + numpy.array(list(range(self.Num_Hei))) * self.__deltaHeigth self.HSDVsign = numpy.fromfile(self.fp, [('HSDV', numpy.str_, 4)], 1) self.SizeHSDV = numpy.fromfile(self.fp, [('SizeHSDV', ' 1: - print "[Reading] %d days were found in date range: %s - %s" %(len(dateList), startDate, endDate) - else: - print "[Reading] data was found for the date %s" %(dateList[0]) - - filenameList = [] - datetimeList = [] - - #---------------------------------------------------------------------------------- - - for thisPath in pathList: -# thisPath = pathList[pathDict[file]] - - fileList = glob.glob1(thisPath, "*%s" %ext) - fileList.sort() - - for file in fileList: - - filename = os.path.join(thisPath,file) - - if not isFileInDateRange(filename, startDate, endDate): - continue - - thisDatetime = self.__isFileInTimeRange(filename, startDate, endDate, startTime, endTime) - - if not(thisDatetime): - continue - - filenameList.append(filename) - datetimeList.append(thisDatetime) - - if not(filenameList): - print "[Reading] Any file was found int time range %s - %s" %(datetime.datetime.combine(startDate,startTime).ctime(), datetime.datetime.combine(endDate,endTime).ctime()) - return None, None - - print "[Reading] %d file(s) was(were) found in time range: %s - %s" %(len(filenameList), startTime, endTime) - print - -# for i in range(len(filenameList)): -# print "[Reading] %s -> [%s]" %(filenameList[i], datetimeList[i].ctime()) - - self.filenameList = filenameList - self.datetimeList = datetimeList - - return pathList, filenameList - - def __isFileInTimeRange(self,filename, startDate, endDate, startTime, endTime): - - """ - Retorna 1 si el archivo de datos se encuentra dentro del rango de horas especificado. - - Inputs: - filename : nombre completo del archivo de datos en formato Jicamarca (.r) - - startDate : fecha inicial del rango seleccionado en formato datetime.date - - endDate : fecha final del rango seleccionado en formato datetime.date - - startTime : tiempo inicial del rango seleccionado en formato datetime.time - - endTime : tiempo final del rango seleccionado en formato datetime.time + def readFirstHeader(self): + '''Read metadata and data''' - Return: - Boolean : Retorna True si el archivo de datos contiene datos en el rango de - fecha especificado, de lo contrario retorna False. - - Excepciones: - Si el archivo no existe o no puede ser abierto - Si la cabecera no puede ser leida. - - """ - - try: - fp = h5py.File(filename,'r') - grp1 = fp['Data'] - - except IOError: - traceback.print_exc() - raise IOError, "The file %s can't be opened" %(filename) - #chino rata - #In case has utctime attribute - grp2 = grp1['utctime'] -# thisUtcTime = grp2.value[0] - 5*3600 #To convert to local time - thisUtcTime = grp2.value[0] - - fp.close() - - if self.timezone == 'lt': - thisUtcTime -= 5*3600 - - thisDatetime = datetime.datetime.fromtimestamp(thisUtcTime[0] + 5*3600) -# thisDatetime = datetime.datetime.fromtimestamp(thisUtcTime[0]) - thisDate = thisDatetime.date() - thisTime = thisDatetime.time() - - startUtcTime = (datetime.datetime.combine(thisDate,startTime)- datetime.datetime(1970, 1, 1)).total_seconds() - endUtcTime = (datetime.datetime.combine(thisDate,endTime)- datetime.datetime(1970, 1, 1)).total_seconds() - - #General case - # o>>>>>>>>>>>>>><<<<<<<<<<<<<= startTime: - thisUtcLog = numpy.logical_and(thisUtcTime > startUtcTime, thisUtcTime < endUtcTime) - if numpy.any(thisUtcLog): #If there is one block between the hours mentioned - return thisDatetime - return None - - #If endTime < startTime then endTime belongs to the next day - #<<<<<<<<<<>>>>>>>>>> - #-----------o----------------------------o----------- - # endTime startTime - - if (thisDate == startDate) and numpy.all(thisUtcTime < startUtcTime): - return None - - if (thisDate == endDate) and numpy.all(thisUtcTime > endUtcTime): - return None - - if numpy.all(thisUtcTime < startUtcTime) and numpy.all(thisUtcTime > endUtcTime): - return None - - return thisDatetime - - def __setNextFileOffline(self): - - self.fileIndex += 1 - idFile = self.fileIndex - - if not(idFile < len(self.filenameList)): - print "No more Files" - return 0 - - filename = self.filenameList[idFile] - - filePointer = h5py.File(filename,'r') - - self.filename = filename - - self.fp = filePointer - - print "Setting the file: %s"%self.filename - -# self.__readMetadata() - self.__setBlockList() + self.__readMetadata() self.__readData() -# self.nRecords = self.fp['Data'].attrs['blocksPerFile'] -# self.nRecords = self.fp['Data'].attrs['nRecords'] + self.__setBlockList() + + if 'type' in self.meta: + self.dataOut = eval(self.meta['type'])() + + for attr in self.meta: + setattr(self.dataOut, attr, self.meta[attr]) + self.blockIndex = 0 - return 1 + + return def __setBlockList(self): ''' @@ -299,511 +162,250 @@ class ParamReader(ProcessingUnit): self.fp self.startTime self.endTime - self.blockList self.blocksPerFile ''' - fp = self.fp + startTime = self.startTime endTime = self.endTime - grp = fp['Data'] - thisUtcTime = grp['utctime'].value.astype(numpy.float)[0] - - #ERROOOOR - if self.timezone == 'lt': - thisUtcTime -= 5*3600 + thisUtcTime = self.data['utctime'] + self.interval = numpy.min(thisUtcTime[1:] - thisUtcTime[:-1]) - thisDatetime = datetime.datetime.fromtimestamp(thisUtcTime[0] + 5*3600) + thisDatetime = datetime.datetime.utcfromtimestamp(thisUtcTime[0]) thisDate = thisDatetime.date() thisTime = thisDatetime.time() - startUtcTime = (datetime.datetime.combine(thisDate,startTime) - datetime.datetime(1970, 1, 1)).total_seconds() - endUtcTime = (datetime.datetime.combine(thisDate,endTime) - datetime.datetime(1970, 1, 1)).total_seconds() + startUtcTime = (datetime.datetime.combine(thisDate, startTime) - datetime.datetime(1970, 1, 1)).total_seconds() + endUtcTime = (datetime.datetime.combine(thisDate, endTime) - datetime.datetime(1970, 1, 1)).total_seconds() ind = numpy.where(numpy.logical_and(thisUtcTime >= startUtcTime, thisUtcTime < endUtcTime))[0] self.blockList = ind self.blocksPerFile = len(ind) - return def __readMetadata(self): ''' Reads Metadata - - self.pathMeta - - self.listShapes - self.listMetaname - self.listMeta - ''' -# grp = self.fp['Data'] -# pathMeta = os.path.join(self.path, grp.attrs['metadata']) -# -# if pathMeta == self.pathMeta: -# return -# else: -# self.pathMeta = pathMeta -# -# filePointer = h5py.File(self.pathMeta,'r') -# groupPointer = filePointer['Metadata'] - - filename = self.filenameList[0] - - fp = h5py.File(filename,'r') - - gp = fp['Metadata'] - - listMetaname = [] - listMetadata = [] - for item in gp.items(): - name = item[0] - - if name=='array dimensions': - table = gp[name][:] - listShapes = {} - for shapes in table: - listShapes[shapes[0]] = numpy.array([shapes[1],shapes[2],shapes[3],shapes[4],shapes[5]]) - else: - data = gp[name].value - listMetaname.append(name) - listMetadata.append(data) + meta = {} -# if name=='type': -# self.__initDataOut(data) + if self.description: + for key, value in self.description['Metadata'].items(): + meta[key] = self.fp[value].value + else: + grp = self.fp['Metadata'] + for name in grp: + meta[name] = grp[name].value - self.listShapes = listShapes - self.listMetaname = listMetaname - self.listMeta = listMetadata + if self.extras: + for key, value in self.extras.items(): + meta[key] = value + self.meta = meta - fp.close() return def __readData(self): - grp = self.fp['Data'] - listdataname = [] - listdata = [] - - for item in grp.items(): - name = item[0] - listdataname.append(name) - - array = self.__setDataArray(grp[name],self.listShapes[name]) - listdata.append(array) - - self.listDataname = listdataname - self.listData = listdata - return - - def __setDataArray(self, dataset, shapes): - - nDims = shapes[0] - - nDim2 = shapes[1] #Dimension 0 - - nDim1 = shapes[2] #Dimension 1, number of Points or Parameters - - nDim0 = shapes[3] #Dimension 2, number of samples or ranges - - mode = shapes[4] #Mode of storing - blockList = self.blockList - - blocksPerFile = self.blocksPerFile - - #Depending on what mode the data was stored - if mode == 0: #Divided in channels - arrayData = dataset.value.astype(numpy.float)[0][blockList] - if mode == 1: #Divided in parameter - strds = 'table' - nDatas = nDim1 - newShapes = (blocksPerFile,nDim2,nDim0) - elif mode==2: #Concatenated in a table - strds = 'table0' - arrayData = dataset[strds].value - #Selecting part of the dataset - utctime = arrayData[:,0] - u, indices = numpy.unique(utctime, return_index=True) - - if blockList.size != indices.size: - indMin = indices[blockList[0]] - if blockList[1] + 1 >= indices.size: - arrayData = arrayData[indMin:,:] + data = {} + + if self.description: + for key, value in self.description['Data'].items(): + if isinstance(value, str): + if isinstance(self.fp[value], h5py.Dataset): + data[key] = self.fp[value].value + elif isinstance(self.fp[value], h5py.Group): + array = [] + for ch in self.fp[value]: + array.append(self.fp[value][ch].value) + data[key] = numpy.array(array) + elif isinstance(value, list): + array = [] + for ch in value: + array.append(self.fp[ch].value) + data[key] = numpy.array(array) + else: + grp = self.fp['Data'] + for name in grp: + if isinstance(grp[name], h5py.Dataset): + array = grp[name].value + elif isinstance(grp[name], h5py.Group): + array = [] + for ch in grp[name]: + array.append(grp[name][ch].value) + array = numpy.array(array) else: - indMax = indices[blockList[1] + 1] - arrayData = arrayData[indMin:indMax,:] - return arrayData - - # One dimension - if nDims == 0: - arrayData = dataset.value.astype(numpy.float)[0][blockList] - - # Two dimensions - elif nDims == 2: - arrayData = numpy.zeros((blocksPerFile,nDim1,nDim0)) - newShapes = (blocksPerFile,nDim0) - nDatas = nDim1 - - for i in range(nDatas): - data = dataset[strds + str(i)].value - arrayData[:,i,:] = data[blockList,:] + log.warning('Unknown type: {}'.format(name)) + + if name in self.description: + key = self.description[name] + else: + key = name + data[key] = array - # Three dimensions - else: - arrayData = numpy.zeros((blocksPerFile,nDim2,nDim1,nDim0)) - for i in range(nDatas): - - data = dataset[strds + str(i)].value - - for b in range(blockList.size): - arrayData[b,:,i,:] = data[:,:,blockList[b]] - - return arrayData - - def __setDataOut(self): - listMeta = self.listMeta - listMetaname = self.listMetaname - listDataname = self.listDataname - listData = self.listData - listShapes = self.listShapes - - blockIndex = self.blockIndex -# blockList = self.blockList - - for i in range(len(listMeta)): - setattr(self.dataOut,listMetaname[i],listMeta[i]) - - for j in range(len(listData)): - nShapes = listShapes[listDataname[j]][0] - mode = listShapes[listDataname[j]][4] - if nShapes == 1: - setattr(self.dataOut,listDataname[j],listData[j][blockIndex]) - elif nShapes > 1: - setattr(self.dataOut,listDataname[j],listData[j][blockIndex,:]) - elif mode==0: - setattr(self.dataOut,listDataname[j],listData[j][blockIndex]) - #Mode Meteors - elif mode ==2: - selectedData = self.__selectDataMode2(listData[j], blockIndex) - setattr(self.dataOut, listDataname[j], selectedData) + self.data = data return - - def __selectDataMode2(self, data, blockIndex): - utctime = data[:,0] - aux, indices = numpy.unique(utctime, return_inverse=True) - selInd = numpy.where(indices == blockIndex)[0] - selData = data[selInd,:] - - return selData - + def getData(self): -# if self.flagNoMoreFiles: -# self.dataOut.flagNoData = True -# print 'Process finished' -# return 0 -# - if self.blockIndex==self.blocksPerFile: - if not( self.__setNextFileOffline() ): - self.dataOut.flagNoData = True - return 0 - -# if self.datablock == None: # setear esta condicion cuando no hayan datos por leers -# self.dataOut.flagNoData = True -# return 0 -# self.__readData() - self.__setDataOut() - self.dataOut.flagNoData = False + for attr in self.data: + if self.data[attr].ndim == 1: + setattr(self.dataOut, attr, self.data[attr][self.blockIndex]) + else: + setattr(self.dataOut, attr, self.data[attr][:, self.blockIndex]) + self.dataOut.flagNoData = False self.blockIndex += 1 + log.log("Block No. {}/{} -> {}".format( + self.blockIndex, + self.blocksPerFile, + self.dataOut.datatime.ctime()), self.name) + return def run(self, **kwargs): if not(self.isConfig): self.setup(**kwargs) -# self.setObjProperties() self.isConfig = True + if self.blockIndex == self.blocksPerFile: + self.setNextFile() + self.getData() return -class ParamWriter(Operation): - ''' - HDF5 Writer, stores parameters data in HDF5 format files - - path: path where the files will be stored - - blocksPerFile: number of blocks that will be saved in per HDF5 format file - - mode: selects the data stacking mode: '0' channels, '1' parameters, '3' table (for meteors) - - metadataList: list of attributes that will be stored as metadata - - dataList: list of attributes that will be stores as data - - ''' - +@MPDecorator +class HDFWriter(Operation): + """Operation to write HDF5 files. + + The HDF5 file contains by default two groups Data and Metadata where + you can save any `dataOut` attribute specified by `dataList` and `metadataList` + parameters, data attributes are normaly time dependent where the metadata + are not. + It is possible to customize the structure of the HDF5 file with the + optional description parameter see the examples. + + Parameters: + ----------- + path : str + Path where files will be saved. + blocksPerFile : int + Number of blocks per file + metadataList : list + List of the dataOut attributes that will be saved as metadata + dataList : int + List of the dataOut attributes that will be saved as data + setType : bool + If True the name of the files corresponds to the timestamp of the data + description : dict, optional + Dictionary with the desired description of the HDF5 file + + Examples + -------- + + desc = { + 'data_output': {'winds': ['z', 'w', 'v']}, + 'utctime': 'timestamps', + 'heightList': 'heights' + } + desc = { + 'data_output': ['z', 'w', 'v'], + 'utctime': 'timestamps', + 'heightList': 'heights' + } + desc = { + 'Data': { + 'data_output': 'winds', + 'utctime': 'timestamps' + }, + 'Metadata': { + 'heightList': 'heights' + } + } + + writer = proc_unit.addOperation(name='HDFWriter') + writer.addParameter(name='path', value='/path/to/file') + writer.addParameter(name='blocksPerFile', value='32') + writer.addParameter(name='metadataList', value='heightList,timeZone') + writer.addParameter(name='dataList',value='data_output,utctime') + # writer.addParameter(name='description',value=json.dumps(desc)) + + """ ext = ".hdf5" - optchar = "D" - - metaoptchar = "M" - - metaFile = None - filename = None - path = None - setFile = None - fp = None - - grp = None - - ds = None - firsttime = True - #Configurations - blocksPerFile = None - blockIndex = None - dataOut = None - #Data Arrays - dataList = None - metadataList = None - -# arrayDim = None - - dsList = None #List of dictionaries with dataset properties - - tableDim = None - -# dtype = [('arrayName', 'S20'),('nChannels', 'i'), ('nPoints', 'i'), ('nSamples', 'i'),('mode', 'b')] - - dtype = [('arrayName', 'S20'),('nDimensions', 'i'), ('dim2', 'i'), ('dim1', 'i'),('dim0', 'i'),('mode', 'b')] - currentDay = None - lastTime = None - setType = None - - def __init__(self, **kwargs): - Operation.__init__(self, **kwargs) - self.isConfig = False + def __init__(self): + + Operation.__init__(self) return - def setup(self, dataOut, path=None, blocksPerFile=10, metadataList=None, dataList=None, mode=None, **kwargs): + def setup(self, path=None, blocksPerFile=10, metadataList=None, dataList=None, setType=None, description=None): self.path = path self.blocksPerFile = blocksPerFile self.metadataList = metadataList self.dataList = dataList - self.dataOut = dataOut - self.mode = mode - - if self.mode is not None: - self.mode = numpy.zeros(len(self.dataList)) + mode - else: - self.mode = numpy.ones(len(self.dataList)) + self.setType = setType + self.description = description - arrayDim = numpy.zeros((len(self.dataList),5)) + for s in ['type', 'timeZone', 'useLocalTime']: + if s not in self.metadataList: + self.metadataList.append(s) - #Table dimensions - dtype0 = self.dtype tableList = [] - - #Dictionary and list of tables dsList = [] for i in range(len(self.dataList)): dsDict = {} dataAux = getattr(self.dataOut, self.dataList[i]) dsDict['variable'] = self.dataList[i] - #--------------------- Conditionals ------------------------ - #There is no data - if dataAux is None: - return 0 - #Not array, just a number - #Mode 0 - if type(dataAux)==float or type(dataAux)==int: - dsDict['mode'] = 0 - dsDict['nDim'] = 0 - arrayDim[i,0] = 0 - dsList.append(dsDict) - - #Mode 2: meteors - elif mode[i] == 2: -# dsDict['nDim'] = 0 - dsDict['dsName'] = 'table0' - dsDict['mode'] = 2 # Mode meteors - dsDict['shape'] = dataAux.shape[-1] + if dataAux is None: + continue + elif isinstance(dataAux, (int, float, numpy.integer, numpy.float)): dsDict['nDim'] = 0 - dsDict['dsNumber'] = 1 - - arrayDim[i,3] = dataAux.shape[-1] - arrayDim[i,4] = mode[i] #Mode the data was stored - - dsList.append(dsDict) - - #Mode 1 - else: - arrayDim0 = dataAux.shape #Data dimensions - arrayDim[i,0] = len(arrayDim0) #Number of array dimensions - arrayDim[i,4] = mode[i] #Mode the data was stored - - strtable = 'table' - dsDict['mode'] = 1 # Mode parameters - - # Three-dimension arrays - if len(arrayDim0) == 3: - arrayDim[i,1:-1] = numpy.array(arrayDim0) - nTables = int(arrayDim[i,2]) - dsDict['dsNumber'] = nTables - dsDict['shape'] = arrayDim[i,2:4] - dsDict['nDim'] = 3 - - for j in range(nTables): - dsDict = dsDict.copy() - dsDict['dsName'] = strtable + str(j) - dsList.append(dsDict) - - # Two-dimension arrays - elif len(arrayDim0) == 2: - arrayDim[i,2:-1] = numpy.array(arrayDim0) - nTables = int(arrayDim[i,2]) - dsDict['dsNumber'] = nTables - dsDict['shape'] = arrayDim[i,3] - dsDict['nDim'] = 2 - - for j in range(nTables): - dsDict = dsDict.copy() - dsDict['dsName'] = strtable + str(j) - dsList.append(dsDict) - - # One-dimension arrays - elif len(arrayDim0) == 1: - arrayDim[i,3] = arrayDim0[0] - dsDict['shape'] = arrayDim0[0] - dsDict['dsNumber'] = 1 - dsDict['dsName'] = strtable + str(0) - dsDict['nDim'] = 1 - dsList.append(dsDict) - - table = numpy.array((self.dataList[i],) + tuple(arrayDim[i,:]),dtype = dtype0) - tableList.append(table) - -# self.arrayDim = arrayDim - self.dsList = dsList - self.tableDim = numpy.array(tableList, dtype = dtype0) - self.blockIndex = 0 - - timeTuple = time.localtime(dataOut.utctime) - self.currentDay = timeTuple.tm_yday - return 1 - - def putMetadata(self): - - fp = self.createMetadataFile() - self.writeMetadata(fp) - fp.close() - return - - def createMetadataFile(self): - ext = self.ext - path = self.path - setFile = self.setFile - - timeTuple = time.localtime(self.dataOut.utctime) - - subfolder = '' - fullpath = os.path.join( path, subfolder ) - - if not( os.path.exists(fullpath) ): - os.mkdir(fullpath) - setFile = -1 #inicializo mi contador de seteo - - subfolder = 'd%4.4d%3.3d' % (timeTuple.tm_year,timeTuple.tm_yday) - fullpath = os.path.join( path, subfolder ) - - if not( os.path.exists(fullpath) ): - os.mkdir(fullpath) - setFile = -1 #inicializo mi contador de seteo - - else: - filesList = os.listdir( fullpath ) - filesList = sorted( filesList, key=str.lower ) - if len( filesList ) > 0: - filesList = [k for k in filesList if 'M' in k] - filen = filesList[-1] - # el filename debera tener el siguiente formato - # 0 1234 567 89A BCDE (hex) - # x YYYY DDD SSS .ext - if isNumber( filen[8:11] ): - setFile = int( filen[8:11] ) #inicializo mi contador de seteo al seteo del ultimo file - else: - setFile = -1 else: - setFile = -1 #inicializo mi contador de seteo - - if self.setType is None: - setFile += 1 - file = '%s%4.4d%3.3d%03d%s' % (self.metaoptchar, - timeTuple.tm_year, - timeTuple.tm_yday, - setFile, - ext ) - else: - setFile = timeTuple.tm_hour*60+timeTuple.tm_min - file = '%s%4.4d%3.3d%04d%s' % (self.metaoptchar, - timeTuple.tm_year, - timeTuple.tm_yday, - setFile, - ext ) - - filename = os.path.join( path, subfolder, file ) - self.metaFile = file - #Setting HDF5 File - fp = h5py.File(filename,'w') - - return fp - - def writeMetadata(self, fp): + dsDict['nDim'] = len(dataAux.shape) + dsDict['shape'] = dataAux.shape + dsDict['dsNumber'] = dataAux.shape[0] + dsDict['dtype'] = dataAux.dtype + + dsList.append(dsDict) - grp = fp.create_group("Metadata") - grp.create_dataset('array dimensions', data = self.tableDim, dtype = self.dtype) - - for i in range(len(self.metadataList)): - grp.create_dataset(self.metadataList[i], data=getattr(self.dataOut, self.metadataList[i])) - return + self.dsList = dsList + self.currentDay = self.dataOut.datatime.date() def timeFlag(self): currentTime = self.dataOut.utctime - - if self.lastTime is None: - self.lastTime = currentTime - - #Day timeTuple = time.localtime(currentTime) dataDay = timeTuple.tm_yday - #Time + if self.lastTime is None: + self.lastTime = currentTime + self.currentDay = dataDay + return False + timeDiff = currentTime - self.lastTime #Si el dia es diferente o si la diferencia entre un dato y otro supera la hora @@ -817,29 +419,42 @@ class ParamWriter(Operation): self.lastTime = currentTime return False - def setNextFile(self): + def run(self, dataOut, path, blocksPerFile=10, metadataList=[], + dataList=[], setType=None, description={}): + + self.dataOut = dataOut + if not(self.isConfig): + self.setup(path=path, blocksPerFile=blocksPerFile, + metadataList=metadataList, dataList=dataList, + setType=setType, description=description) + self.isConfig = True + self.setNextFile() + + self.putData() + return + + def setNextFile(self): + ext = self.ext path = self.path setFile = self.setFile - mode = self.mode timeTuple = time.localtime(self.dataOut.utctime) subfolder = 'd%4.4d%3.3d' % (timeTuple.tm_year,timeTuple.tm_yday) - - fullpath = os.path.join( path, subfolder ) + fullpath = os.path.join(path, subfolder) if os.path.exists(fullpath): - filesList = os.listdir( fullpath ) - filesList = [k for k in filesList if 'M' in k] + filesList = os.listdir(fullpath) + filesList = [k for k in filesList if k.startswith(self.optchar)] if len( filesList ) > 0: - filesList = sorted( filesList, key=str.lower ) + filesList = sorted(filesList, key=str.lower) filen = filesList[-1] # el filename debera tener el siguiente formato # 0 1234 567 89A BCDE (hex) # x YYYY DDD SSS .ext - if isNumber( filen[8:11] ): - setFile = int( filen[8:11] ) #inicializo mi contador de seteo al seteo del ultimo file + if isNumber(filen[8:11]): + setFile = int(filen[8:11]) #inicializo mi contador de seteo al seteo del ultimo file else: setFile = -1 else: @@ -850,248 +465,156 @@ class ParamWriter(Operation): if self.setType is None: setFile += 1 - file = '%s%4.4d%3.3d%03d%s' % (self.metaoptchar, + file = '%s%4.4d%3.3d%03d%s' % (self.optchar, timeTuple.tm_year, timeTuple.tm_yday, setFile, ext ) else: setFile = timeTuple.tm_hour*60+timeTuple.tm_min - file = '%s%4.4d%3.3d%04d%s' % (self.metaoptchar, + file = '%s%4.4d%3.3d%04d%s' % (self.optchar, timeTuple.tm_year, timeTuple.tm_yday, setFile, ext ) - filename = os.path.join( path, subfolder, file ) + self.filename = os.path.join( path, subfolder, file ) #Setting HDF5 File - fp = h5py.File(filename,'w') + self.fp = h5py.File(self.filename, 'w') #write metadata - self.writeMetadata(fp) + self.writeMetadata(self.fp) #Write data - grp = fp.create_group("Data") -# grp.attrs['metadata'] = self.metaFile + self.writeData(self.fp) -# grp.attrs['blocksPerFile'] = 0 - ds = [] - data = [] - dsList = self.dsList - i = 0 - while i < len(dsList): - dsInfo = dsList[i] - #One-dimension data - if dsInfo['mode'] == 0: -# ds0 = grp.create_dataset(self.dataList[i], (1,1), maxshape=(1,self.blocksPerFile) , chunks = True, dtype='S20') - ds0 = grp.create_dataset(dsInfo['variable'], (1,1), maxshape=(1,self.blocksPerFile) , chunks = True, dtype=numpy.float64) - ds.append(ds0) - data.append([]) - i += 1 - continue -# nDimsForDs.append(nDims[i]) - - elif dsInfo['mode'] == 2: - grp0 = grp.create_group(dsInfo['variable']) - ds0 = grp0.create_dataset(dsInfo['dsName'], (1,dsInfo['shape']), data = numpy.zeros((1,dsInfo['shape'])) , maxshape=(None,dsInfo['shape']), chunks=True) - ds.append(ds0) - data.append([]) - i += 1 - continue + def getLabel(self, name, x=None): - elif dsInfo['mode'] == 1: - grp0 = grp.create_group(dsInfo['variable']) + if x is None: + if 'Data' in self.description: + data = self.description['Data'] + if 'Metadata' in self.description: + data.update(self.description['Metadata']) + else: + data = self.description + if name in data: + if isinstance(data[name], str): + return data[name] + elif isinstance(data[name], list): + return None + elif isinstance(data[name], dict): + for key, value in data[name].items(): + return key + return name + else: + if 'Metadata' in self.description: + meta = self.description['Metadata'] + else: + meta = self.description + if name in meta: + if isinstance(meta[name], list): + return meta[name][x] + elif isinstance(meta[name], dict): + for key, value in meta[name].items(): + return value[x] + return 'channel{:02d}'.format(x) + + def writeMetadata(self, fp): - for j in range(dsInfo['dsNumber']): - dsInfo = dsList[i] - tableName = dsInfo['dsName'] - shape = int(dsInfo['shape']) + if self.description: + if 'Metadata' in self.description: + grp = fp.create_group('Metadata') + else: + grp = fp + else: + grp = fp.create_group('Metadata') - if dsInfo['nDim'] == 3: - ds0 = grp0.create_dataset(tableName, (shape[0],shape[1],1) , data = numpy.zeros((shape[0],shape[1],1)), maxshape = (None,shape[1],None), chunks=True) - else: - ds0 = grp0.create_dataset(tableName, (1,shape), data = numpy.zeros((1,shape)) , maxshape=(None,shape), chunks=True) + for i in range(len(self.metadataList)): + if not hasattr(self.dataOut, self.metadataList[i]): + log.warning('Metadata: `{}` not found'.format(self.metadataList[i]), self.name) + continue + value = getattr(self.dataOut, self.metadataList[i]) + if isinstance(value, bool): + if value is True: + value = 1 + else: + value = 0 + grp.create_dataset(self.getLabel(self.metadataList[i]), data=value) + return - ds.append(ds0) - data.append([]) - i += 1 -# nDimsForDs.append(nDims[i]) + def writeData(self, fp): + + if self.description: + if 'Data' in self.description: + grp = fp.create_group('Data') + else: + grp = fp + else: + grp = fp.create_group('Data') + dtsets = [] + data = [] + + for dsInfo in self.dsList: + if dsInfo['nDim'] == 0: + ds = grp.create_dataset( + self.getLabel(dsInfo['variable']), + (self.blocksPerFile, ), + chunks=True, + dtype=numpy.float64) + dtsets.append(ds) + data.append((dsInfo['variable'], -1)) + else: + label = self.getLabel(dsInfo['variable']) + if label is not None: + sgrp = grp.create_group(label) + else: + sgrp = grp + for i in range(dsInfo['dsNumber']): + ds = sgrp.create_dataset( + self.getLabel(dsInfo['variable'], i), + (self.blocksPerFile, ) + dsInfo['shape'][1:], + chunks=True, + dtype=dsInfo['dtype']) + dtsets.append(ds) + data.append((dsInfo['variable'], i)) fp.flush() - fp.close() - -# self.nDatas = nDatas -# self.nDims = nDims -# self.nDimsForDs = nDimsForDs - #Saving variables - print 'Writing the file: %s'%filename - self.filename = filename -# self.fp = fp -# self.grp = grp -# self.grp.attrs.modify('nRecords', 1) - self.ds = ds + + log.log('Creating file: {}'.format(fp.filename), self.name) + + self.ds = dtsets self.data = data -# self.setFile = setFile self.firsttime = True self.blockIndex = 0 return def putData(self): - if self.blockIndex == self.blocksPerFile or self.timeFlag(): + if (self.blockIndex == self.blocksPerFile) or self.timeFlag(): + self.closeFile() self.setNextFile() -# if not self.firsttime: - self.readBlock() - self.setBlock() #Prepare data to be written - self.writeBlock() #Write data - - return - - def readBlock(self): - - ''' - data Array configured - - - self.data - ''' - dsList = self.dsList - ds = self.ds - #Setting HDF5 File - fp = h5py.File(self.filename,'r+') - grp = fp["Data"] - ind = 0 - -# grp.attrs['blocksPerFile'] = 0 - while ind < len(dsList): - dsInfo = dsList[ind] - - if dsInfo['mode'] == 0: - ds0 = grp[dsInfo['variable']] - ds[ind] = ds0 - ind += 1 + for i, ds in enumerate(self.ds): + attr, ch = self.data[i] + if ch == -1: + ds[self.blockIndex] = getattr(self.dataOut, attr) else: + ds[self.blockIndex] = getattr(self.dataOut, attr)[ch] - grp0 = grp[dsInfo['variable']] - - for j in range(dsInfo['dsNumber']): - dsInfo = dsList[ind] - ds0 = grp0[dsInfo['dsName']] - ds[ind] = ds0 - ind += 1 - - self.fp = fp - self.grp = grp - self.ds = ds - - return - - def setBlock(self): - ''' - data Array configured - - - self.data - ''' - #Creating Arrays - dsList = self.dsList - data = self.data - ind = 0 - - while ind < len(dsList): - dsInfo = dsList[ind] - dataAux = getattr(self.dataOut, dsInfo['variable']) - - mode = dsInfo['mode'] - nDim = dsInfo['nDim'] - - if mode == 0 or mode == 2 or nDim == 1: - data[ind] = dataAux - ind += 1 -# elif nDim == 1: -# data[ind] = numpy.reshape(dataAux,(numpy.size(dataAux),1)) -# ind += 1 - elif nDim == 2: - for j in range(dsInfo['dsNumber']): - data[ind] = dataAux[j,:] - ind += 1 - elif nDim == 3: - for j in range(dsInfo['dsNumber']): - data[ind] = dataAux[:,j,:] - ind += 1 + self.fp.flush() + self.blockIndex += 1 + log.log('Block No. {}/{}'.format(self.blockIndex, self.blocksPerFile), self.name) - self.data = data return - def writeBlock(self): - ''' - Saves the block in the HDF5 file - ''' - dsList = self.dsList - - for i in range(len(self.ds)): - dsInfo = dsList[i] - nDim = dsInfo['nDim'] - mode = dsInfo['mode'] - - # First time - if self.firsttime: -# self.ds[i].resize(self.data[i].shape) -# self.ds[i][self.blockIndex,:] = self.data[i] - if type(self.data[i]) == numpy.ndarray: - - if nDim == 3: - self.data[i] = self.data[i].reshape((self.data[i].shape[0],self.data[i].shape[1],1)) - self.ds[i].resize(self.data[i].shape) - if mode == 2: - self.ds[i].resize(self.data[i].shape) - self.ds[i][:] = self.data[i] - else: + def closeFile(self): - # From second time - # Meteors! - if mode == 2: - dataShape = self.data[i].shape - dsShape = self.ds[i].shape - self.ds[i].resize((self.ds[i].shape[0] + dataShape[0],self.ds[i].shape[1])) - self.ds[i][dsShape[0]:,:] = self.data[i] - # No dimension - elif mode == 0: - self.ds[i].resize((self.ds[i].shape[0], self.ds[i].shape[1] + 1)) - self.ds[i][0,-1] = self.data[i] - # One dimension - elif nDim == 1: - self.ds[i].resize((self.ds[i].shape[0] + 1, self.ds[i].shape[1])) - self.ds[i][-1,:] = self.data[i] - # Two dimension - elif nDim == 2: - self.ds[i].resize((self.ds[i].shape[0] + 1,self.ds[i].shape[1])) - self.ds[i][self.blockIndex,:] = self.data[i] - # Three dimensions - elif nDim == 3: - self.ds[i].resize((self.ds[i].shape[0],self.ds[i].shape[1],self.ds[i].shape[2]+1)) - self.ds[i][:,:,-1] = self.data[i] - - self.firsttime = False - self.blockIndex += 1 + if self.blockIndex != self.blocksPerFile: + for ds in self.ds: + ds.resize(self.blockIndex, axis=0) - #Close to save changes self.fp.flush() self.fp.close() - return - - def run(self, dataOut, path=None, blocksPerFile=10, metadataList=None, dataList=None, mode=None, **kwargs): - - if not(self.isConfig): - flagdata = self.setup(dataOut, path=path, blocksPerFile=blocksPerFile, - metadataList=metadataList, dataList=dataList, mode=mode, **kwargs) - if not(flagdata): - return + def close(self): - self.isConfig = True -# self.putMetadata() - self.setNextFile() - - self.putData() - return + self.closeFile() diff --git a/schainpy/model/io/jroIO_simulator.py b/schainpy/model/io/jroIO_simulator.py new file mode 100644 index 0000000..8dd305a --- /dev/null +++ b/schainpy/model/io/jroIO_simulator.py @@ -0,0 +1,519 @@ +import numpy,math,random,time +#---------------1 Heredamos JRODatareader +from schainpy.model.io.jroIO_base import * +#---------------2 Heredamos las propiedades de ProcessingUnit +from schainpy.model.proc.jroproc_base import ProcessingUnit,Operation,MPDecorator +#---------------3 Importaremos las clases BascicHeader, SystemHeader, RadarControlHeader, ProcessingHeader +from schainpy.model.data.jroheaderIO import PROCFLAG, BasicHeader,SystemHeader,RadarControllerHeader, ProcessingHeader +#---------------4 Importaremos el objeto Voltge +from schainpy.model.data.jrodata import Voltage + +class SimulatorReader(JRODataReader, ProcessingUnit): + incIntFactor = 1 + nFFTPoints = 0 + FixPP_IncInt = 1 + FixRCP_IPP = 1000 + FixPP_CohInt = 1 + Tau_0 = 250 + AcqH0_0 = 70 + H0 = AcqH0_0 + AcqDH_0 = 1.25 + DH0 = AcqDH_0 + Bauds = 32 + BaudWidth = None + FixRCP_TXA = 40 + FixRCP_TXB = 70 + fAngle = 2.0*math.pi*(1/16) + DC_level = 500 + stdev = 8 + Num_Codes = 2 + #code0 = numpy.array([1,1,1,0,1,1,0,1,1,1,1,0,0,0,1,0,1,1,1,0,1,1,0,1,0,0,0,1,1,1,0,1]) + #code1 = numpy.array([1,1,1,0,1,1,0,1,1,1,1,0,0,0,1,0,0,0,0,1,0,0,1,0,1,1,1,0,0,0,1,0]) + #Dyn_snCode = numpy.array([Num_Codes,Bauds]) + Dyn_snCode = None + Samples = 200 + channels = 2 + pulses = None + Reference = None + pulse_size = None + prof_gen = None + Fdoppler = 100 + Hdoppler = 36 + Adoppler = 300 + frequency = 9345 + nTotalReadFiles = 1000 + + def __init__(self): + """ + Inicializador de la clases SimulatorReader para + generar datos de voltage simulados. + Input: + dataOut: Objeto de la clase Voltage. + Este Objeto sera utilizado apra almacenar + un perfil de datos cada vez qe se haga + un requerimiento (getData) + """ + ProcessingUnit.__init__(self) + print(" [ START ] init - Metodo Simulator Reader") + + self.isConfig = False + self.basicHeaderObj = BasicHeader(LOCALTIME) + self.systemHeaderObj = SystemHeader() + self.radarControllerHeaderObj = RadarControllerHeader() + self.processingHeaderObj = ProcessingHeader() + self.profileIndex = 2**32-1 + self.dataOut = Voltage() + #code0 = numpy.array([1,1,1,0,1,1,0,1,1,1,1,0,0,0,1,0,1,1,1,0,1,1,0,1,0,0,0,1,1,1,0,1]) + code0 = numpy.array([1,1,1,-1,1,1,-1,1,1,1,1,-1,-1,-1,1,-1,1,1,1,-1,1,1,-1,1,-1,-1,-1,1,1,1,-1,1]) + #code1 = numpy.array([1,1,1,0,1,1,0,1,1,1,1,0,0,0,1,0,0,0,0,1,0,0,1,0,1,1,1,0,0,0,1,0]) + code1 = numpy.array([1,1,1,-1,1,1,-1,1,1,1,1,-1,-1,-1,1,-1,-1,-1,-1,1,-1,-1,1,-1,1,1,1,-1,-1,-1,1,-1]) + #self.Dyn_snCode = numpy.array([code0,code1]) + self.Dyn_snCode = None + + def set_kwargs(self, **kwargs): + for key, value in kwargs.items(): + setattr(self, key, value) + + def __hasNotDataInBuffer(self): + + if self.profileIndex >= self.processingHeaderObj.profilesPerBlock* self.nTxs: + if self.nReadBlocks>0: + tmp = self.dataOut.utctime + tmp_utc = int(self.dataOut.utctime) + tmp_milisecond = int((tmp-tmp_utc)*1000) + self.basicHeaderObj.utc = tmp_utc + self.basicHeaderObj.miliSecond= tmp_milisecond + return 1 + return 0 + + def setNextFile(self): + """Set the next file to be readed open it and parse de file header""" + + if (self.nReadBlocks >= self.processingHeaderObj.dataBlocksPerFile): + self.nReadFiles=self.nReadFiles+1 + if self.nReadFiles > self.nTotalReadFiles: + self.flagNoMoreFiles=1 + raise schainpy.admin.SchainWarning('No more files to read') + + print('------------------- [Opening file] ------------------------------',self.nReadFiles) + self.nReadBlocks = 0 + #if self.nReadBlocks==0: + # self.readFirstHeader() + + def __setNewBlock(self): + self.setNextFile() + if self.flagIsNewFile: + return 1 + + def readNextBlock(self): + while True: + self.__setNewBlock() + if not(self.readBlock()): + return 0 + self.getBasicHeader() + break + if self.verbose: + print("[Reading] Block No. %d/%d -> %s" %(self.nReadBlocks, + self.processingHeaderObj.dataBlocksPerFile, + self.dataOut.datatime.ctime()) ) + return 1 + + def getFirstHeader(self): + self.getBasicHeader() + self.dataOut.processingHeaderObj = self.processingHeaderObj.copy() + self.dataOut.systemHeaderObj = self.systemHeaderObj.copy() + self.dataOut.radarControllerHeaderObj = self.radarControllerHeaderObj.copy() + self.dataOut.dtype = self.dtype + + self.dataOut.nProfiles = self.processingHeaderObj.profilesPerBlock + self.dataOut.heightList = numpy.arange(self.processingHeaderObj.nHeights) * self.processingHeaderObj.deltaHeight + self.processingHeaderObj.firstHeight + self.dataOut.channelList = list(range(self.systemHeaderObj.nChannels)) + self.dataOut.nCohInt = self.processingHeaderObj.nCohInt + # asumo q la data no esta decodificada + self.dataOut.flagDecodeData = self.processingHeaderObj.flag_decode + # asumo q la data no esta sin flip + self.dataOut.flagDeflipData = self.processingHeaderObj.flag_deflip + self.dataOut.flagShiftFFT = self.processingHeaderObj.shif_fft + self.dataOut.frequency = self.frequency + + def getBasicHeader(self): + self.dataOut.utctime = self.basicHeaderObj.utc + self.basicHeaderObj.miliSecond / \ + 1000. + self.profileIndex * self.radarControllerHeaderObj.ippSeconds + + self.dataOut.flagDiscontinuousBlock = self.flagDiscontinuousBlock + self.dataOut.timeZone = self.basicHeaderObj.timeZone + self.dataOut.dstFlag = self.basicHeaderObj.dstFlag + self.dataOut.errorCount = self.basicHeaderObj.errorCount + self.dataOut.useLocalTime = self.basicHeaderObj.useLocalTime + self.dataOut.ippSeconds = self.radarControllerHeaderObj.ippSeconds / self.nTxs + + def readFirstHeader(self): + + datatype = int(numpy.log2((self.processingHeaderObj.processFlags & + PROCFLAG.DATATYPE_MASK)) - numpy.log2(PROCFLAG.DATATYPE_CHAR)) + if datatype == 0: + datatype_str = numpy.dtype([('real', ' self.incIntFactor): + self.incIntFactor = self.FixPP_IncInt/ self.incIntFactor + elif(self.FixPP_IncInt< self.incIntFactor): + print("False alert...") + + ProfilesperBlock = self.processingHeaderObj.profilesPerBlock + + self.timeperblock =int(((self.FixRCP_IPP + *ProfilesperBlock + *self.FixPP_CohInt + *self.incIntFactor) + /150.0) + *0.9 + +0.5) + # para cada canal + self.profiles = ProfilesperBlock*self.FixPP_CohInt + self.profiles = ProfilesperBlock + self.Reference = int((self.Tau_0-self.AcqH0_0)/(self.AcqDH_0)+0.5) + self.BaudWidth = int((self.FixRCP_TXA/self.AcqDH_0)/self.Bauds + 0.5 ) + + if (self.BaudWidth==0): + self.BaudWidth=1 + + def init_pulse(self,Num_Codes=Num_Codes,Bauds=Bauds,BaudWidth=BaudWidth,Dyn_snCode=Dyn_snCode): + + Num_Codes = Num_Codes + Bauds = Bauds + BaudWidth = BaudWidth + Dyn_snCode = Dyn_snCode + + if Dyn_snCode: + print("EXISTE") + else: + print("No existe") + + if Dyn_snCode: # if Bauds: + pulses = list(range(0,Num_Codes)) + num_codes = Num_Codes + for i in range(num_codes): + pulse_size = Bauds*BaudWidth + pulses[i] = numpy.zeros(pulse_size) + for j in range(Bauds): + for k in range(BaudWidth): + pulses[i][j*BaudWidth+k] = int(Dyn_snCode[i][j]*600) + else: + print("sin code") + pulses = list(range(1)) + if self.AcqDH_0>0.149: + pulse_size = int(self.FixRCP_TXB/0.15+0.5) + else: + pulse_size = int((self.FixRCP_TXB/self.AcqDH_0)+0.5) #0.0375 + pulses[0] = numpy.ones(pulse_size) + pulses = 600*pulses[0] + + return pulses,pulse_size + + def jro_GenerateBlockOfData(self,Samples=Samples,DC_level= DC_level,stdev=stdev, + Reference= Reference,pulses= pulses, + Num_Codes= Num_Codes,pulse_size=pulse_size, + prof_gen= prof_gen,H0 = H0,DH0=DH0, + Adoppler=Adoppler,Fdoppler= Fdoppler,Hdoppler=Hdoppler): + Samples = Samples + DC_level = DC_level + stdev = stdev + m_nR = Reference + pulses = pulses + num_codes = Num_Codes + ps = pulse_size + prof_gen = prof_gen + channels = self.channels + H0 = H0 + DH0 = DH0 + ippSec = self.radarControllerHeaderObj.ippSeconds + Fdoppler = self.Fdoppler + Hdoppler = self.Hdoppler + Adoppler = self.Adoppler + + self.datablock = numpy.zeros([channels,prof_gen,Samples],dtype= numpy.complex64) + for i in range(channels): + for k in range(prof_gen): + #·······················NOISE··············· + Noise_r = numpy.random.normal(DC_level,stdev,Samples) + Noise_i = numpy.random.normal(DC_level,stdev,Samples) + Noise = numpy.zeros(Samples,dtype=complex) + Noise.real = Noise_r + Noise.imag = Noise_i + #·······················PULSOS·············· + Pulso = numpy.zeros(pulse_size,dtype=complex) + Pulso.real = pulses[k%num_codes] + Pulso.imag = pulses[k%num_codes] + #····················· PULSES+NOISE·········· + InBuffer = numpy.zeros(Samples,dtype=complex) + InBuffer[m_nR:m_nR+ps] = Pulso + InBuffer = InBuffer+Noise + #····················· ANGLE ······························· + InBuffer.real[m_nR:m_nR+ps] = InBuffer.real[m_nR:m_nR+ps]*(math.cos( self.fAngle)*5) + InBuffer.imag[m_nR:m_nR+ps] = InBuffer.imag[m_nR:m_nR+ps]*(math.sin( self.fAngle)*5) + InBuffer=InBuffer + self.datablock[i][k]= InBuffer + + #················DOPPLER SIGNAL............................................... + time_vec = numpy.linspace(0,(prof_gen-1)*ippSec,int(prof_gen))+self.nReadBlocks*ippSec*prof_gen+(self.nReadFiles-1)*ippSec*prof_gen + fd = Fdoppler #+(600.0/120)*self.nReadBlocks + d_signal = Adoppler*numpy.array(numpy.exp(1.0j*2.0*math.pi*fd*time_vec),dtype=numpy.complex64) + #·············Señal con ancho espectral···················· + if prof_gen%2==0: + min = int(prof_gen/2.0-1.0) + max = int(prof_gen/2.0) + else: + min = int(prof_gen/2.0) + max = int(prof_gen/2.0) + specw_sig = numpy.linspace(-min,max,prof_gen) + w = 4 + A = 20 + specw_sig = specw_sig/w + specw_sig = numpy.sinc(specw_sig) + specw_sig = A*numpy.array(specw_sig,dtype=numpy.complex64) + #·················· DATABLOCK + DOPPLER···················· + HD=int(Hdoppler/self.AcqDH_0) + for i in range(12): + self.datablock[0,:,HD+i]=self.datablock[0,:,HD+i]+ d_signal# RESULT + #·················· DATABLOCK + DOPPLER*Sinc(x)···················· + HD=int(Hdoppler/self.AcqDH_0) + HD=int(HD/2) + for i in range(12): + self.datablock[0,:,HD+i]=self.datablock[0,:,HD+i]+ specw_sig*d_signal# RESULT + + def readBlock(self): + + self.jro_GenerateBlockOfData(Samples= self.samples,DC_level=self.DC_level, + stdev=self.stdev,Reference= self.Reference, + pulses = self.pulses,Num_Codes=self.Num_Codes, + pulse_size=self.pulse_size,prof_gen=self.profiles, + H0=self.H0,DH0=self.DH0) + + self.profileIndex = 0 + self.flagIsNewFile = 0 + self.flagIsNewBlock = 1 + self.nTotalBlocks += 1 + self.nReadBlocks += 1 + + return 1 + + + def getData(self): + if self.flagNoMoreFiles: + self.dataOut.flagNodata = True + return 0 + self.flagDiscontinuousBlock = 0 + self.flagIsNewBlock = 0 + if self.__hasNotDataInBuffer(): # aqui es verdad + if not(self.readNextBlock()): # return 1 y por eso el if not salta a getBasic Header + return 0 + self.getFirstHeader() # atributo + + if not self.getByBlock: + self.dataOut.flagDataAsBlock = False + self.dataOut.data = self.datablock[:, self.profileIndex, :] + self.dataOut.profileIndex = self.profileIndex + self.profileIndex += 1 + else: + pass + self.dataOut.flagNoData = False + self.getBasicHeader() + self.dataOut.realtime = self.online + return self.dataOut.data + + + def setup(self,frequency=49.92e6,incIntFactor= 1, nFFTPoints = 0, FixPP_IncInt=1,FixRCP_IPP=1000, + FixPP_CohInt= 1,Tau_0= 250,AcqH0_0 = 70 ,AcqDH_0=1.25, Bauds= 32, + FixRCP_TXA = 40, FixRCP_TXB = 50, fAngle = 2.0*math.pi*(1/16),DC_level= 50, + stdev= 8,Num_Codes = 1 , Dyn_snCode = None, samples=200, + channels=2,Fdoppler=20,Hdoppler=36,Adoppler=500, + profilesPerBlock=300,dataBlocksPerFile=120,nTotalReadFiles=10000, + **kwargs): + + self.set_kwargs(**kwargs) + self.nReadBlocks = 0 + self.nReadFiles = 1 + print('------------------- [Opening file: ] ------------------------------',self.nReadFiles) + + tmp = time.time() + tmp_utc = int(tmp) + tmp_milisecond = int((tmp-tmp_utc)*1000) + print(" SETUP -basicHeaderObj.utc",datetime.datetime.utcfromtimestamp(tmp)) + if Dyn_snCode is None: + Num_Codes=1 + Bauds =1 + + + + self.set_BH(utc= tmp_utc,miliSecond= tmp_milisecond,timeZone=300 ) + self.set_RCH( expType=0, nTx=150,ipp=FixRCP_IPP, txA=FixRCP_TXA, txB= FixRCP_TXB, + nWindows=1 , nHeights=samples, firstHeight=AcqH0_0, deltaHeight=AcqDH_0, + numTaus=1, line6Function=0, line5Function=0, fClock=None, + prePulseBefore=0, prePulseAfter=0, + codeType=0, nCode=Num_Codes, nBaud=32, code=Dyn_snCode, + flip1=0, flip2=0,Taus=Tau_0) + + self.set_PH(dtype=0, blockSize=0, profilesPerBlock=profilesPerBlock, + dataBlocksPerFile=dataBlocksPerFile, nWindows=1, processFlags=numpy.array([1024]), nCohInt=1, + nIncohInt=1, totalSpectra=0, nHeights=samples, firstHeight=AcqH0_0, + deltaHeight=AcqDH_0, samplesWin=samples, spectraComb=0, nCode=0, + code=0, nBaud=None, shif_fft=False, flag_dc=False, + flag_cspc=False, flag_decode=False, flag_deflip=False) + + self.set_SH(nSamples=samples, nProfiles=profilesPerBlock, nChannels=channels) + + self.readFirstHeader() + + self.frequency = frequency + self.incIntFactor = incIntFactor + self.nFFTPoints = nFFTPoints + self.FixPP_IncInt = FixPP_IncInt + self.FixRCP_IPP = FixRCP_IPP + self.FixPP_CohInt = FixPP_CohInt + self.Tau_0 = Tau_0 + self.AcqH0_0 = AcqH0_0 + self.H0 = AcqH0_0 + self.AcqDH_0 = AcqDH_0 + self.DH0 = AcqDH_0 + self.Bauds = Bauds + self.FixRCP_TXA = FixRCP_TXA + self.FixRCP_TXB = FixRCP_TXB + self.fAngle = fAngle + self.DC_level = DC_level + self.stdev = stdev + self.Num_Codes = Num_Codes + self.Dyn_snCode = Dyn_snCode + self.samples = samples + self.channels = channels + self.profiles = None + self.m_nReference = None + self.Baudwidth = None + self.Fdoppler = Fdoppler + self.Hdoppler = Hdoppler + self.Adoppler = Adoppler + self.nTotalReadFiles = int(nTotalReadFiles) + + print("IPP ", self.FixRCP_IPP) + print("Tau_0 ",self.Tau_0) + print("AcqH0_0",self.AcqH0_0) + print("samples,window ",self.samples) + print("AcqDH_0",AcqDH_0) + print("FixRCP_TXA",self.FixRCP_TXA) + print("FixRCP_TXB",self.FixRCP_TXB) + print("Dyn_snCode",Dyn_snCode) + print("Fdoppler", Fdoppler) + print("Hdoppler",Hdoppler) + print("Vdopplermax",Fdoppler*(3.0e8/self.frequency)/2.0) + print("nTotalReadFiles", nTotalReadFiles) + + self.init_acquisition() + self.pulses,self.pulse_size=self.init_pulse(Num_Codes=self.Num_Codes,Bauds=self.Bauds,BaudWidth=self.BaudWidth,Dyn_snCode=Dyn_snCode) + print(" [ END ] - SETUP metodo") + return + + def run(self,**kwargs): # metodo propio + if not(self.isConfig): + self.setup(**kwargs) + self.isConfig = True + self.getData() diff --git a/schainpy/model/io/jroIO_spectra.py b/schainpy/model/io/jroIO_spectra.py index 51f5a65..589c9b6 100644 --- a/schainpy/model/io/jroIO_spectra.py +++ b/schainpy/model/io/jroIO_spectra.py @@ -5,10 +5,12 @@ Created on Jul 2, 2014 ''' import numpy -from jroIO_base import LOCALTIME, JRODataReader, JRODataWriter -from schainpy.model.proc.jroproc_base import ProcessingUnit, Operation +from schainpy.model.io.jroIO_base import LOCALTIME, JRODataReader, JRODataWriter +from schainpy.model.proc.jroproc_base import ProcessingUnit, Operation, MPDecorator from schainpy.model.data.jroheaderIO import PROCFLAG, BasicHeader, SystemHeader, RadarControllerHeader, ProcessingHeader from schainpy.model.data.jrodata import Spectra +from schainpy.utils import log + class SpectraReader(JRODataReader, ProcessingUnit): """ @@ -51,25 +53,7 @@ class SpectraReader(JRODataReader, ProcessingUnit): """ - pts2read_SelfSpectra = 0 - - pts2read_CrossSpectra = 0 - - pts2read_DCchannels = 0 - - ext = ".pdata" - - optchar = "P" - - dataOut = None - - nRdChannels = None - - nRdPairs = None - - rdPairList = [] - - def __init__(self, **kwargs): + def __init__(self):#, **kwargs): """ Inicializador de la clase SpectraReader para la lectura de datos de espectros. @@ -87,91 +71,24 @@ class SpectraReader(JRODataReader, ProcessingUnit): Return : None """ - #Eliminar de la base la herencia - ProcessingUnit.__init__(self, **kwargs) - -# self.isConfig = False + ProcessingUnit.__init__(self) self.pts2read_SelfSpectra = 0 - self.pts2read_CrossSpectra = 0 - - self.pts2read_DCchannels = 0 - - self.datablock = None - - self.utc = None - + self.pts2read_DCchannels = 0 self.ext = ".pdata" - self.optchar = "P" - self.basicHeaderObj = BasicHeader(LOCALTIME) - self.systemHeaderObj = SystemHeader() - self.radarControllerHeaderObj = RadarControllerHeader() - self.processingHeaderObj = ProcessingHeader() - - self.online = 0 - - self.fp = None - - self.idFile = None - - self.dtype = None - - self.fileSizeByHeader = None - - self.filenameList = [] - - self.filename = None - - self.fileSize = None - - self.firstHeaderSize = 0 - - self.basicHeaderSize = 24 - - self.pathList = [] - self.lastUTTime = 0 - self.maxTimeStep = 30 - - self.flagNoMoreFiles = 0 - - self.set = 0 - - self.path = None - - self.delay = 60 #seconds - - self.nTries = 3 #quantity tries - - self.nFiles = 3 #number of files for searching - - self.nReadBlocks = 0 - - self.flagIsNewFile = 1 - - self.__isFirstTimeOnline = 1 - -# self.ippSeconds = 0 - - self.flagDiscontinuousBlock = 0 - - self.flagIsNewBlock = 0 - - self.nTotalBlocks = 0 - - self.blocksize = 0 - - self.dataOut = self.createObjByDefault() - - self.profileIndex = 1 #Always - + self.dataOut = Spectra() + self.profileIndex = 1 + self.nRdChannels = None + self.nRdPairs = None + self.rdPairList = [] def createObjByDefault(self): @@ -224,9 +141,6 @@ class SpectraReader(JRODataReader, ProcessingUnit): self.pts2read_DCchannels = int(self.systemHeaderObj.nChannels * self.processingHeaderObj.nHeights) self.blocksize += self.pts2read_DCchannels -# self.blocksize = self.pts2read_SelfSpectra + self.pts2read_CrossSpectra + self.pts2read_DCchannels - - def readBlock(self): """ Lee el bloque de datos desde la posicion actual del puntero del archivo @@ -248,7 +162,7 @@ class SpectraReader(JRODataReader, ProcessingUnit): Exceptions: Si un bloque leido no es un bloque valido """ - blockOk_flag = False + fpointer = self.fp.tell() spc = numpy.fromfile( self.fp, self.dtype[0], self.pts2read_SelfSpectra ) @@ -262,7 +176,6 @@ class SpectraReader(JRODataReader, ProcessingUnit): dc = numpy.fromfile( self.fp, self.dtype, self.pts2read_DCchannels ) #int(self.processingHeaderObj.nHeights*self.systemHeaderObj.nChannels) ) dc = dc.reshape( (self.systemHeaderObj.nChannels, self.processingHeaderObj.nHeights) ) #transforma a un arreglo 2D - if not self.processingHeaderObj.shif_fft: #desplaza a la derecha en el eje 2 determinadas posiciones shift = int(self.processingHeaderObj.profilesPerBlock/2) @@ -298,39 +211,19 @@ class SpectraReader(JRODataReader, ProcessingUnit): def getFirstHeader(self): self.getBasicHeader() - self.dataOut.systemHeaderObj = self.systemHeaderObj.copy() - self.dataOut.radarControllerHeaderObj = self.radarControllerHeaderObj.copy() - -# self.dataOut.ippSeconds = self.ippSeconds - -# self.dataOut.timeInterval = self.radarControllerHeaderObj.ippSeconds * self.processingHeaderObj.nCohInt * self.processingHeaderObj.nIncohInt * self.processingHeaderObj.profilesPerBlock - self.dataOut.dtype = self.dtype - -# self.dataOut.nPairs = self.nPairs - self.dataOut.pairsList = self.rdPairList - self.dataOut.nProfiles = self.processingHeaderObj.profilesPerBlock - self.dataOut.nFFTPoints = self.processingHeaderObj.profilesPerBlock - self.dataOut.nCohInt = self.processingHeaderObj.nCohInt - self.dataOut.nIncohInt = self.processingHeaderObj.nIncohInt - xf = self.processingHeaderObj.firstHeight + self.processingHeaderObj.nHeights*self.processingHeaderObj.deltaHeight - self.dataOut.heightList = numpy.arange(self.processingHeaderObj.firstHeight, xf, self.processingHeaderObj.deltaHeight) - - self.dataOut.channelList = range(self.systemHeaderObj.nChannels) - + self.dataOut.channelList = list(range(self.systemHeaderObj.nChannels)) self.dataOut.flagShiftFFT = True #Data is always shifted - self.dataOut.flagDecodeData = self.processingHeaderObj.flag_decode #asumo q la data no esta decodificada - self.dataOut.flagDeflipData = self.processingHeaderObj.flag_deflip #asumo q la data esta sin flip def getData(self): @@ -347,14 +240,12 @@ class SpectraReader(JRODataReader, ProcessingUnit): Affected: self.dataOut - self.flagDiscontinuousBlock self.flagIsNewBlock """ if self.flagNoMoreFiles: self.dataOut.flagNoData = True - print 'Process finished' return 0 self.flagDiscontinuousBlock = 0 @@ -373,21 +264,17 @@ class SpectraReader(JRODataReader, ProcessingUnit): return 0 self.getBasicHeader() - self.getFirstHeader() - self.dataOut.data_spc = self.data_spc - self.dataOut.data_cspc = self.data_cspc - self.dataOut.data_dc = self.data_dc - self.dataOut.flagNoData = False - self.dataOut.realtime = self.online return self.dataOut.data_spc + +@MPDecorator class SpectraWriter(JRODataWriter, Operation): """ @@ -395,25 +282,7 @@ class SpectraWriter(JRODataWriter, Operation): de los datos siempre se realiza por bloques. """ - ext = ".pdata" - - optchar = "P" - - shape_spc_Buffer = None - - shape_cspc_Buffer = None - - shape_dc_Buffer = None - - data_spc = None - - data_cspc = None - - data_dc = None - -# dataOut = None - - def __init__(self, **kwargs): + def __init__(self): """ Inicializador de la clase SpectraWriter para la escritura de datos de espectros. @@ -427,45 +296,23 @@ class SpectraWriter(JRODataWriter, Operation): Return: None """ - Operation.__init__(self, **kwargs) - - self.isConfig = False - - self.nTotalBlocks = 0 + Operation.__init__(self) + self.ext = ".pdata" + self.optchar = "P" + self.shape_spc_Buffer = None + self.shape_cspc_Buffer = None + self.shape_dc_Buffer = None self.data_spc = None - self.data_cspc = None - self.data_dc = None - - self.fp = None - - self.flagIsNewFile = 1 - - self.nTotalBlocks = 0 - - self.flagIsNewBlock = 0 - self.setFile = None - - self.dtype = None - - self.path = None - self.noMoreFiles = 0 - - self.filename = None - self.basicHeaderObj = BasicHeader(LOCALTIME) - self.systemHeaderObj = SystemHeader() - self.radarControllerHeaderObj = RadarControllerHeader() - self.processingHeaderObj = ProcessingHeader() - def hasAllDataInBuffer(self): return 1 @@ -494,7 +341,7 @@ class SpectraWriter(JRODataWriter, Operation): def writeBlock(self): - """ + """processingHeaderObj Escribe el buffer en el file designado Affected: @@ -511,24 +358,27 @@ class SpectraWriter(JRODataWriter, Operation): spc = numpy.transpose( self.data_spc, (0,2,1) ) if not self.processingHeaderObj.shif_fft: - spc = numpy.roll( spc, self.processingHeaderObj.profilesPerBlock/2, axis=2 ) #desplaza a la derecha en el eje 2 determinadas posiciones + spc = numpy.roll( spc, int(self.processingHeaderObj.profilesPerBlock/2), axis=2 ) #desplaza a la derecha en el eje 2 determinadas posiciones data = spc.reshape((-1)) data = data.astype(self.dtype[0]) data.tofile(self.fp) if self.data_cspc is not None: - data = numpy.zeros( self.shape_cspc_Buffer, self.dtype ) + cspc = numpy.transpose( self.data_cspc, (0,2,1) ) + data = numpy.zeros( numpy.shape(cspc), self.dtype ) + #print 'data.shape', self.shape_cspc_Buffer if not self.processingHeaderObj.shif_fft: - cspc = numpy.roll( cspc, self.processingHeaderObj.profilesPerBlock/2, axis=2 ) #desplaza a la derecha en el eje 2 determinadas posiciones + cspc = numpy.roll( cspc, int(self.processingHeaderObj.profilesPerBlock/2), axis=2 ) #desplaza a la derecha en el eje 2 determinadas posiciones data['real'] = cspc.real data['imag'] = cspc.imag data = data.reshape((-1)) data.tofile(self.fp) if self.data_dc is not None: - data = numpy.zeros( self.shape_dc_Buffer, self.dtype ) + dc = self.data_dc + data = numpy.zeros( numpy.shape(dc), self.dtype ) data['real'] = dc.real data['imag'] = dc.imag data = data.reshape((-1)) @@ -593,8 +443,6 @@ class SpectraWriter(JRODataWriter, Operation): # self.setFirstHeader() self.writeNextBlock() - return 1 - def __getBlockSize(self): ''' Este metodos determina el cantidad de bytes para un bloque de datos de tipo Spectra @@ -676,4 +524,4 @@ class SpectraWriter(JRODataWriter, Operation): self.processingHeaderObj.processFlags = self.getProcessFlags() - self.setBasicHeader() + self.setBasicHeader() \ No newline at end of file diff --git a/schainpy/model/io/jroIO_usrp.py b/schainpy/model/io/jroIO_usrp.py index c3d32bf..5aac6de 100644 --- a/schainpy/model/io/jroIO_usrp.py +++ b/schainpy/model/io/jroIO_usrp.py @@ -14,12 +14,12 @@ except: from schainpy.model.data.jroheaderIO import RadarControllerHeader, SystemHeader from schainpy.model.data.jrodata import Voltage -from schainpy.model.proc.jroproc_base import ProcessingUnit, Operation +from schainpy.model.proc.jroproc_base import ProcessingUnit, Operation, MPDecorator try: import digital_rf_hdf5 except: - print 'You should install "digital_rf_hdf5" module if you want to read USRP data' + pass class USRPReader(ProcessingUnit): ''' @@ -209,7 +209,7 @@ class USRPReader(ProcessingUnit): ''' if not os.path.isdir(path): - raise ValueError, "[Reading] Directory %s does not exist" %path + raise ValueError("[Reading] Directory %s does not exist" %path) try: self.digitalReadObj = digital_rf_hdf5.read_hdf5(path, load_all_metadata=True) @@ -219,10 +219,10 @@ class USRPReader(ProcessingUnit): channelNameList = self.digitalReadObj.get_channels() if not channelNameList: - raise ValueError, "[Reading] Directory %s does not have any files" %path + raise ValueError("[Reading] Directory %s does not have any files" %path) if not channelList: - channelList = range(len(channelNameList)) + channelList = list(range(len(channelNameList))) ########## Reading metadata ###################### @@ -241,7 +241,7 @@ class USRPReader(ProcessingUnit): self.__frequency = this_metadata_file['fc'].value if not self.__frequency: - raise ValueError, "Center Frequency is not defined in metadata file" + raise ValueError("Center Frequency is not defined in metadata file") try: self.__timezone = this_metadata_file['timezone'].value @@ -299,7 +299,7 @@ class USRPReader(ProcessingUnit): if not nSamples: if not ippKm: - raise ValueError, "[Reading] nSamples or ippKm should be defined" + raise ValueError("[Reading] nSamples or ippKm should be defined") nSamples = int(ippKm / (1e6*0.15/self.__sample_rate)) @@ -346,14 +346,14 @@ class USRPReader(ProcessingUnit): self.__setFileHeader() self.isConfig = True - print "[Reading] USRP Data was found from %s to %s " %( + print("[Reading] USRP Data was found from %s to %s " %( datetime.datetime.utcfromtimestamp(self.__startUTCSecond - self.__timezone), datetime.datetime.utcfromtimestamp(self.__endUTCSecond - self.__timezone) - ) + )) - print "[Reading] Starting process from %s to %s" %(datetime.datetime.utcfromtimestamp(startUTCSecond - self.__timezone), + print("[Reading] Starting process from %s to %s" %(datetime.datetime.utcfromtimestamp(startUTCSecond - self.__timezone), datetime.datetime.utcfromtimestamp(endUTCSecond - self.__timezone) - ) + )) def __reload(self): @@ -366,7 +366,7 @@ class USRPReader(ProcessingUnit): # datetime.datetime.utcfromtimestamp(self.__startUTCSecond - self.__timezone), # datetime.datetime.utcfromtimestamp(self.__endUTCSecond - self.__timezone) # ) - print "[Reading] reloading metadata ..." + print("[Reading] reloading metadata ...") try: self.digitalReadObj.reload(complete_update=True) @@ -380,11 +380,11 @@ class USRPReader(ProcessingUnit): if end_index > self.__endUTCSecond*self.__sample_rate: self.__endUTCSecond = 1.0*end_index/self.__sample_rate - print - print "[Reading] New timerange found [%s, %s] " %( + print() + print("[Reading] New timerange found [%s, %s] " %( datetime.datetime.utcfromtimestamp(self.__startUTCSecond - self.__timezone), datetime.datetime.utcfromtimestamp(self.__endUTCSecond - self.__timezone) - ) + )) return True @@ -399,7 +399,7 @@ class USRPReader(ProcessingUnit): self.__thisUnixSample += self.__samples_to_read if self.__thisUnixSample + 2*self.__samples_to_read > self.__endUTCSecond*self.__sample_rate: - print "[Reading] There are no more data into selected time-range" + print("[Reading] There are no more data into selected time-range") self.__reload() @@ -418,17 +418,17 @@ class USRPReader(ProcessingUnit): self.__samples_to_read, thisChannelName) - except IOError, e: + except IOError as e: #read next profile self.__flagDiscontinuousBlock = True - print "[Reading] %s" %datetime.datetime.utcfromtimestamp(self.thisSecond - self.__timezone), e + print("[Reading] %s" %datetime.datetime.utcfromtimestamp(self.thisSecond - self.__timezone), e) break if result.shape[0] != self.__samples_to_read: self.__flagDiscontinuousBlock = True - print "[Reading] %s: Too few samples were found, just %d/%d samples" %(datetime.datetime.utcfromtimestamp(self.thisSecond - self.__timezone), + print("[Reading] %s: Too few samples were found, just %d/%d samples" %(datetime.datetime.utcfromtimestamp(self.thisSecond - self.__timezone), result.shape[0], - self.__samples_to_read) + self.__samples_to_read)) break self.__data_buffer[indexChannel,:] = result*volt_scale @@ -442,9 +442,9 @@ class USRPReader(ProcessingUnit): if not dataOk: return False - print "[Reading] %s: %d samples <> %f sec" %(datetime.datetime.utcfromtimestamp(self.thisSecond - self.__timezone), + print("[Reading] %s: %d samples <> %f sec" %(datetime.datetime.utcfromtimestamp(self.thisSecond - self.__timezone), self.__samples_to_read, - self.__timeInterval) + self.__timeInterval)) self.__bufferIndex = 0 @@ -490,7 +490,7 @@ class USRPReader(ProcessingUnit): return False if self.__flagDiscontinuousBlock: - print '[Reading] discontinuous block found ... continue with the next block' + print('[Reading] discontinuous block found ... continue with the next block') continue if not self.__online: @@ -500,7 +500,7 @@ class USRPReader(ProcessingUnit): if err_counter > nTries: return False - print '[Reading] waiting %d seconds to read a new block' %seconds + print('[Reading] waiting %d seconds to read a new block' %seconds) sleep(seconds) self.dataOut.data = self.__data_buffer[:,self.__bufferIndex:self.__bufferIndex+self.__nSamples] @@ -532,7 +532,7 @@ class USRPReader(ProcessingUnit): ''' ''' - print self.profileIndex + print(self.profileIndex) def run(self, **kwargs): ''' @@ -546,6 +546,8 @@ class USRPReader(ProcessingUnit): return + +@MPDecorator class USRPWriter(Operation): ''' classdocs @@ -597,4 +599,4 @@ if __name__ == '__main__': while True: readObj.run(path='/Volumes/DATA/haystack/passive_radar/') # readObj.printInfo() - readObj.printNumberOfBlock() + readObj.printNumberOfBlock() \ No newline at end of file diff --git a/schainpy/model/io/jroIO_usrp_api.py b/schainpy/model/io/jroIO_usrp_api.py index 62c4321..1fc35e0 100644 --- a/schainpy/model/io/jroIO_usrp_api.py +++ b/schainpy/model/io/jroIO_usrp_api.py @@ -5,7 +5,7 @@ Created on Jul 15, 2014 ''' import time import threading -import cPickle +import pickle # try: # from gevent import sleep @@ -109,9 +109,9 @@ class USRPReaderAPI(USRPReader, threading.Thread): ''' if not self.isConfig: - raise RuntimeError, 'setup() method has to be called before start()' + raise RuntimeError('setup() method has to be called before start()') - print "Running ..." + print("Running ...") while True: @@ -122,7 +122,7 @@ class USRPReaderAPI(USRPReader, threading.Thread): if not self.getData(): break - print ".", + print(".", end=' ') self.__mySerial = obj2Serial(self.dataOut, keyList = self.__DATAKEYLIST, @@ -134,6 +134,6 @@ class USRPReaderAPI(USRPReader, threading.Thread): # sleep(0.1) - print "Closing thread" + print("Closing thread") return \ No newline at end of file diff --git a/schainpy/model/io/jroIO_voltage.py b/schainpy/model/io/jroIO_voltage.py index 95a897f..cb484db 100644 --- a/schainpy/model/io/jroIO_voltage.py +++ b/schainpy/model/io/jroIO_voltage.py @@ -6,14 +6,10 @@ Created on Jul 2, 2014 import numpy -from jroIO_base import LOCALTIME, JRODataReader, JRODataWriter -from schainpy.model.proc.jroproc_base import ProcessingUnit, Operation +from .jroIO_base import LOCALTIME, JRODataReader, JRODataWriter +from schainpy.model.proc.jroproc_base import ProcessingUnit, Operation, MPDecorator from schainpy.model.data.jroheaderIO import PROCFLAG, BasicHeader, SystemHeader, RadarControllerHeader, ProcessingHeader from schainpy.model.data.jrodata import Voltage -import zmq -import tempfile -from StringIO import StringIO -# from _sha import blocksize class VoltageReader(JRODataReader, ProcessingUnit): @@ -57,12 +53,7 @@ class VoltageReader(JRODataReader, ProcessingUnit): """ - ext = ".r" - - optchar = "D" - dataOut = None - - def __init__(self, **kwargs): + def __init__(self): """ Inicializador de la clase VoltageReader para la lectura de datos de voltage. @@ -81,89 +72,19 @@ class VoltageReader(JRODataReader, ProcessingUnit): None """ - ProcessingUnit.__init__(self, **kwargs) - - self.isConfig = False - - self.datablock = None - - self.utc = 0 - + ProcessingUnit.__init__(self) + self.ext = ".r" - self.optchar = "D" - self.basicHeaderObj = BasicHeader(LOCALTIME) - self.systemHeaderObj = SystemHeader() - self.radarControllerHeaderObj = RadarControllerHeader() - self.processingHeaderObj = ProcessingHeader() - - self.online = 0 - - self.fp = None - - self.idFile = None - - self.dtype = None - - self.fileSizeByHeader = None - - self.filenameList = [] - - self.filename = None - - self.fileSize = None - - self.firstHeaderSize = 0 - - self.basicHeaderSize = 24 - - self.pathList = [] - - self.filenameList = [] - self.lastUTTime = 0 - - self.maxTimeStep = 30 - - self.flagNoMoreFiles = 0 - - self.set = 0 - - self.path = None - - self.profileIndex = 2**32 - 1 - - self.delay = 3 # seconds - - self.nTries = 3 # quantity tries - - self.nFiles = 3 # number of files for searching - - self.nReadBlocks = 0 - - self.flagIsNewFile = 1 - - self.__isFirstTimeOnline = 1 - -# self.ippSeconds = 0 - - self.flagDiscontinuousBlock = 0 - - self.flagIsNewBlock = 0 - - self.nTotalBlocks = 0 - - self.blocksize = 0 - - self.dataOut = self.createObjByDefault() - - self.nTxs = 1 - - self.txIndex = 0 + self.profileIndex = 2**32 - 1 + self.dataOut = Voltage() + self.selBlocksize = None + self.selBlocktime = None def createObjByDefault(self): @@ -286,7 +207,7 @@ class VoltageReader(JRODataReader, ProcessingUnit): self.dataOut.heightList = numpy.arange( self.processingHeaderObj.nHeights) * self.processingHeaderObj.deltaHeight + self.processingHeaderObj.firstHeight - self.dataOut.channelList = range(self.systemHeaderObj.nChannels) + self.dataOut.channelList = list(range(self.systemHeaderObj.nChannels)) self.dataOut.nCohInt = self.processingHeaderObj.nCohInt @@ -307,15 +228,15 @@ class VoltageReader(JRODataReader, ProcessingUnit): return if self.nTxs < 1 and self.processingHeaderObj.profilesPerBlock % (1. / self.nTxs) != 0: - raise ValueError, "1./nTxs (=%f), should be a multiple of nProfiles (=%d)" % ( - 1. / self.nTxs, self.processingHeaderObj.profilesPerBlock) + raise ValueError("1./nTxs (=%f), should be a multiple of nProfiles (=%d)" % ( + 1. / self.nTxs, self.processingHeaderObj.profilesPerBlock)) if self.nTxs > 1 and self.processingHeaderObj.nHeights % self.nTxs != 0: - raise ValueError, "nTxs (=%d), should be a multiple of nHeights (=%d)" % ( - self.nTxs, self.processingHeaderObj.nHeights) + raise ValueError("nTxs (=%d), should be a multiple of nHeights (=%d)" % ( + self.nTxs, self.processingHeaderObj.nHeights)) self.datablock = self.datablock.reshape( - (self.systemHeaderObj.nChannels, self.processingHeaderObj.profilesPerBlock * self.nTxs, self.processingHeaderObj.nHeights / self.nTxs)) + (self.systemHeaderObj.nChannels, self.processingHeaderObj.profilesPerBlock * self.nTxs, int(self.processingHeaderObj.nHeights / self.nTxs))) self.dataOut.nProfiles = self.processingHeaderObj.profilesPerBlock * self.nTxs self.dataOut.heightList = numpy.arange(self.processingHeaderObj.nHeights / self.nTxs) * \ @@ -345,7 +266,7 @@ class VoltageReader(JRODataReader, ProcessingUnit): elif datatype == 5: datatype_str = numpy.dtype([('real', ' -#include -#include -#include -#include - -// void printArr(int *array); -static PyObject *hildebrand_sekhon(PyObject *self, PyObject *args); -static PyObject *correlateByBlock(PyObject *self, PyObject *args); -#ifndef PyMODINIT_FUNC /* declarations for DLL import/export */ -#define PyMODINIT_FUNC void -#endif - -static PyMethodDef extensionsMethods[] = { - { "correlateByBlock", (PyCFunction)correlateByBlock, METH_VARARGS, "get correlation by block" }, - { "hildebrand_sekhon", (PyCFunction)hildebrand_sekhon, METH_VARARGS, "get noise with hildebrand_sekhon" }, - { NULL, NULL, 0, NULL } -}; - -PyMODINIT_FUNC initcSchain() { - Py_InitModule("cSchain", extensionsMethods); - import_array(); -} - -static PyObject *correlateByBlock(PyObject *self, PyObject *args) { - - // int *x = (int*) malloc(4000000 * 216 * sizeof(int));; - // int a = 5; - // x = &a; - // int b = 6; - // x = &b; - // printf("Antes de imprimir x \n"); - // printf("%d \n", x[0]); - - PyObject *data_obj1, *data_obj2; - PyArrayObject *data_array1, *data_array2, *correlateRow, *out, *dataRow, *codeRow; //, , - int mode; - - if (!PyArg_ParseTuple(args, "OOi", &data_obj1, &data_obj2, &mode)) return NULL; - - data_array1 = (PyArrayObject *) PyArray_FROM_OTF(data_obj1, NPY_COMPLEX128, NPY_ARRAY_IN_ARRAY); - data_array2 = (PyArrayObject *) PyArray_FROM_OTF(data_obj2, NPY_FLOAT64, NPY_ARRAY_IN_ARRAY); - - npy_intp dims[1]; - dims[0] = 200; - npy_intp dims_code[1]; - dims_code[0] = 16; - - double complex * dataRaw; - double * codeRaw; - dataRaw = (double complex*) PyArray_DATA(data_array1); - codeRaw = (double *) PyArray_DATA(data_array2); - double complex ** outC = malloc(40000*200*sizeof(double complex)); - int i; - - clock_t start = clock(); - for(i=0; i<40000; i++){ - // codeRow = PyArray_SimpleNewFromData(1, dims_code, NPY_FLOAT64, codeRaw + 16 * i); - // dataRow = PyArray_SimpleNewFromData(1, dims, NPY_COMPLEX128, dataRaw + 200 * i); - // Py_INCREF(codeRow); - // Py_INCREF(dataRow); - // PyArray_ENABLEFLAGS(codeRow, NPY_ARRAY_OWNDATA); - // PyArray_ENABLEFLAGS(dataRow, NPY_ARRAY_OWNDATA); - correlateRow = (PyArrayObject *) PyArray_Correlate2(PyArray_SimpleNewFromData(1, dims_code, NPY_FLOAT64, codeRaw + 16 * i), PyArray_SimpleNewFromData(1, dims, NPY_COMPLEX128, dataRaw + 200 * i), (npy_intp) 2); - //Py_INCREF(correlateRow); - // PyArray_ENABLEFLAGS(correlateRow, NPY_ARRAY_OWNDATA); - memcpy(outC + 200*i, (double complex*) PyArray_DATA(correlateRow), 200 * sizeof(double complex)); - - Py_DECREF(correlateRow); - // Py_DECREF(codeRow); - // Py_DECREF(dataRow); - } - clock_t end = clock(); - float seconds = (float)(end - start) / CLOCKS_PER_SEC; - printf("%f", seconds); - // - npy_intp dimsret[2]; - dimsret[0] = 40000; - dimsret[1] = 200; - out = PyArray_SimpleNewFromData(2, dimsret, NPY_COMPLEX128, outC); - PyArray_ENABLEFLAGS(out, NPY_ARRAY_OWNDATA); - //Py_INCREF(out); - Py_DECREF(data_array1); - Py_DECREF(data_array2); - // PyArray_DebugPrint(out); - // Py_DECREF(data_obj2); - // Py_DECREF(data_obj1); - // Py_DECREF(codeRow); - // Py_DECREF(dataRow); - // free(dataRaw); - // free(codeRaw); - - return PyArray_Return(out); -} - -static PyObject *hildebrand_sekhon(PyObject *self, PyObject *args) { - double navg; - PyObject *data_obj, *data_array; - - if (!PyArg_ParseTuple(args, "Od", &data_obj, &navg)) return NULL; - data_array = PyArray_FROM_OTF(data_obj, NPY_FLOAT64, NPY_ARRAY_IN_ARRAY); - if (data_array == NULL) { - Py_XDECREF(data_array); - Py_XDECREF(data_obj); - return NULL; - } - double *sortdata = (double*)PyArray_DATA(data_array); - int lenOfData = (int)PyArray_SIZE(data_array) ; - double nums_min = lenOfData*0.2; - if (nums_min <= 5) nums_min = 5; - double sump = 0; - double sumq = 0; - int j = 0; - int cont = 1; - double rtest = 0; - while ((cont == 1) && (j < lenOfData)) { - sump = sump + sortdata[j]; - sumq = sumq + pow(sortdata[j], 2); - if (j > nums_min) { - rtest = (double)j/(j-1) + 1/navg; - if ((sumq*j) > (rtest*pow(sump, 2))) { - j = j - 1; - sump = sump - sortdata[j]; - sumq = sumq - pow(sortdata[j],2); - cont = 0; - } - } - j = j + 1; - } - - double lnoise = sump / j; - - Py_DECREF(data_array); - - return Py_BuildValue("d", lnoise); -} - diff --git a/schainpy/model/proc/jroproc_amisr.py b/schainpy/model/proc/jroproc_amisr.py index 13e571d..93f3dd8 100644 --- a/schainpy/model/proc/jroproc_amisr.py +++ b/schainpy/model/proc/jroproc_amisr.py @@ -2,7 +2,7 @@ @author: Daniel Suarez ''' import numpy -from jroproc_base import ProcessingUnit, Operation +from .jroproc_base import ProcessingUnit, Operation from schainpy.model.data.jroamisr import AMISR class AMISRProc(ProcessingUnit): @@ -16,7 +16,7 @@ class AMISRProc(ProcessingUnit): self.dataOut.copy(self.dataIn) -class PrintInfo(Operation): +class PrintInfoAMISR(Operation): def __init__(self, **kwargs): Operation.__init__(self, **kwargs) self.__isPrinted = False @@ -24,16 +24,16 @@ class PrintInfo(Operation): def run(self, dataOut): if not self.__isPrinted: - print 'Number of Records by File: %d'%dataOut.nRecords - print 'Number of Pulses: %d'%dataOut.nProfiles - print 'Number of Pulses by Frame: %d'%dataOut.npulseByFrame - print 'Number of Samples by Pulse: %d'%len(dataOut.heightList) - print 'Ipp Seconds: %f'%dataOut.ippSeconds - print 'Number of Beams: %d'%dataOut.nBeams - print 'BeamCodes:' - beamStrList = ['Beam %d -> Code=%d, azimuth=%2.2f, zenith=%2.2f, gain=%2.2f'%(k,v[0],v[1],v[2],v[3]) for k,v in dataOut.beamCodeDict.items()] + print('Number of Records by File: %d'%dataOut.nRecords) + print('Number of Pulses: %d'%dataOut.nProfiles) + print('Number of Pulses by Frame: %d'%dataOut.npulseByFrame) + print('Number of Samples by Pulse: %d'%len(dataOut.heightList)) + print('Ipp Seconds: %f'%dataOut.ippSeconds) + print('Number of Beams: %d'%dataOut.nBeams) + print('BeamCodes:') + beamStrList = ['Beam %d -> Code=%d, azimuth=%2.2f, zenith=%2.2f, gain=%2.2f'%(k,v[0],v[1],v[2],v[3]) for k,v in list(dataOut.beamCodeDict.items())] for b in beamStrList: - print b + print(b) self.__isPrinted = True return @@ -93,7 +93,7 @@ class BeamSelector(Operation): return 1 else: - raise ValueError, "BeamSelector needs beam value" + raise ValueError("BeamSelector needs beam value") return 0 @@ -117,7 +117,7 @@ class ProfileToChannels(Operation): dataOut.flagNoData = True if not(self.__isConfig): - nchannels = len(dataOut.beamRangeDict.keys()) + nchannels = len(list(dataOut.beamRangeDict.keys())) nsamples = dataOut.nHeights self.buffer = numpy.zeros((nchannels, nsamples), dtype = 'complex128') dataOut.beam.codeList = [dataOut.beamCodeDict[x][0] for x in range(nchannels)] @@ -136,7 +136,7 @@ class ProfileToChannels(Operation): if self.__counter_chan >= self.buffer.shape[0]: self.__counter_chan = 0 dataOut.data = self.buffer.copy() - dataOut.channelList = range(self.buffer.shape[0]) + dataOut.channelList = list(range(self.buffer.shape[0])) self.__isConfig = False dataOut.flagNoData = False pass diff --git a/schainpy/model/proc/jroproc_base.py b/schainpy/model/proc/jroproc_base.py index 31873d6..62cb9d6 100644 --- a/schainpy/model/proc/jroproc_base.py +++ b/schainpy/model/proc/jroproc_base.py @@ -1,289 +1,91 @@ ''' - -$Author: murco $ -$Id: jroproc_base.py 1 2012-11-12 18:56:07Z murco $ +Base clases to create Processing units and operations, the MPDecorator +must be used in plotting and writing operations to allow to run as an +external process. ''' -import inspect -from fuzzywuzzy import process - -def checkKwargs(method, kwargs): - currentKwargs = kwargs - choices = inspect.getargspec(method).args - try: - choices.remove('self') - except Exception as e: - pass - - try: - choices.remove('dataOut') - except Exception as e: - pass - - for kwarg in kwargs: - fuzz = process.extractOne(kwarg, choices) - if fuzz is None: - continue - if fuzz[1] < 100: - raise Exception('\x1b[0;32;40mDid you mean {} instead of {} in {}? \x1b[0m'. - format(fuzz[0], kwarg, method.__self__.__class__.__name__)) - -class ProcessingUnit(object): - - """ - Esta es la clase base para el procesamiento de datos. - - Contiene el metodo "call" para llamar operaciones. Las operaciones pueden ser: - - Metodos internos (callMethod) - - Objetos del tipo Operation (callObject). Antes de ser llamados, estos objetos - tienen que ser agreagados con el metodo "add". - - """ - # objeto de datos de entrada (Voltage, Spectra o Correlation) - dataIn = None - dataInList = [] - # objeto de datos de entrada (Voltage, Spectra o Correlation) - dataOut = None +import inspect +import zmq +import time +import pickle +import traceback +try: + from queue import Queue +except: + from Queue import Queue +from threading import Thread +from multiprocessing import Process, Queue +from schainpy.utils import log - operations2RunDict = None - isConfig = False +class ProcessingUnit(object): + ''' + Base class to create Signal Chain Units + ''' + proc_type = 'processing' - def __init__(self, *args, **kwargs): + def __init__(self): self.dataIn = None - self.dataInList = [] - self.dataOut = None - - self.operations2RunDict = {} - self.operationKwargs = {} - self.isConfig = False + self.operations = [] + + def setInput(self, unit): - self.args = args - self.kwargs = kwargs - - if not hasattr(self, 'name'): - self.name = self.__class__.__name__ - - checkKwargs(self.run, kwargs) - + self.dataIn = unit.dataOut + def getAllowedArgs(self): if hasattr(self, '__attrs__'): return self.__attrs__ else: return inspect.getargspec(self.run).args - def addOperationKwargs(self, objId, **kwargs): + def addOperation(self, conf, operation): ''' ''' - - self.operationKwargs[objId] = kwargs - - - def addOperation(self, opObj, objId): - - """ - Agrega un objeto del tipo "Operation" (opObj) a la lista de objetos "self.objectList" y retorna el - identificador asociado a este objeto. - - Input: - - object : objeto de la clase "Operation" - - Return: - - objId : identificador del objeto, necesario para ejecutar la operacion - """ - - self.operations2RunDict[objId] = opObj - - return objId + + self.operations.append((operation, conf.type, conf.getKwargs())) def getOperationObj(self, objId): - if objId not in self.operations2RunDict.keys(): + if objId not in list(self.operations.keys()): return None - return self.operations2RunDict[objId] + return self.operations[objId] - def operation(self, **kwargs): - - """ - Operacion directa sobre la data (dataOut.data). Es necesario actualizar los valores de los - atributos del objeto dataOut - - Input: - - **kwargs : Diccionario de argumentos de la funcion a ejecutar - """ - - raise NotImplementedError - - def callMethod(self, name, opId): - - """ - Ejecuta el metodo con el nombre "name" y con argumentos **kwargs de la propia clase. - - Input: - name : nombre del metodo a ejecutar - - **kwargs : diccionario con los nombres y valores de la funcion a ejecutar. - - """ - - #Checking the inputs - if name == 'run': + def call(self, **kwargs): + ''' + ''' - if not self.checkInputs(): + try: + if self.dataIn is not None and self.dataIn.flagNoData and not self.dataIn.error: + return self.dataIn.isReady() + elif self.dataIn is None or not self.dataIn.error: + self.run(**kwargs) + elif self.dataIn.error: + self.dataOut.error = self.dataIn.error self.dataOut.flagNoData = True - return False - else: - #Si no es un metodo RUN la entrada es la misma dataOut (interna) - if self.dataOut is not None and self.dataOut.isEmpty(): - return False - - #Getting the pointer to method - methodToCall = getattr(self, name) - - #Executing the self method - - if hasattr(self, 'mp'): - if name=='run': - if self.mp is False: - self.mp = True - self.start() - else: - self.operationKwargs[opId]['parent'] = self.kwargs - methodToCall(**self.operationKwargs[opId]) - else: - if name=='run': - methodToCall(**self.kwargs) + except: + err = traceback.format_exc() + if 'SchainWarning' in err: + log.warning(err.split('SchainWarning:')[-1].split('\n')[0].strip(), self.name) + elif 'SchainError' in err: + log.error(err.split('SchainError:')[-1].split('\n')[0].strip(), self.name) else: - methodToCall(**self.operationKwargs[opId]) - - if self.dataOut is None: - return False - - if self.dataOut.isEmpty(): - return False - - return True - - def callObject(self, objId): - - """ - Ejecuta la operacion asociada al identificador del objeto "objId" - - Input: - - objId : identificador del objeto a ejecutar - - **kwargs : diccionario con los nombres y valores de la funcion a ejecutar. - - Return: - - None - """ - - if self.dataOut is not None and self.dataOut.isEmpty(): - return False - - externalProcObj = self.operations2RunDict[objId] - - if hasattr(externalProcObj, 'mp'): - if externalProcObj.mp is False: - externalProcObj.kwargs['parent'] = self.kwargs - self.operationKwargs[objId] = externalProcObj.kwargs - externalProcObj.mp = True - externalProcObj.start() - else: - externalProcObj.run(self.dataOut, **externalProcObj.kwargs) - self.operationKwargs[objId] = externalProcObj.kwargs - - - return True - - def call(self, opType, opName=None, opId=None): - """ - Return True si ejecuta la operacion interna nombrada "opName" o la operacion externa - identificada con el id "opId"; con los argumentos "**kwargs". - - False si la operacion no se ha ejecutado. - - Input: - - opType : Puede ser "self" o "external" - - Depende del tipo de operacion para llamar a:callMethod or callObject: - - 1. If opType = "self": Llama a un metodo propio de esta clase: - - name_method = getattr(self, name) - name_method(**kwargs) - - - 2. If opType = "other" o"external": Llama al metodo "run()" de una instancia de la - clase "Operation" o de un derivado de ella: - - instanceName = self.operationList[opId] - instanceName.run(**kwargs) - - opName : Si la operacion es interna (opType = 'self'), entonces el "opName" sera - usada para llamar a un metodo interno de la clase Processing - - opId : Si la operacion es externa (opType = 'other' o 'external), entonces el - "opId" sera usada para llamar al metodo "run" de la clase Operation - registrada anteriormente con ese Id - - Exception: - Este objeto de tipo Operation debe de haber sido agregado antes con el metodo: - "addOperation" e identificado con el valor "opId" = el id de la operacion. - De lo contrario retornara un error del tipo ValueError - - """ - - if opType == 'self': - - if not opName: - raise ValueError, "opName parameter should be defined" - - sts = self.callMethod(opName, opId) - - elif opType == 'other' or opType == 'external' or opType == 'plotter': - - if not opId: - raise ValueError, "opId parameter should be defined" - - if opId not in self.operations2RunDict.keys(): - raise ValueError, "Any operation with id=%s has been added" %str(opId) - - sts = self.callObject(opId) - - else: - raise ValueError, "opType should be 'self', 'external' or 'plotter'; and not '%s'" %opType - - return sts - - def setInput(self, dataIn): - - self.dataIn = dataIn - self.dataInList.append(dataIn) - - def getOutputObj(self): - - return self.dataOut - - def checkInputs(self): - - for thisDataIn in self.dataInList: - - if thisDataIn.isEmpty(): - return False - - return True + log.error(err, self.name) + self.dataOut.error = True + + for op, optype, opkwargs in self.operations: + if optype == 'other' and not self.dataOut.flagNoData: + self.dataOut = op.run(self.dataOut, **opkwargs) + elif optype == 'external' and not self.dataOut.flagNoData: + op.queue.put(self.dataOut) + elif optype == 'external' and self.dataOut.error: + op.queue.put(self.dataOut) + + return 'Error' if self.dataOut.error else self.dataOut.isReady() def setup(self): @@ -294,31 +96,24 @@ class ProcessingUnit(object): raise NotImplementedError def close(self): - #Close every thread, queue or any other object here is it is neccesary. - return - -class Operation(object): - """ - Clase base para definir las operaciones adicionales que se pueden agregar a la clase ProcessingUnit - y necesiten acumular informacion previa de los datos a procesar. De preferencia usar un buffer de - acumulacion dentro de esta clase + return - Ejemplo: Integraciones coherentes, necesita la informacion previa de los n perfiles anteriores (bufffer) - """ +class Operation(object): - __buffer = None - isConfig = False + ''' + ''' + + proc_type = 'operation' - def __init__(self, **kwargs): + def __init__(self): - self.__buffer = None + self.id = None self.isConfig = False - self.kwargs = kwargs + if not hasattr(self, 'name'): self.name = self.__class__.__name__ - checkKwargs(self.run, kwargs) def getAllowedArgs(self): if hasattr(self, '__attrs__'): @@ -333,7 +128,6 @@ class Operation(object): raise NotImplementedError def run(self, dataIn, **kwargs): - """ Realiza las operaciones necesarias sobre la dataIn.data y actualiza los atributos del objeto dataIn. @@ -357,4 +151,57 @@ class Operation(object): def close(self): - pass + return + + +def MPDecorator(BaseClass): + """ + Multiprocessing class decorator + + This function add multiprocessing features to a BaseClass. + """ + + class MPClass(BaseClass, Process): + + def __init__(self, *args, **kwargs): + super(MPClass, self).__init__() + Process.__init__(self) + + self.args = args + self.kwargs = kwargs + self.t = time.time() + self.op_type = 'external' + self.name = BaseClass.__name__ + self.__doc__ = BaseClass.__doc__ + + if 'plot' in self.name.lower() and not self.name.endswith('_'): + self.name = '{}{}'.format(self.CODE.upper(), 'Plot') + + self.start_time = time.time() + self.err_queue = args[3] + self.queue = Queue(maxsize=1) + self.myrun = BaseClass.run + + def run(self): + + while True: + + dataOut = self.queue.get() + + if not dataOut.error: + try: + BaseClass.run(self, dataOut, **self.kwargs) + except: + err = traceback.format_exc() + log.error(err, self.name) + else: + break + + self.close() + + def close(self): + + BaseClass.close(self) + log.success('Done...(Time:{:4.2f} secs)'.format(time.time()-self.start_time), self.name) + + return MPClass diff --git a/schainpy/model/proc/jroproc_correlation.py b/schainpy/model/proc/jroproc_correlation.py index 67c72d3..76a92fe 100644 --- a/schainpy/model/proc/jroproc_correlation.py +++ b/schainpy/model/proc/jroproc_correlation.py @@ -1,7 +1,7 @@ import numpy -from jroproc_base import ProcessingUnit, Operation -from schainpy.model.data.jrodata import Correlation, hildebrand_sekhon +from .jroproc_base import ProcessingUnit, Operation +from schainpy.model.data.jrodata import Correlation class CorrelationProc(ProcessingUnit): diff --git a/schainpy/model/proc/jroproc_heispectra.py b/schainpy/model/proc/jroproc_heispectra.py index 1c6976f..2fbd0b1 100644 --- a/schainpy/model/proc/jroproc_heispectra.py +++ b/schainpy/model/proc/jroproc_heispectra.py @@ -1,13 +1,16 @@ import numpy -from jroproc_base import ProcessingUnit, Operation +from .jroproc_base import ProcessingUnit, Operation, MPDecorator from schainpy.model.data.jrodata import SpectraHeis +from schainpy.utils import log + + class SpectraHeisProc(ProcessingUnit): - def __init__(self, **kwargs): + def __init__(self):#, **kwargs): - ProcessingUnit.__init__(self, **kwargs) + ProcessingUnit.__init__(self)#, **kwargs) # self.buffer = None # self.firstdatatime = None @@ -86,7 +89,7 @@ class SpectraHeisProc(ProcessingUnit): if self.dataIn.type == "Fits": self.__updateObjFromFits() self.dataOut.flagNoData = False - return + return if self.dataIn.type == "SpectraHeis": self.dataOut.copy(self.dataIn) @@ -99,7 +102,7 @@ class SpectraHeisProc(ProcessingUnit): return - raise ValueError, "The type object %s is not valid"%(self.dataIn.type) + raise ValueError("The type object %s is not valid"%(self.dataIn.type)) def selectChannels(self, channelList): @@ -133,8 +136,8 @@ class SpectraHeisProc(ProcessingUnit): for channelIndex in channelIndexList: if channelIndex not in self.dataOut.channelIndexList: - print channelIndexList - raise ValueError, "The value %d in channelIndexList is not valid" %channelIndex + print(channelIndexList) + raise ValueError("The value %d in channelIndexList is not valid" %channelIndex) # nChannels = len(channelIndexList) @@ -145,6 +148,7 @@ class SpectraHeisProc(ProcessingUnit): return 1 + class IncohInt4SpectraHeis(Operation): isConfig = False @@ -163,9 +167,9 @@ class IncohInt4SpectraHeis(Operation): n = None - def __init__(self, **kwargs): + def __init__(self):#, **kwargs): - Operation.__init__(self, **kwargs) + Operation.__init__(self)#, **kwargs) # self.isConfig = False def setup(self, n=None, timeInterval=None, overlapping=False): @@ -187,7 +191,7 @@ class IncohInt4SpectraHeis(Operation): if n == None and timeInterval == None: - raise ValueError, "n or timeInterval should be specified ..." + raise ValueError("n or timeInterval should be specified ...") if n != None: self.n = n @@ -342,3 +346,5 @@ class IncohInt4SpectraHeis(Operation): # dataOut.timeInterval = dataOut.ippSeconds * dataOut.nIncohInt # dataOut.timeInterval = self.__timeInterval*self.n dataOut.flagNoData = False + + return dataOut \ No newline at end of file diff --git a/schainpy/model/proc/jroproc_parameters.py b/schainpy/model/proc/jroproc_parameters.py old mode 100644 new mode 100755 index 113190d..f521bb6 --- a/schainpy/model/proc/jroproc_parameters.py +++ b/schainpy/model/proc/jroproc_parameters.py @@ -8,27 +8,22 @@ import copy import sys import importlib import itertools -from multiprocessing import Pool, TimeoutError +from multiprocessing import Pool, TimeoutError from multiprocessing.pool import ThreadPool -import copy_reg -import cPickle -import types -from functools import partial import time -#from sklearn.cluster import KMeans - from scipy.optimize import fmin_l_bfgs_b #optimize with bounds on state papameters -from jroproc_base import ProcessingUnit, Operation +from .jroproc_base import ProcessingUnit, Operation, MPDecorator from schainpy.model.data.jrodata import Parameters, hildebrand_sekhon from scipy import asarray as ar,exp from scipy.optimize import curve_fit - +from schainpy.utils import log import warnings from numpy import NaN from scipy.optimize.optimize import OptimizeWarning warnings.filterwarnings('ignore') +import matplotlib.pyplot as plt SPEED_OF_LIGHT = 299792458 @@ -36,9 +31,9 @@ SPEED_OF_LIGHT = 299792458 '''solving pickling issue''' def _pickle_method(method): - func_name = method.im_func.__name__ - obj = method.im_self - cls = method.im_class + func_name = method.__func__.__name__ + obj = method.__self__ + cls = method.__self__.__class__ return _unpickle_method, (func_name, obj, cls) def _unpickle_method(func_name, obj, cls): @@ -51,28 +46,31 @@ def _unpickle_method(func_name, obj, cls): break return func.__get__(obj, cls) + class ParametersProc(ProcessingUnit): - + + METHODS = {} nSeconds = None def __init__(self): ProcessingUnit.__init__(self) - + # self.objectDict = {} self.buffer = None self.firstdatatime = None self.profIndex = 0 self.dataOut = Parameters() - + self.setupReq = False #Agregar a todas las unidades de proc + def __updateObjFromInput(self): - + self.dataOut.inputUnit = self.dataIn.type - + self.dataOut.timeZone = self.dataIn.timeZone self.dataOut.dstFlag = self.dataIn.dstFlag self.dataOut.errorCount = self.dataIn.errorCount self.dataOut.useLocalTime = self.dataIn.useLocalTime - + self.dataOut.radarControllerHeaderObj = self.dataIn.radarControllerHeaderObj.copy() self.dataOut.systemHeaderObj = self.dataIn.systemHeaderObj.copy() self.dataOut.channelList = self.dataIn.channelList @@ -94,25 +92,41 @@ class ParametersProc(ProcessingUnit): self.dataOut.ippSeconds = self.dataIn.ippSeconds # self.dataOut.windowOfFilter = self.dataIn.windowOfFilter self.dataOut.timeInterval1 = self.dataIn.timeInterval - self.dataOut.heightList = self.dataIn.getHeiRange() + self.dataOut.heightList = self.dataIn.getHeiRange() self.dataOut.frequency = self.dataIn.frequency # self.dataOut.noise = self.dataIn.noise - + def run(self): - + + + #---------------------- Voltage Data --------------------------- - + if self.dataIn.type == "Voltage": self.__updateObjFromInput() self.dataOut.data_pre = self.dataIn.data.copy() self.dataOut.flagNoData = False self.dataOut.utctimeInit = self.dataIn.utctime - self.dataOut.paramInterval = self.dataIn.nProfiles*self.dataIn.nCohInt*self.dataIn.ippSeconds + self.dataOut.paramInterval = self.dataIn.nProfiles*self.dataIn.nCohInt*self.dataIn.ippSeconds + if hasattr(self.dataIn, 'dataPP_POW'): + self.dataOut.dataPP_POW = self.dataIn.dataPP_POW + + if hasattr(self.dataIn, 'dataPP_POWER'): + self.dataOut.dataPP_POWER = self.dataIn.dataPP_POWER + + if hasattr(self.dataIn, 'dataPP_DOP'): + self.dataOut.dataPP_DOP = self.dataIn.dataPP_DOP + + if hasattr(self.dataIn, 'dataPP_SNR'): + self.dataOut.dataPP_SNR = self.dataIn.dataPP_SNR + + if hasattr(self.dataIn, 'dataPP_WIDTH'): + self.dataOut.dataPP_WIDTH = self.dataIn.dataPP_WIDTH return - + #---------------------- Spectra Data --------------------------- - + if self.dataIn.type == "Spectra": self.dataOut.data_pre = (self.dataIn.data_spc, self.dataIn.data_cspc) @@ -124,493 +138,307 @@ class ParametersProc(ProcessingUnit): self.dataOut.ippFactor = self.dataIn.ippFactor self.dataOut.abscissaList = self.dataIn.getVelRange(1) self.dataOut.spc_noise = self.dataIn.getNoise() - self.dataOut.spc_range = (self.dataIn.getFreqRange(1)/1000. , self.dataIn.getAcfRange(1) , self.dataIn.getVelRange(1)) - self.dataOut.pairsList = self.dataIn.pairsList + self.dataOut.spc_range = (self.dataIn.getFreqRange(1) , self.dataIn.getAcfRange(1) , self.dataIn.getVelRange(1)) + # self.dataOut.normFactor = self.dataIn.normFactor + self.dataOut.pairsList = self.dataIn.pairsList self.dataOut.groupList = self.dataIn.pairsList self.dataOut.flagNoData = False - + if hasattr(self.dataIn, 'ChanDist'): #Distances of receiver channels self.dataOut.ChanDist = self.dataIn.ChanDist - else: self.dataOut.ChanDist = None - - if hasattr(self.dataIn, 'VelRange'): #Velocities range - self.dataOut.VelRange = self.dataIn.VelRange - else: self.dataOut.VelRange = None - + else: self.dataOut.ChanDist = None + + #if hasattr(self.dataIn, 'VelRange'): #Velocities range + # self.dataOut.VelRange = self.dataIn.VelRange + #else: self.dataOut.VelRange = None + if hasattr(self.dataIn, 'RadarConst'): #Radar Constant self.dataOut.RadarConst = self.dataIn.RadarConst - + if hasattr(self.dataIn, 'NPW'): #NPW self.dataOut.NPW = self.dataIn.NPW - + if hasattr(self.dataIn, 'COFA'): #COFA self.dataOut.COFA = self.dataIn.COFA - - - + + + #---------------------- Correlation Data --------------------------- - + if self.dataIn.type == "Correlation": acf_ind, ccf_ind, acf_pairs, ccf_pairs, data_acf, data_ccf = self.dataIn.splitFunctions() - + self.dataOut.data_pre = (self.dataIn.data_cf[acf_ind,:], self.dataIn.data_cf[ccf_ind,:,:]) self.dataOut.normFactor = (self.dataIn.normFactor[acf_ind,:], self.dataIn.normFactor[ccf_ind,:]) self.dataOut.groupList = (acf_pairs, ccf_pairs) - + self.dataOut.abscissaList = self.dataIn.lagRange self.dataOut.noise = self.dataIn.noise self.dataOut.data_SNR = self.dataIn.SNR self.dataOut.flagNoData = False self.dataOut.nAvg = self.dataIn.nAvg - + #---------------------- Parameters Data --------------------------- - + if self.dataIn.type == "Parameters": self.dataOut.copy(self.dataIn) self.dataOut.flagNoData = False - + return True - + self.__updateObjFromInput() self.dataOut.utctimeInit = self.dataIn.utctime self.dataOut.paramInterval = self.dataIn.timeInterval - + return def target(tups): - + obj, args = tups - #print 'TARGETTT', obj, args + return obj.FitGau(args) - + + +class SpectralFilters(Operation): + + '''This class allows the Rainfall / Wind Selection for CLAIRE RADAR + + LimitR : It is the limit in m/s of Rainfall + LimitW : It is the limit in m/s for Winds + + Input: + + self.dataOut.data_pre : SPC and CSPC + self.dataOut.spc_range : To select wind and rainfall velocities + + Affected: + + self.dataOut.data_pre : It is used for the new SPC and CSPC ranges of wind + self.dataOut.spcparam_range : Used in SpcParamPlot + self.dataOut.SPCparam : Used in PrecipitationProc + + + ''' + + def __init__(self): + Operation.__init__(self) + self.i=0 + + def run(self, dataOut, PositiveLimit=1.5, NegativeLimit=2.5): + + + #Limite de vientos + LimitR = PositiveLimit + LimitN = NegativeLimit + + self.spc = dataOut.data_pre[0].copy() + self.cspc = dataOut.data_pre[1].copy() + + self.Num_Hei = self.spc.shape[2] + self.Num_Bin = self.spc.shape[1] + self.Num_Chn = self.spc.shape[0] + + VelRange = dataOut.spc_range[2] + TimeRange = dataOut.spc_range[1] + FrecRange = dataOut.spc_range[0] + + Vmax= 2*numpy.max(dataOut.spc_range[2]) + Tmax= 2*numpy.max(dataOut.spc_range[1]) + Fmax= 2*numpy.max(dataOut.spc_range[0]) + + Breaker1R=VelRange[numpy.abs(VelRange-(-LimitN)).argmin()] + Breaker1R=numpy.where(VelRange == Breaker1R) + + Delta = self.Num_Bin/2 - Breaker1R[0] + + + '''Reacomodando SPCrange''' + + VelRange=numpy.roll(VelRange,-(int(self.Num_Bin/2)) ,axis=0) + + VelRange[-(int(self.Num_Bin/2)):]+= Vmax + + FrecRange=numpy.roll(FrecRange,-(int(self.Num_Bin/2)),axis=0) + + FrecRange[-(int(self.Num_Bin/2)):]+= Fmax + + TimeRange=numpy.roll(TimeRange,-(int(self.Num_Bin/2)),axis=0) + + TimeRange[-(int(self.Num_Bin/2)):]+= Tmax + + ''' ------------------ ''' + + Breaker2R=VelRange[numpy.abs(VelRange-(LimitR)).argmin()] + Breaker2R=numpy.where(VelRange == Breaker2R) + + + SPCroll = numpy.roll(self.spc,-(int(self.Num_Bin/2)) ,axis=1) + + SPCcut = SPCroll.copy() + for i in range(self.Num_Chn): + + SPCcut[i,0:int(Breaker2R[0]),:] = dataOut.noise[i] + SPCcut[i,-int(Delta):,:] = dataOut.noise[i] + + SPCcut[i]=SPCcut[i]- dataOut.noise[i] + SPCcut[ numpy.where( SPCcut<0 ) ] = 1e-20 + + SPCroll[i]=SPCroll[i]-dataOut.noise[i] + SPCroll[ numpy.where( SPCroll<0 ) ] = 1e-20 + + SPC_ch1 = SPCroll + + SPC_ch2 = SPCcut + + SPCparam = (SPC_ch1, SPC_ch2, self.spc) + dataOut.SPCparam = numpy.asarray(SPCparam) + + + dataOut.spcparam_range=numpy.zeros([self.Num_Chn,self.Num_Bin+1]) + + dataOut.spcparam_range[2]=VelRange + dataOut.spcparam_range[1]=TimeRange + dataOut.spcparam_range[0]=FrecRange + return dataOut + class GaussianFit(Operation): - + ''' - Function that fit of one and two generalized gaussians (gg) based - on the PSD shape across an "power band" identified from a cumsum of + Function that fit of one and two generalized gaussians (gg) based + on the PSD shape across an "power band" identified from a cumsum of the measured spectrum - noise. - + Input: self.dataOut.data_pre : SelfSpectra - + Output: - self.dataOut.GauSPC : SPC_ch1, SPC_ch2 - + self.dataOut.SPCparam : SPC_ch1, SPC_ch2 + ''' - def __init__(self, **kwargs): - Operation.__init__(self, **kwargs) + def __init__(self): + Operation.__init__(self) self.i=0 - - - def run(self, dataOut, num_intg=7, pnoise=1., vel_arr=None, SNRlimit=-9): #num_intg: Incoherent integrations, pnoise: Noise, vel_arr: range of velocities, similar to the ftt points + + + def run(self, dataOut, num_intg=7, pnoise=1., SNRlimit=-9): #num_intg: Incoherent integrations, pnoise: Noise, vel_arr: range of velocities, similar to the ftt points """This routine will find a couple of generalized Gaussians to a power spectrum input: spc output: Amplitude0,shift0,width0,p0,Amplitude1,shift1,width1,p1,noise """ - + self.spc = dataOut.data_pre[0].copy() - - - print 'SelfSpectra Shape', numpy.asarray(self.spc).shape - - - #plt.figure(50) - #plt.subplot(121) - #plt.plot(self.spc,'k',label='spc(66)') - #plt.plot(xFrec,ySamples[1],'g',label='Ch1') - #plt.plot(xFrec,ySamples[2],'r',label='Ch2') - #plt.plot(xFrec,FitGauss,'yo:',label='fit') - #plt.legend() - #plt.title('DATOS A ALTURA DE 7500 METROS') - #plt.show() - self.Num_Hei = self.spc.shape[2] - #self.Num_Bin = len(self.spc) self.Num_Bin = self.spc.shape[1] self.Num_Chn = self.spc.shape[0] - Vrange = dataOut.abscissaList - - #print 'self.spc2', numpy.asarray(self.spc).shape - - GauSPC = numpy.empty([2,self.Num_Bin,self.Num_Hei]) + + GauSPC = numpy.empty([self.Num_Chn,self.Num_Bin,self.Num_Hei]) SPC_ch1 = numpy.empty([self.Num_Bin,self.Num_Hei]) SPC_ch2 = numpy.empty([self.Num_Bin,self.Num_Hei]) SPC_ch1[:] = numpy.NaN SPC_ch2[:] = numpy.NaN - + start_time = time.time() - + noise_ = dataOut.spc_noise[0].copy() - - - - pool = Pool(processes=self.Num_Chn) + + + pool = Pool(processes=self.Num_Chn) args = [(Vrange, Ch, pnoise, noise_, num_intg, SNRlimit) for Ch in range(self.Num_Chn)] - objs = [self for __ in range(self.Num_Chn)] - attrs = zip(objs, args) + objs = [self for __ in range(self.Num_Chn)] + attrs = list(zip(objs, args)) gauSPC = pool.map(target, attrs) - dataOut.GauSPC = numpy.asarray(gauSPC) -# ret = [] -# for n in range(self.Num_Chn): -# self.FitGau(args[n]) -# dataOut.GauSPC = ret - - - -# for ch in range(self.Num_Chn): -# -# for ht in range(self.Num_Hei): -# #print (numpy.asarray(self.spc).shape) -# spc = numpy.asarray(self.spc)[ch,:,ht] -# -# ############################################# -# # normalizing spc and noise -# # This part differs from gg1 -# spc_norm_max = max(spc) -# spc = spc / spc_norm_max -# pnoise = pnoise / spc_norm_max -# ############################################# -# -# if abs(vel_arr[0])<15.0: # this switch is for spectra collected with different length IPP's -# fatspectra=1.0 -# else: -# fatspectra=0.5 -# -# wnoise = noise_ / spc_norm_max -# #print 'wnoise', noise_, dataOut.spc_noise[0], wnoise -# #wnoise,stdv,i_max,index =enoise(spc,num_intg) #noise estimate using Hildebrand Sekhon, only wnoise is used -# #if wnoise>1.1*pnoise: # to be tested later -# # wnoise=pnoise -# noisebl=wnoise*0.9; noisebh=wnoise*1.1 -# spc=spc-wnoise -# -# minx=numpy.argmin(spc) -# spcs=numpy.roll(spc,-minx) -# cum=numpy.cumsum(spcs) -# tot_noise=wnoise * self.Num_Bin #64; -# #tot_signal=sum(cum[-5:])/5.; ''' How does this line work? ''' -# #snr=tot_signal/tot_noise -# #snr=cum[-1]/tot_noise -# -# #print 'spc' , spcs[5:8] , 'tot_noise', tot_noise -# -# snr = sum(spcs)/tot_noise -# snrdB=10.*numpy.log10(snr) -# -# #if snrdB < -9 : -# # snrdB = numpy.NaN -# # continue -# -# #print 'snr',snrdB # , sum(spcs) , tot_noise -# -# -# #if snrdB<-18 or numpy.isnan(snrdB) or num_intg<4: -# # return [None,]*4,[None,]*4,None,snrdB,None,None,[None,]*5,[None,]*9,None -# -# cummax=max(cum); epsi=0.08*fatspectra # cumsum to narrow down the energy region -# cumlo=cummax*epsi; -# cumhi=cummax*(1-epsi) -# powerindex=numpy.array(numpy.where(numpy.logical_and(cum>cumlo, cum-9: # when SNR is strong pick the peak with least shift (LOS velocity) error -# if oneG: -# choice=0 -# else: -# w1=lsq2[0][1]; w2=lsq2[0][5] -# a1=lsq2[0][2]; a2=lsq2[0][6] -# p1=lsq2[0][3]; p2=lsq2[0][7] -# s1=(2**(1+1./p1))*scipy.special.gamma(1./p1)/p1; s2=(2**(1+1./p2))*scipy.special.gamma(1./p2)/p2; -# gp1=a1*w1*s1; gp2=a2*w2*s2 # power content of each ggaussian with proper p scaling -# -# if gp1>gp2: -# if a1>0.7*a2: -# choice=1 -# else: -# choice=2 -# elif gp2>gp1: -# if a2>0.7*a1: -# choice=2 -# else: -# choice=1 -# else: -# choice=numpy.argmax([a1,a2])+1 -# #else: -# #choice=argmin([std2a,std2b])+1 -# -# else: # with low SNR go to the most energetic peak -# choice=numpy.argmax([lsq1[0][2]*lsq1[0][1],lsq2[0][2]*lsq2[0][1],lsq2[0][6]*lsq2[0][5]]) -# -# #print 'choice',choice -# -# if choice==0: # pick the single gaussian fit -# Amplitude0=lsq1[0][2] -# shift0=lsq1[0][0] -# width0=lsq1[0][1] -# p0=lsq1[0][3] -# Amplitude1=0. -# shift1=0. -# width1=0. -# p1=0. -# noise=lsq1[0][4] -# elif choice==1: # take the first one of the 2 gaussians fitted -# Amplitude0 = lsq2[0][2] -# shift0 = lsq2[0][0] -# width0 = lsq2[0][1] -# p0 = lsq2[0][3] -# Amplitude1 = lsq2[0][6] # This is 0 in gg1 -# shift1 = lsq2[0][4] # This is 0 in gg1 -# width1 = lsq2[0][5] # This is 0 in gg1 -# p1 = lsq2[0][7] # This is 0 in gg1 -# noise = lsq2[0][8] -# else: # the second one -# Amplitude0 = lsq2[0][6] -# shift0 = lsq2[0][4] -# width0 = lsq2[0][5] -# p0 = lsq2[0][7] -# Amplitude1 = lsq2[0][2] # This is 0 in gg1 -# shift1 = lsq2[0][0] # This is 0 in gg1 -# width1 = lsq2[0][1] # This is 0 in gg1 -# p1 = lsq2[0][3] # This is 0 in gg1 -# noise = lsq2[0][8] -# -# #print len(noise + Amplitude0*numpy.exp(-0.5*(abs(x-shift0))/width0)**p0) -# SPC_ch1[:,ht] = noise + Amplitude0*numpy.exp(-0.5*(abs(x-shift0))/width0)**p0 -# SPC_ch2[:,ht] = noise + Amplitude1*numpy.exp(-0.5*(abs(x-shift1))/width1)**p1 -# #print 'SPC_ch1.shape',SPC_ch1.shape -# #print 'SPC_ch2.shape',SPC_ch2.shape -# #dataOut.data_param = SPC_ch1 -# GauSPC[0] = SPC_ch1 -# GauSPC[1] = SPC_ch2 - -# #plt.gcf().clear() -# plt.figure(50+self.i) -# self.i=self.i+1 -# #plt.subplot(121) -# plt.plot(self.spc,'k')#,label='spc(66)') -# plt.plot(SPC_ch1[ch,ht],'b')#,label='gg1') -# #plt.plot(SPC_ch2,'r')#,label='gg2') -# #plt.plot(xFrec,ySamples[1],'g',label='Ch1') -# #plt.plot(xFrec,ySamples[2],'r',label='Ch2') -# #plt.plot(xFrec,FitGauss,'yo:',label='fit') -# plt.legend() -# plt.title('DATOS A ALTURA DE 7500 METROS') -# plt.show() -# print 'shift0', shift0 -# print 'Amplitude0', Amplitude0 -# print 'width0', width0 -# print 'p0', p0 -# print '========================' -# print 'shift1', shift1 -# print 'Amplitude1', Amplitude1 -# print 'width1', width1 -# print 'p1', p1 -# print 'noise', noise -# print 's_noise', wnoise - - print '========================================================' - print 'total_time: ', time.time()-start_time - - # re-normalizing spc and noise - # This part differs from gg1 - - - + dataOut.SPCparam = numpy.asarray(SPCparam) + ''' Parameters: 1. Amplitude 2. Shift 3. Width 4. Power ''' - - - ############################################################################### + def FitGau(self, X): - + Vrange, ch, pnoise, noise_, num_intg, SNRlimit = X - #print 'VARSSSS', ch, pnoise, noise, num_intg - - #print 'HEIGHTS', self.Num_Hei - - GauSPC = [] + + SPCparam = [] SPC_ch1 = numpy.empty([self.Num_Bin,self.Num_Hei]) SPC_ch2 = numpy.empty([self.Num_Bin,self.Num_Hei]) SPC_ch1[:] = 0#numpy.NaN SPC_ch2[:] = 0#numpy.NaN - - - + + + for ht in range(self.Num_Hei): - #print (numpy.asarray(self.spc).shape) - - #print 'TTTTT', ch , ht - #print self.spc.shape - - + + spc = numpy.asarray(self.spc)[ch,:,ht] - + ############################################# # normalizing spc and noise # This part differs from gg1 spc_norm_max = max(spc) - spc = spc / spc_norm_max - pnoise = pnoise / spc_norm_max + #spc = spc / spc_norm_max + pnoise = pnoise #/ spc_norm_max ############################################# - + fatspectra=1.0 - - wnoise = noise_ / spc_norm_max + + wnoise = noise_ #/ spc_norm_max #wnoise,stdv,i_max,index =enoise(spc,num_intg) #noise estimate using Hildebrand Sekhon, only wnoise is used - #if wnoise>1.1*pnoise: # to be tested later + #if wnoise>1.1*pnoise: # to be tested later # wnoise=pnoise - noisebl=wnoise*0.9; noisebh=wnoise*1.1 + noisebl=wnoise*0.9; + noisebh=wnoise*1.1 spc=spc-wnoise - # print 'wnoise', noise_[0], spc_norm_max, wnoise + minx=numpy.argmin(spc) + #spcs=spc.copy() spcs=numpy.roll(spc,-minx) cum=numpy.cumsum(spcs) tot_noise=wnoise * self.Num_Bin #64; - #print 'spc' , spcs[5:8] , 'tot_noise', tot_noise - #tot_signal=sum(cum[-5:])/5.; ''' How does this line work? ''' - #snr=tot_signal/tot_noise - #snr=cum[-1]/tot_noise + snr = sum(spcs)/tot_noise snrdB=10.*numpy.log10(snr) - + if snrdB < SNRlimit : snr = numpy.NaN SPC_ch1[:,ht] = 0#numpy.NaN SPC_ch1[:,ht] = 0#numpy.NaN - GauSPC = (SPC_ch1,SPC_ch2) + SPCparam = (SPC_ch1,SPC_ch2) continue - #print 'snr',snrdB #, sum(spcs) , tot_noise - - - + + #if snrdB<-18 or numpy.isnan(snrdB) or num_intg<4: # return [None,]*4,[None,]*4,None,snrdB,None,None,[None,]*5,[None,]*9,None - - cummax=max(cum); epsi=0.08*fatspectra # cumsum to narrow down the energy region - cumlo=cummax*epsi; + + cummax=max(cum); + epsi=0.08*fatspectra # cumsum to narrow down the energy region + cumlo=cummax*epsi; cumhi=cummax*(1-epsi) powerindex=numpy.array(numpy.where(numpy.logical_and(cum>cumlo, cum-6: # when SNR is strong pick the peak with least shift (LOS velocity) error + + if snrdB>-12: # when SNR is strong pick the peak with least shift (LOS velocity) error if oneG: choice=0 else: w1=lsq2[0][1]; w2=lsq2[0][5] a1=lsq2[0][2]; a2=lsq2[0][6] p1=lsq2[0][3]; p2=lsq2[0][7] - s1=(2**(1+1./p1))*scipy.special.gamma(1./p1)/p1; + s1=(2**(1+1./p1))*scipy.special.gamma(1./p1)/p1; s2=(2**(1+1./p2))*scipy.special.gamma(1./p2)/p2; gp1=a1*w1*s1; gp2=a2*w2*s2 # power content of each ggaussian with proper p scaling - + if gp1>gp2: if a1>0.7*a2: choice=1 @@ -702,219 +513,157 @@ class GaussianFit(Operation): choice=numpy.argmax([a1,a2])+1 #else: #choice=argmin([std2a,std2b])+1 - + else: # with low SNR go to the most energetic peak choice=numpy.argmax([lsq1[0][2]*lsq1[0][1],lsq2[0][2]*lsq2[0][1],lsq2[0][6]*lsq2[0][5]]) - - - shift0=lsq2[0][0]; vel0=Vrange[0] + shift0*(Vrange[1]-Vrange[0]) - shift1=lsq2[0][4]; vel1=Vrange[0] + shift1*(Vrange[1]-Vrange[0]) - - max_vel = 20 - + + + shift0=lsq2[0][0]; + vel0=Vrange[0] + shift0*(Vrange[1]-Vrange[0]) + shift1=lsq2[0][4]; + vel1=Vrange[0] + shift1*(Vrange[1]-Vrange[0]) + + max_vel = 1.0 + #first peak will be 0, second peak will be 1 - if vel0 > 0 and vel0 < max_vel : #first peak is in the correct range + if vel0 > -1.0 and vel0 < max_vel : #first peak is in the correct range shift0=lsq2[0][0] width0=lsq2[0][1] Amplitude0=lsq2[0][2] p0=lsq2[0][3] - + shift1=lsq2[0][4] width1=lsq2[0][5] Amplitude1=lsq2[0][6] p1=lsq2[0][7] - noise=lsq2[0][8] + noise=lsq2[0][8] else: shift1=lsq2[0][0] width1=lsq2[0][1] Amplitude1=lsq2[0][2] p1=lsq2[0][3] - + shift0=lsq2[0][4] width0=lsq2[0][5] Amplitude0=lsq2[0][6] - p0=lsq2[0][7] - noise=lsq2[0][8] - - if Amplitude0<0.1: # in case the peak is noise - shift0,width0,Amplitude0,p0 = 4*[numpy.NaN] - if Amplitude1<0.1: - shift1,width1,Amplitude1,p1 = 4*[numpy.NaN] - - -# if choice==0: # pick the single gaussian fit -# Amplitude0=lsq1[0][2] -# shift0=lsq1[0][0] -# width0=lsq1[0][1] -# p0=lsq1[0][3] -# Amplitude1=0. -# shift1=0. -# width1=0. -# p1=0. -# noise=lsq1[0][4] -# elif choice==1: # take the first one of the 2 gaussians fitted -# Amplitude0 = lsq2[0][2] -# shift0 = lsq2[0][0] -# width0 = lsq2[0][1] -# p0 = lsq2[0][3] -# Amplitude1 = lsq2[0][6] # This is 0 in gg1 -# shift1 = lsq2[0][4] # This is 0 in gg1 -# width1 = lsq2[0][5] # This is 0 in gg1 -# p1 = lsq2[0][7] # This is 0 in gg1 -# noise = lsq2[0][8] -# else: # the second one -# Amplitude0 = lsq2[0][6] -# shift0 = lsq2[0][4] -# width0 = lsq2[0][5] -# p0 = lsq2[0][7] -# Amplitude1 = lsq2[0][2] # This is 0 in gg1 -# shift1 = lsq2[0][0] # This is 0 in gg1 -# width1 = lsq2[0][1] # This is 0 in gg1 -# p1 = lsq2[0][3] # This is 0 in gg1 -# noise = lsq2[0][8] - - #print len(noise + Amplitude0*numpy.exp(-0.5*(abs(x-shift0))/width0)**p0) + p0=lsq2[0][7] + noise=lsq2[0][8] + + if Amplitude0<0.05: # in case the peak is noise + shift0,width0,Amplitude0,p0 = [0,0,0,0]#4*[numpy.NaN] + if Amplitude1<0.05: + shift1,width1,Amplitude1,p1 = [0,0,0,0]#4*[numpy.NaN] + + SPC_ch1[:,ht] = noise + Amplitude0*numpy.exp(-0.5*(abs(x-shift0))/width0)**p0 SPC_ch2[:,ht] = noise + Amplitude1*numpy.exp(-0.5*(abs(x-shift1))/width1)**p1 - #print 'SPC_ch1.shape',SPC_ch1.shape - #print 'SPC_ch2.shape',SPC_ch2.shape - #dataOut.data_param = SPC_ch1 - GauSPC = (SPC_ch1,SPC_ch2) - #GauSPC[1] = SPC_ch2 - -# print 'shift0', shift0 -# print 'Amplitude0', Amplitude0 -# print 'width0', width0 -# print 'p0', p0 -# print '========================' -# print 'shift1', shift1 -# print 'Amplitude1', Amplitude1 -# print 'width1', width1 -# print 'p1', p1 -# print 'noise', noise -# print 's_noise', wnoise - + SPCparam = (SPC_ch1,SPC_ch2) + + return GauSPC - - - def y_jacobian1(self,x,state): # This function is for further analysis of generalized Gaussians, it is not too importan for the signal discrimination. - y_model=self.y_model1(x,state) - s0,w0,a0,p0,n=state - e0=((x-s0)/w0)**2; - - e0u=((x-s0-self.Num_Bin)/w0)**2; - - e0d=((x-s0+self.Num_Bin)/w0)**2 - m0=numpy.exp(-0.5*e0**(p0/2.)); - m0u=numpy.exp(-0.5*e0u**(p0/2.)); - m0d=numpy.exp(-0.5*e0d**(p0/2.)) - JA=m0+m0u+m0d - JP=(-1/4.)*a0*m0*e0**(p0/2.)*numpy.log(e0)+(-1/4.)*a0*m0u*e0u**(p0/2.)*numpy.log(e0u)+(-1/4.)*a0*m0d*e0d**(p0/2.)*numpy.log(e0d) - - JS=(p0/w0/2.)*a0*m0*e0**(p0/2.-1)*((x-s0)/w0)+(p0/w0/2.)*a0*m0u*e0u**(p0/2.-1)*((x-s0- self.Num_Bin )/w0)+(p0/w0/2.)*a0*m0d*e0d**(p0/2.-1)*((x-s0+ self.Num_Bin )/w0) - - JW=(p0/w0/2.)*a0*m0*e0**(p0/2.-1)*((x-s0)/w0)**2+(p0/w0/2.)*a0*m0u*e0u**(p0/2.-1)*((x-s0- self.Num_Bin )/w0)**2+(p0/w0/2.)*a0*m0d*e0d**(p0/2.-1)*((x-s0+ self.Num_Bin )/w0)**2 - jack1=numpy.sqrt(7)*numpy.array([JS/y_model,JW/y_model,JA/y_model,JP/y_model,1./y_model]) - return jack1.T - - def y_jacobian2(self,x,state): - y_model=self.y_model2(x,state) - s0,w0,a0,p0,s1,w1,a1,p1,n=state - e0=((x-s0)/w0)**2; - - e0u=((x-s0- self.Num_Bin )/w0)**2; - - e0d=((x-s0+ self.Num_Bin )/w0)**2 - e1=((x-s1)/w1)**2; - - e1u=((x-s1- self.Num_Bin )/w1)**2; - - e1d=((x-s1+ self.Num_Bin )/w1)**2 - m0=numpy.exp(-0.5*e0**(p0/2.)); - m0u=numpy.exp(-0.5*e0u**(p0/2.)); - m0d=numpy.exp(-0.5*e0d**(p0/2.)) - m1=numpy.exp(-0.5*e1**(p1/2.)); - m1u=numpy.exp(-0.5*e1u**(p1/2.)); - m1d=numpy.exp(-0.5*e1d**(p1/2.)) - JA=m0+m0u+m0d - JA1=m1+m1u+m1d - JP=(-1/4.)*a0*m0*e0**(p0/2.)*numpy.log(e0)+(-1/4.)*a0*m0u*e0u**(p0/2.)*numpy.log(e0u)+(-1/4.)*a0*m0d*e0d**(p0/2.)*numpy.log(e0d) - JP1=(-1/4.)*a1*m1*e1**(p1/2.)*numpy.log(e1)+(-1/4.)*a1*m1u*e1u**(p1/2.)*numpy.log(e1u)+(-1/4.)*a1*m1d*e1d**(p1/2.)*numpy.log(e1d) - - JS=(p0/w0/2.)*a0*m0*e0**(p0/2.-1)*((x-s0)/w0)+(p0/w0/2.)*a0*m0u*e0u**(p0/2.-1)*((x-s0- self.Num_Bin )/w0)+(p0/w0/2.)*a0*m0d*e0d**(p0/2.-1)*((x-s0+ self.Num_Bin )/w0) - - JS1=(p1/w1/2.)*a1*m1*e1**(p1/2.-1)*((x-s1)/w1)+(p1/w1/2.)*a1*m1u*e1u**(p1/2.-1)*((x-s1- self.Num_Bin )/w1)+(p1/w1/2.)*a1*m1d*e1d**(p1/2.-1)*((x-s1+ self.Num_Bin )/w1) - - JW=(p0/w0/2.)*a0*m0*e0**(p0/2.-1)*((x-s0)/w0)**2+(p0/w0/2.)*a0*m0u*e0u**(p0/2.-1)*((x-s0- self.Num_Bin )/w0)**2+(p0/w0/2.)*a0*m0d*e0d**(p0/2.-1)*((x-s0+ self.Num_Bin )/w0)**2 - - JW1=(p1/w1/2.)*a1*m1*e1**(p1/2.-1)*((x-s1)/w1)**2+(p1/w1/2.)*a1*m1u*e1u**(p1/2.-1)*((x-s1- self.Num_Bin )/w1)**2+(p1/w1/2.)*a1*m1d*e1d**(p1/2.-1)*((x-s1+ self.Num_Bin )/w1)**2 - jack2=numpy.sqrt(7)*numpy.array([JS/y_model,JW/y_model,JA/y_model,JP/y_model,JS1/y_model,JW1/y_model,JA1/y_model,JP1/y_model,1./y_model]) - return jack2.T - + def y_model1(self,x,state): shift0,width0,amplitude0,power0,noise=state model0=amplitude0*numpy.exp(-0.5*abs((x-shift0)/width0)**power0) - + model0u=amplitude0*numpy.exp(-0.5*abs((x-shift0- self.Num_Bin )/width0)**power0) - + model0d=amplitude0*numpy.exp(-0.5*abs((x-shift0+ self.Num_Bin )/width0)**power0) return model0+model0u+model0d+noise - - def y_model2(self,x,state): #Equation for two generalized Gaussians with Nyquist + + def y_model2(self,x,state): #Equation for two generalized Gaussians with Nyquist shift0,width0,amplitude0,power0,shift1,width1,amplitude1,power1,noise=state model0=amplitude0*numpy.exp(-0.5*abs((x-shift0)/width0)**power0) - + model0u=amplitude0*numpy.exp(-0.5*abs((x-shift0- self.Num_Bin )/width0)**power0) - + model0d=amplitude0*numpy.exp(-0.5*abs((x-shift0+ self.Num_Bin )/width0)**power0) model1=amplitude1*numpy.exp(-0.5*abs((x-shift1)/width1)**power1) - + model1u=amplitude1*numpy.exp(-0.5*abs((x-shift1- self.Num_Bin )/width1)**power1) - + model1d=amplitude1*numpy.exp(-0.5*abs((x-shift1+ self.Num_Bin )/width1)**power1) return model0+model0u+model0d+model1+model1u+model1d+noise - - def misfit1(self,state,y_data,x,num_intg): # This function compares how close real data is with the model data, the close it is, the better it is. + + def misfit1(self,state,y_data,x,num_intg): # This function compares how close real data is with the model data, the close it is, the better it is. return num_intg*sum((numpy.log(y_data)-numpy.log(self.y_model1(x,state)))**2)#/(64-5.) # /(64-5.) can be commented - + def misfit2(self,state,y_data,x,num_intg): return num_intg*sum((numpy.log(y_data)-numpy.log(self.y_model2(x,state)))**2)#/(64-9.) - + + class PrecipitationProc(Operation): - + ''' Operator that estimates Reflectivity factor (Z), and estimates rainfall Rate (R) - - Input: + + Input: self.dataOut.data_pre : SelfSpectra - - Output: - - self.dataOut.data_output : Reflectivity factor, rainfall Rate - - - Parameters affected: + + Output: + + self.dataOut.data_output : Reflectivity factor, rainfall Rate + + + Parameters affected: ''' - - - def run(self, dataOut, radar=None, Pt=None, Gt=None, Gr=None, Lambda=None, aL=None, - tauW=None, ThetaT=None, ThetaR=None, Km = 0.93, Altitude=None): - - self.spc = dataOut.data_pre[0].copy() - self.Num_Hei = self.spc.shape[2] - self.Num_Bin = self.spc.shape[1] - self.Num_Chn = self.spc.shape[0] - - Velrange = dataOut.abscissaList - + + def __init__(self): + Operation.__init__(self) + self.i=0 + + + def gaus(self,xSamples,Amp,Mu,Sigma): + return ( Amp / ((2*numpy.pi)**0.5 * Sigma) ) * numpy.exp( -( xSamples - Mu )**2 / ( 2 * (Sigma**2) )) + + + + def Moments(self, ySamples, xSamples): + Pot = numpy.nansum( ySamples ) # Potencia, momento 0 + yNorm = ySamples / Pot + + Vr = numpy.nansum( yNorm * xSamples ) # Velocidad radial, mu, corrimiento doppler, primer momento + Sigma2 = abs(numpy.nansum( yNorm * ( xSamples - Vr )**2 )) # Segundo Momento + Desv = Sigma2**0.5 # Desv. Estandar, Ancho espectral + + return numpy.array([Pot, Vr, Desv]) + + def run(self, dataOut, radar=None, Pt=5000, Gt=295.1209, Gr=70.7945, Lambda=0.6741, aL=2.5118, + tauW=4e-06, ThetaT=0.1656317, ThetaR=0.36774087, Km = 0.93, Altitude=3350): + + + Velrange = dataOut.spcparam_range[2] + FrecRange = dataOut.spcparam_range[0] + + dV= Velrange[1]-Velrange[0] + dF= FrecRange[1]-FrecRange[0] + if radar == "MIRA35C" : - + + self.spc = dataOut.data_pre[0].copy() + self.Num_Hei = self.spc.shape[2] + self.Num_Bin = self.spc.shape[1] + self.Num_Chn = self.spc.shape[0] Ze = self.dBZeMODE2(dataOut) - + else: - + + self.spc = dataOut.SPCparam[1].copy() #dataOut.data_pre[0].copy() # + + """NOTA SE DEBE REMOVER EL RANGO DEL PULSO TX""" + + self.spc[:,:,0:7]= numpy.NaN + + """##########################################""" + + self.Num_Hei = self.spc.shape[2] + self.Num_Bin = self.spc.shape[1] + self.Num_Chn = self.spc.shape[0] + + ''' Se obtiene la constante del RADAR ''' + self.Pt = Pt self.Gt = Gt self.Gr = Gr @@ -923,493 +672,699 @@ class PrecipitationProc(Operation): self.tauW = tauW self.ThetaT = ThetaT self.ThetaR = ThetaR - - RadarConstant = GetRadarConstant() - SPCmean = numpy.mean(self.spc,0) - ETA = numpy.zeros(self.Num_Hei) - Pr = numpy.sum(SPCmean,0) - - #for R in range(self.Num_Hei): - # ETA[R] = RadarConstant * Pr[R] * R**2 #Reflectivity (ETA) - - D_range = numpy.zeros(self.Num_Hei) - EqSec = numpy.zeros(self.Num_Hei) + + Numerator = ( (4*numpy.pi)**3 * aL**2 * 16 * numpy.log(2) ) + Denominator = ( Pt * Gt * Gr * Lambda**2 * SPEED_OF_LIGHT * tauW * numpy.pi * ThetaT * ThetaR) + RadarConstant = 10e-26 * Numerator / Denominator # + + ''' ============================= ''' + + self.spc[0] = (self.spc[0]-dataOut.noise[0]) + self.spc[1] = (self.spc[1]-dataOut.noise[1]) + self.spc[2] = (self.spc[2]-dataOut.noise[2]) + + self.spc[ numpy.where(self.spc < 0)] = 0 + + SPCmean = (numpy.mean(self.spc,0) - numpy.mean(dataOut.noise)) + SPCmean[ numpy.where(SPCmean < 0)] = 0 + + ETAn = numpy.zeros([self.Num_Bin,self.Num_Hei]) + ETAv = numpy.zeros([self.Num_Bin,self.Num_Hei]) + ETAd = numpy.zeros([self.Num_Bin,self.Num_Hei]) + + Pr = SPCmean[:,:] + + VelMeteoro = numpy.mean(SPCmean,axis=0) + + D_range = numpy.zeros([self.Num_Bin,self.Num_Hei]) + SIGMA = numpy.zeros([self.Num_Bin,self.Num_Hei]) + N_dist = numpy.zeros([self.Num_Bin,self.Num_Hei]) + V_mean = numpy.zeros(self.Num_Hei) del_V = numpy.zeros(self.Num_Hei) - + Z = numpy.zeros(self.Num_Hei) + Ze = numpy.zeros(self.Num_Hei) + RR = numpy.zeros(self.Num_Hei) + + Range = dataOut.heightList*1000. + for R in range(self.Num_Hei): - ETA[R] = RadarConstant * Pr[R] * R**2 #Reflectivity (ETA) - - h = R + Altitude #Range from ground to radar pulse altitude + + h = Range[R] + Altitude #Range from ground to radar pulse altitude del_V[R] = 1 + 3.68 * 10**-5 * h + 1.71 * 10**-9 * h**2 #Density change correction for velocity - - D_range[R] = numpy.log( (9.65 - (Velrange[R]/del_V[R])) / 10.3 ) / -0.6 #Range of Diameter of drops related to velocity - SIGMA[R] = numpy.pi**5 / Lambda**4 * Km * D_range[R]**6 #Equivalent Section of drops (sigma) - - N_dist[R] = ETA[R] / SIGMA[R] - - Ze = (ETA * Lambda**4) / (numpy.pi * Km) - Z = numpy.sum( N_dist * D_range**6 ) - RR = 6*10**-4*numpy.pi * numpy.sum( D_range**3 * N_dist * Velrange ) #Rainfall rate - - - RR = (Ze/200)**(1/1.6) + + D_range[:,R] = numpy.log( (9.65 - (Velrange[0:self.Num_Bin] / del_V[R])) / 10.3 ) / -0.6 #Diameter range [m]x10**-3 + + '''NOTA: ETA(n) dn = ETA(f) df + + dn = 1 Diferencial de muestreo + df = ETA(n) / ETA(f) + + ''' + + ETAn[:,R] = RadarConstant * Pr[:,R] * (Range[R] )**2 #Reflectivity (ETA) + + ETAv[:,R]=ETAn[:,R]/dV + + ETAd[:,R]=ETAv[:,R]*6.18*exp(-0.6*D_range[:,R]) + + SIGMA[:,R] = Km * (D_range[:,R] * 1e-3 )**6 * numpy.pi**5 / Lambda**4 #Equivalent Section of drops (sigma) + + N_dist[:,R] = ETAn[:,R] / SIGMA[:,R] + + DMoments = self.Moments(Pr[:,R], Velrange[0:self.Num_Bin]) + + try: + popt01,pcov = curve_fit(self.gaus, Velrange[0:self.Num_Bin] , Pr[:,R] , p0=DMoments) + except: + popt01=numpy.zeros(3) + popt01[1]= DMoments[1] + + if popt01[1]<0 or popt01[1]>20: + popt01[1]=numpy.NaN + + + V_mean[R]=popt01[1] + + Z[R] = numpy.nansum( N_dist[:,R] * (D_range[:,R])**6 )#*10**-18 + + RR[R] = 0.0006*numpy.pi * numpy.nansum( D_range[:,R]**3 * N_dist[:,R] * Velrange[0:self.Num_Bin] ) #Rainfall rate + + Ze[R] = (numpy.nansum( ETAn[:,R]) * Lambda**4) / ( 10**-18*numpy.pi**5 * Km) + + + + RR2 = (Z/200)**(1/1.6) dBRR = 10*numpy.log10(RR) - + dBRR2 = 10*numpy.log10(RR2) + dBZe = 10*numpy.log10(Ze) - dataOut.data_output = Ze - dataOut.data_param = numpy.ones([2,self.Num_Hei]) - dataOut.channelList = [0,1] - print 'channelList', dataOut.channelList - dataOut.data_param[0]=dBZe - dataOut.data_param[1]=dBRR - print 'RR SHAPE', dBRR.shape - print 'Ze SHAPE', dBZe.shape - print 'dataOut.data_param SHAPE', dataOut.data_param.shape - - + dBZ = 10*numpy.log10(Z) + + dataOut.data_output = RR[8] + dataOut.data_param = numpy.ones([3,self.Num_Hei]) + dataOut.channelList = [0,1,2] + + dataOut.data_param[0]=dBZ + dataOut.data_param[1]=V_mean + dataOut.data_param[2]=RR + + return dataOut + def dBZeMODE2(self, dataOut): # Processing for MIRA35C - + NPW = dataOut.NPW COFA = dataOut.COFA - + SNR = numpy.array([self.spc[0,:,:] / NPW[0]]) #, self.spc[1,:,:] / NPW[1]]) RadarConst = dataOut.RadarConst #frequency = 34.85*10**9 - + ETA = numpy.zeros(([self.Num_Chn ,self.Num_Hei])) data_output = numpy.ones([self.Num_Chn , self.Num_Hei])*numpy.NaN - + ETA = numpy.sum(SNR,1) - print 'ETA' , ETA + ETA = numpy.where(ETA is not 0. , ETA, numpy.NaN) - + Ze = numpy.ones([self.Num_Chn, self.Num_Hei] ) - + for r in range(self.Num_Hei): - + Ze[0,r] = ( ETA[0,r] ) * COFA[0,r][0] * RadarConst * ((r/5000.)**2) #Ze[1,r] = ( ETA[1,r] ) * COFA[1,r][0] * RadarConst * ((r/5000.)**2) - + return Ze - - def GetRadarConstant(self): - - """ - Constants: - - Pt: Transmission Power dB - Gt: Transmission Gain dB - Gr: Reception Gain dB - Lambda: Wavelenght m - aL: Attenuation loses dB - tauW: Width of transmission pulse s - ThetaT: Transmission antenna bean angle rad - ThetaR: Reception antenna beam angle rad - - """ - Numerator = ( (4*numpy.pi)**3 * aL**2 * 16 * numpy.log(2) ) - Denominator = ( Pt * Gt * Gr * Lambda**2 * SPEED_OF_LIGHT * TauW * numpy.pi * ThetaT * TheraR) - RadarConstant = Numerator / Denominator - - return RadarConstant - - - -class FullSpectralAnalysis(Operation): - + +# def GetRadarConstant(self): +# +# """ +# Constants: +# +# Pt: Transmission Power dB 5kW 5000 +# Gt: Transmission Gain dB 24.7 dB 295.1209 +# Gr: Reception Gain dB 18.5 dB 70.7945 +# Lambda: Wavelenght m 0.6741 m 0.6741 +# aL: Attenuation loses dB 4dB 2.5118 +# tauW: Width of transmission pulse s 4us 4e-6 +# ThetaT: Transmission antenna bean angle rad 0.1656317 rad 0.1656317 +# ThetaR: Reception antenna beam angle rad 0.36774087 rad 0.36774087 +# +# """ +# +# Numerator = ( (4*numpy.pi)**3 * aL**2 * 16 * numpy.log(2) ) +# Denominator = ( Pt * Gt * Gr * Lambda**2 * SPEED_OF_LIGHT * TauW * numpy.pi * ThetaT * TheraR) +# RadarConstant = Numerator / Denominator +# +# return RadarConstant + + + +class FullSpectralAnalysis(Operation): + """ - Function that implements Full Spectral Analisys technique. - - Input: - self.dataOut.data_pre : SelfSpectra and CrossSPectra data + Function that implements Full Spectral Analysis technique. + + Input: + self.dataOut.data_pre : SelfSpectra and CrossSpectra data self.dataOut.groupList : Pairlist of channels self.dataOut.ChanDist : Physical distance between receivers - - - Output: - - self.dataOut.data_output : Zonal wind, Meridional wind and Vertical wind - - + + + Output: + + self.dataOut.data_output : Zonal wind, Meridional wind and Vertical wind + + Parameters affected: Winds, height range, SNR - + """ - def run(self, dataOut, E01=None, E02=None, E12=None, N01=None, N02=None, N12=None, SNRlimit=7): - + def run(self, dataOut, Xi01=None, Xi02=None, Xi12=None, Eta01=None, Eta02=None, Eta12=None, SNRlimit=7, minheight=None, maxheight=None): + + self.indice=int(numpy.random.rand()*1000) + spc = dataOut.data_pre[0].copy() - cspc = dataOut.data_pre[1].copy() - + cspc = dataOut.data_pre[1] + + """Erick: NOTE THE RANGE OF THE PULSE TX MUST BE REMOVED""" + + SNRspc = spc.copy() + SNRspc[:,:,0:7]= numpy.NaN + + """##########################################""" + + nChannel = spc.shape[0] nProfiles = spc.shape[1] nHeights = spc.shape[2] - + + # first_height = 0.75 #km (ref: data header 20170822) + # resolution_height = 0.075 #km + ''' + finding height range. check this when radar parameters are changed! + ''' + if maxheight is not None: + # range_max = math.ceil((maxheight - first_height) / resolution_height) # theoretical + range_max = math.ceil(13.26 * maxheight - 3) # empirical, works better + else: + range_max = nHeights + if minheight is not None: + # range_min = int((minheight - first_height) / resolution_height) # theoretical + range_min = int(13.26 * minheight - 5) # empirical, works better + if range_min < 0: + range_min = 0 + else: + range_min = 0 + pairsList = dataOut.groupList if dataOut.ChanDist is not None : ChanDist = dataOut.ChanDist else: - ChanDist = numpy.array([[E01, N01],[E02,N02],[E12,N12]]) - - #print 'ChanDist', ChanDist - - if dataOut.VelRange is not None: - VelRange= dataOut.VelRange - else: - VelRange= dataOut.abscissaList - - ySamples=numpy.ones([nChannel,nProfiles]) - phase=numpy.ones([nChannel,nProfiles]) - CSPCSamples=numpy.ones([nChannel,nProfiles],dtype=numpy.complex_) - coherence=numpy.ones([nChannel,nProfiles]) - PhaseSlope=numpy.ones(nChannel) - PhaseInter=numpy.ones(nChannel) - dataSNR = dataOut.data_SNR - - - - data = dataOut.data_pre + ChanDist = numpy.array([[Xi01, Eta01],[Xi02,Eta02],[Xi12,Eta12]]) + + FrecRange = dataOut.spc_range[0] + + data_SNR=numpy.zeros([nProfiles]) noise = dataOut.noise - print 'noise',noise - #SNRdB = 10*numpy.log10(dataOut.data_SNR) - - FirstMoment = numpy.average(dataOut.data_param[:,1,:],0) - #SNRdBMean = [] - - - #for j in range(nHeights): - # FirstMoment = numpy.append(FirstMoment,numpy.mean([dataOut.data_param[0,1,j],dataOut.data_param[1,1,j],dataOut.data_param[2,1,j]])) - # SNRdBMean = numpy.append(SNRdBMean,numpy.mean([SNRdB[0,j],SNRdB[1,j],SNRdB[2,j]])) - - data_output=numpy.ones([3,spc.shape[2]])*numpy.NaN - + + dataOut.data_SNR = (numpy.mean(SNRspc,axis=1)- noise[0]) / noise[0] + + dataOut.data_SNR[numpy.where( dataOut.data_SNR <0 )] = 1e-20 + + + data_output=numpy.ones([spc.shape[0],spc.shape[2]])*numpy.NaN + velocityX=[] velocityY=[] velocityV=[] - - dbSNR = 10*numpy.log10(dataSNR) + + dbSNR = 10*numpy.log10(dataOut.data_SNR) dbSNR = numpy.average(dbSNR,0) + + '''***********************************************WIND ESTIMATION**************************************''' + for Height in range(nHeights): - - [Vzon,Vmer,Vver, GaussCenter]= self.WindEstimation(spc, cspc, pairsList, ChanDist, Height, noise, VelRange, dbSNR[Height], SNRlimit) - - if abs(Vzon)<100. and abs(Vzon)> 0.: - velocityX=numpy.append(velocityX, Vzon)#Vmag - + + if Height >= range_min and Height < range_max: + # error_code unused, yet maybe useful for future analysis. + [Vzon,Vmer,Vver, error_code] = self.WindEstimation(spc[:,:,Height], cspc[:,:,Height], pairsList, ChanDist, Height, noise, dataOut.spc_range, dbSNR[Height], SNRlimit) else: - print 'Vzon',Vzon - velocityX=numpy.append(velocityX, numpy.NaN) - - if abs(Vmer)<100. and abs(Vmer) > 0.: - velocityY=numpy.append(velocityY, Vmer)#Vang - + Vzon,Vmer,Vver = 0., 0., numpy.NaN + + + if abs(Vzon) < 100. and abs(Vzon) > 0. and abs(Vmer) < 100. and abs(Vmer) > 0.: + velocityX=numpy.append(velocityX, Vzon) + velocityY=numpy.append(velocityY, -Vmer) + else: - print 'Vmer',Vmer + velocityX=numpy.append(velocityX, numpy.NaN) velocityY=numpy.append(velocityY, numpy.NaN) - + if dbSNR[Height] > SNRlimit: - velocityV=numpy.append(velocityV, FirstMoment[Height]) + velocityV=numpy.append(velocityV, -Vver) # reason for this minus sign -> convention? (taken from Ericks version) else: velocityV=numpy.append(velocityV, numpy.NaN) - #FirstMoment[Height]= numpy.NaN -# if SNRdBMean[Height] <12: -# FirstMoment[Height] = numpy.NaN -# velocityX[Height] = numpy.NaN -# velocityY[Height] = numpy.NaN - - - data_output[0]=numpy.array(velocityX) - data_output[1]=numpy.array(velocityY) - data_output[2]=-velocityV#FirstMoment - - print ' ' - #print 'FirstMoment' - #print FirstMoment - print 'velocityX',data_output[0] - print ' ' - print 'velocityY',data_output[1] - #print numpy.array(velocityY) - print ' ' - #print 'SNR' - #print 10*numpy.log10(dataOut.data_SNR) - #print numpy.shape(10*numpy.log10(dataOut.data_SNR)) - print ' ' - - - dataOut.data_output=data_output - return - - + + + '''Change the numpy.array (velocityX) sign when trying to process BLTR data (Erick)''' + data_output[0] = numpy.array(velocityX) + data_output[1] = numpy.array(velocityY) + data_output[2] = velocityV + + + dataOut.data_output = data_output + + return dataOut + + def moving_average(self,x, N=2): + """ convolution for smoothenig data. note that last N-1 values are convolution with zeroes """ return numpy.convolve(x, numpy.ones((N,))/N)[(N-1):] - - def gaus(self,xSamples,a,x0,sigma): - return a*numpy.exp(-(xSamples-x0)**2/(2*sigma**2)) - - def Find(self,x,value): - for index in range(len(x)): - if x[index]==value: - return index - - def WindEstimation(self, spc, cspc, pairsList, ChanDist, Height, noise, VelRange, dbSNR, SNRlimit): - - ySamples=numpy.ones([spc.shape[0],spc.shape[1]]) - phase=numpy.ones([spc.shape[0],spc.shape[1]]) - CSPCSamples=numpy.ones([spc.shape[0],spc.shape[1]],dtype=numpy.complex_) - coherence=numpy.ones([spc.shape[0],spc.shape[1]]) - PhaseSlope=numpy.ones(spc.shape[0]) - PhaseInter=numpy.ones(spc.shape[0]) - xFrec=VelRange - + + def gaus(self,xSamples,Amp,Mu,Sigma): + return ( Amp / ((2*numpy.pi)**0.5 * Sigma) ) * numpy.exp( -( xSamples - Mu )**2 / ( 2 * (Sigma**2) )) + + def Moments(self, ySamples, xSamples): + '''*** + Variables corresponding to moments of distribution. + Also used as initial coefficients for curve_fit. + Vr was corrected. Only a velocity when x is velocity, of course. + ***''' + Pot = numpy.nansum( ySamples ) # Potencia, momento 0 + yNorm = ySamples / Pot + x_range = (numpy.max(xSamples)-numpy.min(xSamples)) + Vr = numpy.nansum( yNorm * xSamples )*x_range/len(xSamples) # Velocidad radial, mu, corrimiento doppler, primer momento + Sigma2 = abs(numpy.nansum( yNorm * ( xSamples - Vr )**2 )) # Segundo Momento + Desv = Sigma2**0.5 # Desv. Estandar, Ancho espectral + + return numpy.array([Pot, Vr, Desv]) + + def StopWindEstimation(self, error_code): + ''' + the wind calculation and returns zeros + ''' + Vzon = 0 + Vmer = 0 + Vver = numpy.nan + return Vzon, Vmer, Vver, error_code + + def AntiAliasing(self, interval, maxstep): + """ + function to prevent errors from aliased values when computing phaseslope + """ + antialiased = numpy.zeros(len(interval))*0.0 + copyinterval = interval.copy() + + antialiased[0] = copyinterval[0] + + for i in range(1,len(antialiased)): + + step = interval[i] - interval[i-1] + + if step > maxstep: + copyinterval -= 2*numpy.pi + antialiased[i] = copyinterval[i] + + elif step < maxstep*(-1): + copyinterval += 2*numpy.pi + antialiased[i] = copyinterval[i] + + else: + antialiased[i] = copyinterval[i].copy() + + return antialiased + + def WindEstimation(self, spc, cspc, pairsList, ChanDist, Height, noise, AbbsisaRange, dbSNR, SNRlimit): + """ + Function that Calculates Zonal, Meridional and Vertical wind velocities. + Initial Version by E. Bocanegra updated by J. Zibell until Nov. 2019. + + Input: + spc, cspc : self spectra and cross spectra data. In Briggs notation something like S_i*(S_i)_conj, (S_j)_conj respectively. + pairsList : Pairlist of channels + ChanDist : array of xi_ij and eta_ij + Height : height at which data is processed + noise : noise in [channels] format for specific height + Abbsisarange : range of the frequencies or velocities + dbSNR, SNRlimit : signal to noise ratio in db, lower limit + + Output: + Vzon, Vmer, Vver : wind velocities + error_code : int that states where code is terminated + + 0 : no error detected + 1 : Gaussian of mean spc exceeds widthlimit + 2 : no Gaussian of mean spc found + 3 : SNR to low or velocity to high -> prec. e.g. + 4 : at least one Gaussian of cspc exceeds widthlimit + 5 : zero out of three cspc Gaussian fits converged + 6 : phase slope fit could not be found + 7 : arrays used to fit phase have different length + 8 : frequency range is either too short (len <= 5) or very long (> 30% of cspc) + + """ + + error_code = 0 + + + SPC_Samples = numpy.ones([spc.shape[0],spc.shape[1]]) # for normalized spc values for one height + phase = numpy.ones([spc.shape[0],spc.shape[1]]) # phase between channels + CSPC_Samples = numpy.ones([spc.shape[0],spc.shape[1]],dtype=numpy.complex_) # for normalized cspc values + PhaseSlope = numpy.zeros(spc.shape[0]) # slope of the phases, channelwise + PhaseInter = numpy.ones(spc.shape[0]) # intercept to the slope of the phases, channelwise + xFrec = AbbsisaRange[0][0:spc.shape[1]] # frequency range + xVel = AbbsisaRange[2][0:spc.shape[1]] # velocity range + SPCav = numpy.average(spc, axis=0)-numpy.average(noise) # spc[0]-noise[0] + + SPCmoments_vel = self.Moments(SPCav, xVel ) # SPCmoments_vel[1] corresponds to vertical velocity and is used to determine if signal corresponds to wind (if .. <3) + CSPCmoments = [] + + '''Getting Eij and Nij''' - - E01=ChanDist[0][0] - N01=ChanDist[0][1] - - E02=ChanDist[1][0] - N02=ChanDist[1][1] - - E12=ChanDist[2][0] - N12=ChanDist[2][1] - - z = spc.copy() - z = numpy.where(numpy.isfinite(z), z, numpy.NAN) - - for i in range(spc.shape[0]): - - '''****** Line of Data SPC ******''' - zline=z[i,:,Height] - - '''****** SPC is normalized ******''' - FactNorm= (zline.copy()-noise[i]) / numpy.sum(zline.copy()) - FactNorm= FactNorm/numpy.sum(FactNorm) - - SmoothSPC=self.moving_average(FactNorm,N=3) - - xSamples = ar(range(len(SmoothSPC))) - ySamples[i] = SmoothSPC - - #dbSNR=10*numpy.log10(dataSNR) - print ' ' - print ' ' - print ' ' - - #print 'dataSNR', dbSNR.shape, dbSNR[0,40:120] - print 'SmoothSPC', SmoothSPC.shape, SmoothSPC[0:20] - print 'noise',noise - print 'zline',zline.shape, zline[0:20] - print 'FactNorm',FactNorm.shape, FactNorm[0:20] - print 'FactNorm suma', numpy.sum(FactNorm) - + + Xi01, Xi02, Xi12 = ChanDist[:,0] + Eta01, Eta02, Eta12 = ChanDist[:,1] + + # update nov 19 + widthlimit = 7 # maximum width in Hz of the gaussian, empirically determined. Anything above 10 is unrealistic, often values between 1 and 5 correspond to proper fits. + + '''************************* SPC is normalized ********************************''' + + spc_norm = spc.copy() # need copy() because untouched spc is needed for normalization of cspc below + spc_norm = numpy.where(numpy.isfinite(spc_norm), spc_norm, numpy.NAN) + for i in range(spc.shape[0]): - - '''****** Line of Data CSPC ******''' - cspcLine=cspc[i,:,Height].copy() - - '''****** CSPC is normalized ******''' - chan_index0 = pairsList[i][0] - chan_index1 = pairsList[i][1] - CSPCFactor= abs(numpy.sum(ySamples[chan_index0]) * numpy.sum(ySamples[chan_index1])) # - - CSPCNorm = (cspcLine.copy() -noise[i]) / numpy.sqrt(CSPCFactor) - - CSPCSamples[i] = CSPCNorm - coherence[i] = numpy.abs(CSPCSamples[i]) / numpy.sqrt(CSPCFactor) - - coherence[i]= self.moving_average(coherence[i],N=2) - - phase[i] = self.moving_average( numpy.arctan2(CSPCSamples[i].imag, CSPCSamples[i].real),N=1)#*180/numpy.pi - - print 'cspcLine', cspcLine.shape, cspcLine[0:20] - print 'CSPCFactor', CSPCFactor#, CSPCFactor[0:20] - print numpy.sum(ySamples[chan_index0]), numpy.sum(ySamples[chan_index1]), -noise[i] - print 'CSPCNorm', CSPCNorm.shape, CSPCNorm[0:20] - print 'CSPCNorm suma', numpy.sum(CSPCNorm) - print 'CSPCSamples', CSPCSamples.shape, CSPCSamples[0,0:20] - - '''****** Getting fij width ******''' - - yMean=[] - yMean2=[] - - for j in range(len(ySamples[1])): - yMean=numpy.append(yMean,numpy.mean([ySamples[0,j],ySamples[1,j],ySamples[2,j]])) - - '''******* Getting fitting Gaussian ******''' - meanGauss=sum(xSamples*yMean) / len(xSamples) - sigma=sum(yMean*(xSamples-meanGauss)**2) / len(xSamples) - - print '****************************' - print 'len(xSamples): ',len(xSamples) - print 'yMean: ', yMean.shape, yMean[0:20] - print 'ySamples', ySamples.shape, ySamples[0,0:20] - print 'xSamples: ',xSamples.shape, xSamples[0:20] - - print 'meanGauss',meanGauss - print 'sigma',sigma - - #if (abs(meanGauss/sigma**2) > 0.0001) : #0.000000001): - if dbSNR > SNRlimit : - try: - popt,pcov = curve_fit(self.gaus,xSamples,yMean,p0=[1,meanGauss,sigma]) - - if numpy.amax(popt)>numpy.amax(yMean)*0.3: - FitGauss=self.gaus(xSamples,*popt) - - else: - FitGauss=numpy.ones(len(xSamples))*numpy.mean(yMean) - print 'Verificador: Dentro', Height + + spc_sub = spc_norm[i,:] - noise[i] # spc not smoothed here or in previous version. + + Factor_Norm = 2*numpy.max(xFrec) / numpy.count_nonzero(~numpy.isnan(spc_sub)) # usually = Freq range / nfft + normalized_spc = spc_sub / (numpy.nansum(numpy.abs(spc_sub)) * Factor_Norm) + + xSamples = xFrec # the frequency range is taken + SPC_Samples[i] = normalized_spc # Normalized SPC values are taken + + '''********************** FITTING MEAN SPC GAUSSIAN **********************''' + + """ the gaussian of the mean: first subtract noise, then normalize. this is legal because + you only fit the curve and don't need the absolute value of height for calculation, + only for estimation of width. for normalization of cross spectra, you need initial, + unnormalized self-spectra With noise. + + Technically, you don't even need to normalize the self-spectra, as you only need the + width of the peak. However, it was left this way. Note that the normalization has a flaw: + due to subtraction of the noise, some values are below zero. Raw "spc" values should be + >= 0, as it is the modulus squared of the signals (complex * it's conjugate) + """ + + SPCMean = numpy.average(SPC_Samples, axis=0) + + popt = [1e-10,0,1e-10] + SPCMoments = self.Moments(SPCMean, xSamples) + + if dbSNR > SNRlimit and numpy.abs(SPCmoments_vel[1]) < 3: + try: + popt,pcov = curve_fit(self.gaus,xSamples,SPCMean,p0=SPCMoments)#, bounds=(-numpy.inf, [numpy.inf, numpy.inf, 10])). Setting bounds does not make the code faster but only keeps the fit from finding the minimum. + if popt[2] > widthlimit: # CONDITION + return self.StopWindEstimation(error_code = 1) + + FitGauss = self.gaus(xSamples,*popt) + except :#RuntimeError: - FitGauss=numpy.ones(len(xSamples))*numpy.mean(yMean) - - - else: - FitGauss=numpy.ones(len(xSamples))*numpy.mean(yMean) - - Maximun=numpy.amax(yMean) - eMinus1=Maximun*numpy.exp(-1)#*0.8 - - HWpos=self.Find(FitGauss,min(FitGauss, key=lambda value:abs(value-eMinus1))) - HalfWidth= xFrec[HWpos] - GCpos=self.Find(FitGauss, numpy.amax(FitGauss)) - Vpos=self.Find(FactNorm, numpy.amax(FactNorm)) - - #Vpos=FirstMoment[] - - '''****** Getting Fij ******''' - - GaussCenter=xFrec[GCpos] - if (GaussCenter<0 and HalfWidth>0) or (GaussCenter>0 and HalfWidth<0): - Fij=abs(GaussCenter)+abs(HalfWidth)+0.0000001 - else: - Fij=abs(GaussCenter-HalfWidth)+0.0000001 - - '''****** Getting Frecuency range of significant data ******''' - - Rangpos=self.Find(FitGauss,min(FitGauss, key=lambda value:abs(value-Maximun*0.10))) - - if Rangpos sum_wind is sum of those frequencies. Next, the area + around it gets cut off and values replaced by mean determined by the boundary + data -> sum_noise (spc is not normalized here, thats why the noise is important) + + The sums are then added and multiplied by range/datapoints, because you need + an integral and not a sum for normalization. + + A norm is found according to Briggs 92. + ''' + + radarWavelength = 0.6741 # meters + count_limit_freq = numpy.abs(popt[1]) + widthlimit # Hz, m/s can be also used if velocity is desired abscissa. + # count_limit_freq = numpy.max(xFrec) + + channel_integrals = numpy.zeros(3) + + for i in range(spc.shape[0]): + ''' + find the point in array corresponding to count_limit frequency. + sum over all frequencies in the range around zero Hz @ math.ceil(N_freq/2) + ''' + N_freq = numpy.count_nonzero(~numpy.isnan(spc[i,:])) + count_limit_int = int(math.ceil( count_limit_freq / numpy.max(xFrec) * (N_freq / 2) )) # gives integer point + sum_wind = numpy.nansum( spc[i, (math.ceil(N_freq/2) - count_limit_int) : (math.ceil(N_freq / 2) + count_limit_int)] ) #N_freq/2 is where frequency (velocity) is zero, i.e. middle of spectrum. + sum_noise = (numpy.mean(spc[i, :4]) + numpy.mean(spc[i, -6:-2]))/2.0 * (N_freq - 2*count_limit_int) + channel_integrals[i] = (sum_noise + sum_wind) * (2*numpy.max(xFrec) / N_freq) + + + cross_integrals_peak = numpy.zeros(3) + # cross_integrals_totalrange = numpy.zeros(3) + for i in range(spc.shape[0]): - - if len(FrecRange)>5 and len(FrecRange) widthlimit: # CONDITION + return self.StopWindEstimation(error_code = 4) + + popt02,pcov = curve_fit(self.gaus,xSamples[mask02],numpy.abs(CSPCMask02),p0=CSPCmoments[1]) + if popt02[2] > widthlimit: # CONDITION + return self.StopWindEstimation(error_code = 4) + + popt12,pcov = curve_fit(self.gaus,xSamples[mask12],numpy.abs(CSPCMask12),p0=CSPCmoments[2]) + if popt12[2] > widthlimit: # CONDITION + return self.StopWindEstimation(error_code = 4) + + FitGauss01 = self.gaus(xSamples, *popt01) + FitGauss02 = self.gaus(xSamples, *popt02) + FitGauss12 = self.gaus(xSamples, *popt12) + + except: + return self.StopWindEstimation(error_code = 5) + + + '''************* Getting Fij ***************''' + + + #Punto en Eje X de la Gaussiana donde se encuentra el centro -- x-axis point of the gaussian where the center is located + # -> PointGauCenter + GaussCenter = popt[1] + ClosestCenter = xSamples[numpy.abs(xSamples-GaussCenter).argmin()] + PointGauCenter = numpy.where(xSamples==ClosestCenter)[0][0] + + #Punto e^-1 hubicado en la Gaussiana -- point where e^-1 is located in the gaussian + PeMinus1 = numpy.max(FitGauss) * numpy.exp(-1) + FijClosest = FitGauss[numpy.abs(FitGauss-PeMinus1).argmin()] # El punto mas cercano a "Peminus1" dentro de "FitGauss" + PointFij = numpy.where(FitGauss==FijClosest)[0][0] + + Fij = numpy.abs(xSamples[PointFij] - xSamples[PointGauCenter]) + + '''********** Taking frequency ranges from mean SPCs **********''' + + #GaussCenter = popt[1] #Primer momento 01 + GauWidth = popt[2] * 3/2 #Ancho de banda de Gau01 -- Bandwidth of Gau01 TODO why *3/2? + Range = numpy.empty(2) + Range[0] = GaussCenter - GauWidth + Range[1] = GaussCenter + GauWidth + #Punto en Eje X de la Gaussiana donde se encuentra ancho de banda (min:max) -- Point in x-axis where the bandwidth is located (min:max) + ClosRangeMin = xSamples[numpy.abs(xSamples-Range[0]).argmin()] + ClosRangeMax = xSamples[numpy.abs(xSamples-Range[1]).argmin()] + + PointRangeMin = numpy.where(xSamples==ClosRangeMin)[0][0] + PointRangeMax = numpy.where(xSamples==ClosRangeMax)[0][0] + + Range = numpy.array([ PointRangeMin, PointRangeMax ]) + + FrecRange = xFrec[ Range[0] : Range[1] ] + + + '''************************** Getting Phase Slope ***************************''' + + for i in range(1,3): # Changed to only compute two + + if len(FrecRange) > 5 and len(FrecRange) < spc.shape[1] * 0.3: + # PhaseRange=self.moving_average(phase[i,Range[0]:Range[1]],N=1) #used before to smooth phase with N=3 + PhaseRange = phase[i,Range[0]:Range[1]].copy() + + mask = ~numpy.isnan(FrecRange) & ~numpy.isnan(PhaseRange) + + if len(FrecRange) == len(PhaseRange): - slope, intercept, r_value, p_value, std_err = stats.linregress(FrecRange,PhaseRange) - PhaseSlope[i]=slope - PhaseInter[i]=intercept + + try: + slope, intercept, _, _, _ = stats.linregress(FrecRange[mask], self.AntiAliasing(PhaseRange[mask], 4.5)) + PhaseSlope[i] = slope + PhaseInter[i] = intercept + + except: + return self.StopWindEstimation(error_code = 6) + else: - PhaseSlope[i]=0 - PhaseInter[i]=0 + return self.StopWindEstimation(error_code = 7) + else: - PhaseSlope[i]=0 - PhaseInter[i]=0 - - '''Getting constant C''' - cC=(Fij*numpy.pi)**2 - - '''****** Getting constants F and G ******''' - MijEijNij=numpy.array([[E02,N02], [E12,N12]]) - MijResult0=(-PhaseSlope[1]*cC) / (2*numpy.pi) - MijResult1=(-PhaseSlope[2]*cC) / (2*numpy.pi) - MijResults=numpy.array([MijResult0,MijResult1]) - (cF,cG) = numpy.linalg.solve(MijEijNij, MijResults) - - '''****** Getting constants A, B and H ******''' - W01=numpy.amax(coherence[0]) - W02=numpy.amax(coherence[1]) - W12=numpy.amax(coherence[2]) - - WijResult0=((cF*E01+cG*N01)**2)/cC - numpy.log(W01 / numpy.sqrt(numpy.pi/cC)) - WijResult1=((cF*E02+cG*N02)**2)/cC - numpy.log(W02 / numpy.sqrt(numpy.pi/cC)) - WijResult2=((cF*E12+cG*N12)**2)/cC - numpy.log(W12 / numpy.sqrt(numpy.pi/cC)) - - WijResults=numpy.array([WijResult0, WijResult1, WijResult2]) - - WijEijNij=numpy.array([ [E01**2, N01**2, 2*E01*N01] , [E02**2, N02**2, 2*E02*N02] , [E12**2, N12**2, 2*E12*N12] ]) - (cA,cB,cH) = numpy.linalg.solve(WijEijNij, WijResults) - - VxVy=numpy.array([[cA,cH],[cH,cB]]) - - VxVyResults=numpy.array([-cF,-cG]) - (Vx,Vy) = numpy.linalg.solve(VxVy, VxVyResults) - - Vzon = Vy - Vmer = Vx - Vmag=numpy.sqrt(Vzon**2+Vmer**2) - Vang=numpy.arctan2(Vmer,Vzon) - Vver=xFrec[Vpos] - print 'vzon y vmer', Vzon, Vmer - return Vzon, Vmer, Vver, GaussCenter - + return self.StopWindEstimation(error_code = 8) + + + + '''*** Constants A-H correspond to the convention as in Briggs and Vincent 1992 ***''' + + '''Getting constant C''' + cC=(Fij*numpy.pi)**2 + + '''****** Getting constants F and G ******''' + MijEijNij = numpy.array([[Xi02,Eta02], [Xi12,Eta12]]) + MijResult0 = (-PhaseSlope[1] * cC) / (2*numpy.pi) + MijResult1 = (-PhaseSlope[2] * cC) / (2*numpy.pi) + MijResults = numpy.array([MijResult0,MijResult1]) + (cF,cG) = numpy.linalg.solve(MijEijNij, MijResults) + + '''****** Getting constants A, B and H ******''' + W01 = numpy.nanmax( FitGauss01 ) + W02 = numpy.nanmax( FitGauss02 ) + W12 = numpy.nanmax( FitGauss12 ) + + WijResult0 = ((cF * Xi01 + cG * Eta01)**2)/cC - numpy.log(W01 / numpy.sqrt(numpy.pi / cC)) + WijResult1 = ((cF * Xi02 + cG * Eta02)**2)/cC - numpy.log(W02 / numpy.sqrt(numpy.pi / cC)) + WijResult2 = ((cF * Xi12 + cG * Eta12)**2)/cC - numpy.log(W12 / numpy.sqrt(numpy.pi / cC)) + + WijResults = numpy.array([WijResult0, WijResult1, WijResult2]) + + WijEijNij = numpy.array([ [Xi01**2, Eta01**2, 2*Xi01*Eta01] , [Xi02**2, Eta02**2, 2*Xi02*Eta02] , [Xi12**2, Eta12**2, 2*Xi12*Eta12] ]) + (cA,cB,cH) = numpy.linalg.solve(WijEijNij, WijResults) + + VxVy = numpy.array([[cA,cH],[cH,cB]]) + VxVyResults = numpy.array([-cF,-cG]) + (Vx,Vy) = numpy.linalg.solve(VxVy, VxVyResults) + + Vzon = Vy + Vmer = Vx + + # Vmag=numpy.sqrt(Vzon**2+Vmer**2) # unused + # Vang=numpy.arctan2(Vmer,Vzon) # unused + + + ''' using frequency as abscissa. Due to three channels, the offzenith angle is zero + and Vrad equal to Vver. formula taken from Briggs 92, figure 4. + ''' + if numpy.abs( popt[1] ) < 3.5 and len(FrecRange) > 4: + Vver = 0.5 * radarWavelength * popt[1] * 100 # *100 to get cm (/s) + else: + Vver = numpy.NaN + + error_code = 0 + + return Vzon, Vmer, Vver, error_code + + class SpectralMoments(Operation): - + ''' Function SpectralMoments() - + Calculates moments (power, mean, standard deviation) and SNR of the signal - + Type of dataIn: Spectra - + Configuration Parameters: - + dirCosx : Cosine director in X axis dirCosy : Cosine director in Y axis - + elevation : azimuth : - + Input: - channelList : simple channel list to select e.g. [2,3,7] + channelList : simple channel list to select e.g. [2,3,7] self.dataOut.data_pre : Spectral data self.dataOut.abscissaList : List of frequencies self.dataOut.noise : Noise level per channel - + Affected: - self.dataOut.data_param : Parameters per channel - self.dataOut.data_SNR : SNR per channel - + self.dataOut.moments : Parameters per channel + self.dataOut.data_SNR : SNR per channel + ''' - + def run(self, dataOut): - - #dataOut.data_pre = dataOut.data_pre[0] + data = dataOut.data_pre[0] absc = dataOut.abscissaList[:-1] noise = dataOut.noise nChannel = data.shape[0] data_param = numpy.zeros((nChannel, 4, data.shape[2])) - + for ind in range(nChannel): data_param[ind,:,:] = self.__calculateMoments( data[ind,:,:] , absc , noise[ind] ) - - dataOut.data_param = data_param[:,1:,:] + + dataOut.moments = data_param[:,1:,:] dataOut.data_SNR = data_param[:,0] - dataOut.data_DOP = data_param[:,1] - dataOut.data_MEAN = data_param[:,2] - dataOut.data_STD = data_param[:,3] - return - - def __calculateMoments(self, oldspec, oldfreq, n0, + dataOut.data_POW = data_param[:,1] + dataOut.data_DOP = data_param[:,2] + dataOut.data_WIDTH = data_param[:,3] + + return dataOut + + def __calculateMoments(self, oldspec, oldfreq, n0, nicoh = None, graph = None, smooth = None, type1 = None, fwindow = None, snrth = None, dc = None, aliasing = None, oldfd = None, wwauto = None): - + if (nicoh is None): nicoh = 1 - if (graph is None): graph = 0 + if (graph is None): graph = 0 if (smooth is None): smooth = 0 elif (self.smooth < 3): smooth = 0 @@ -1420,96 +1375,102 @@ class SpectralMoments(Operation): if (aliasing is None): aliasing = 0 if (oldfd is None): oldfd = 0 if (wwauto is None): wwauto = 0 - + if (n0 < 1.e-20): n0 = 1.e-20 - + freq = oldfreq vec_power = numpy.zeros(oldspec.shape[1]) vec_fd = numpy.zeros(oldspec.shape[1]) vec_w = numpy.zeros(oldspec.shape[1]) vec_snr = numpy.zeros(oldspec.shape[1]) + # oldspec = numpy.ma.masked_invalid(oldspec) + for ind in range(oldspec.shape[1]): - + spec = oldspec[:,ind] aux = spec*fwindow max_spec = aux.max() - m = list(aux).index(max_spec) - - #Smooth - if (smooth == 0): spec2 = spec - else: spec2 = scipy.ndimage.filters.uniform_filter1d(spec,size=smooth) - + m = aux.tolist().index(max_spec) + + #Smooth + if (smooth == 0): + spec2 = spec + else: + spec2 = scipy.ndimage.filters.uniform_filter1d(spec,size=smooth) + # Calculo de Momentos - bb = spec2[range(m,spec2.size)] + bb = spec2[numpy.arange(m,spec2.size)] bb = (bb m): ss1 = m - - valid = numpy.asarray(range(int(m + bb0 - ss1 + 1))) + ss1 - power = ((spec2[valid] - n0)*fwindow[valid]).sum() - fd = ((spec2[valid]- n0)*freq[valid]*fwindow[valid]).sum()/power - w = math.sqrt(((spec2[valid] - n0)*fwindow[valid]*(freq[valid]- fd)**2).sum()/power) - snr = (spec2.mean()-n0)/n0 - - if (snr < 1.e-20) : + + if (ss.size == 0): + ss1 = 1 + else: + ss1 = max(ss) + 1 + + if (ss1 > m): + ss1 = m + + valid = numpy.arange(int(m + bb0 - ss1 + 1)) + ss1 + + power = ((spec2[valid] - n0) * fwindow[valid]).sum() + fd = ((spec2[valid]- n0)*freq[valid] * fwindow[valid]).sum() / power + w = numpy.sqrt(((spec2[valid] - n0)*fwindow[valid]*(freq[valid]- fd)**2).sum() / power) + snr = (spec2.mean()-n0)/n0 + if (snr < 1.e-20) : snr = 1.e-20 - + vec_power[ind] = power vec_fd[ind] = fd vec_w[ind] = w vec_snr[ind] = snr - - moments = numpy.vstack((vec_snr, vec_power, vec_fd, vec_w)) - return moments - + + return numpy.vstack((vec_snr, vec_power, vec_fd, vec_w)) + #------------------ Get SA Parameters -------------------------- - + def GetSAParameters(self): #SA en frecuencia pairslist = self.dataOut.groupList num_pairs = len(pairslist) - + vel = self.dataOut.abscissaList spectra = self.dataOut.data_pre cspectra = self.dataIn.data_cspc - delta_v = vel[1] - vel[0] - + delta_v = vel[1] - vel[0] + #Calculating the power spectrum spc_pow = numpy.sum(spectra, 3)*delta_v #Normalizing Spectra norm_spectra = spectra/spc_pow #Calculating the norm_spectra at peak - max_spectra = numpy.max(norm_spectra, 3) - + max_spectra = numpy.max(norm_spectra, 3) + #Normalizing Cross Spectra norm_cspectra = numpy.zeros(cspectra.shape) - + for i in range(num_chan): norm_cspectra[i,:,:] = cspectra[i,:,:]/numpy.sqrt(spc_pow[pairslist[i][0],:]*spc_pow[pairslist[i][1],:]) - + max_cspectra = numpy.max(norm_cspectra,2) max_cspectra_index = numpy.argmax(norm_cspectra, 2) - + for i in range(num_pairs): cspc_par[i,:,:] = __calculateMoments(norm_cspectra) #------------------- Get Lags ---------------------------------- - + class SALags(Operation): ''' Function GetMoments() @@ -1522,19 +1483,19 @@ class SALags(Operation): self.dataOut.data_SNR self.dataOut.groupList self.dataOut.nChannels - + Affected: self.dataOut.data_param - + ''' - def run(self, dataOut): + def run(self, dataOut): data_acf = dataOut.data_pre[0] data_ccf = dataOut.data_pre[1] normFactor_acf = dataOut.normFactor[0] normFactor_ccf = dataOut.normFactor[1] pairs_acf = dataOut.groupList[0] pairs_ccf = dataOut.groupList[1] - + nHeights = dataOut.nHeights absc = dataOut.abscissaList noise = dataOut.noise @@ -1545,97 +1506,97 @@ class SALags(Operation): for l in range(len(pairs_acf)): data_acf[l,:,:] = data_acf[l,:,:]/normFactor_acf[l,:] - + for l in range(len(pairs_ccf)): data_ccf[l,:,:] = data_ccf[l,:,:]/normFactor_ccf[l,:] - + dataOut.data_param = numpy.zeros((len(pairs_ccf)*2 + 1, nHeights)) dataOut.data_param[:-1,:] = self.__calculateTaus(data_acf, data_ccf, absc) dataOut.data_param[-1,:] = self.__calculateLag1Phase(data_acf, absc) return - + # def __getPairsAutoCorr(self, pairsList, nChannels): -# +# # pairsAutoCorr = numpy.zeros(nChannels, dtype = 'int')*numpy.nan -# -# for l in range(len(pairsList)): +# +# for l in range(len(pairsList)): # firstChannel = pairsList[l][0] # secondChannel = pairsList[l][1] -# -# #Obteniendo pares de Autocorrelacion +# +# #Obteniendo pares de Autocorrelacion # if firstChannel == secondChannel: # pairsAutoCorr[firstChannel] = int(l) -# +# # pairsAutoCorr = pairsAutoCorr.astype(int) -# +# # pairsCrossCorr = range(len(pairsList)) # pairsCrossCorr = numpy.delete(pairsCrossCorr,pairsAutoCorr) -# +# # return pairsAutoCorr, pairsCrossCorr - + def __calculateTaus(self, data_acf, data_ccf, lagRange): - + lag0 = data_acf.shape[1]/2 #Funcion de Autocorrelacion mean_acf = stats.nanmean(data_acf, axis = 0) - + #Obtencion Indice de TauCross ind_ccf = data_ccf.argmax(axis = 1) #Obtencion Indice de TauAuto ind_acf = numpy.zeros(ind_ccf.shape,dtype = 'int') ccf_lag0 = data_ccf[:,lag0,:] - + for i in range(ccf_lag0.shape[0]): ind_acf[i,:] = numpy.abs(mean_acf - ccf_lag0[i,:]).argmin(axis = 0) - + #Obtencion de TauCross y TauAuto tau_ccf = lagRange[ind_ccf] tau_acf = lagRange[ind_acf] - + Nan1, Nan2 = numpy.where(tau_ccf == lagRange[0]) - + tau_ccf[Nan1,Nan2] = numpy.nan tau_acf[Nan1,Nan2] = numpy.nan tau = numpy.vstack((tau_ccf,tau_acf)) - + return tau - + def __calculateLag1Phase(self, data, lagTRange): data1 = stats.nanmean(data, axis = 0) lag1 = numpy.where(lagTRange == 0)[0][0] + 1 phase = numpy.angle(data1[lag1,:]) - + return phase - + class SpectralFitting(Operation): ''' Function GetMoments() - + Input: Output: Variables modified: ''' - - def run(self, dataOut, getSNR = True, path=None, file=None, groupList=None): - - + + def run(self, dataOut, getSNR = True, path=None, file=None, groupList=None): + + if path != None: sys.path.append(path) self.dataOut.library = importlib.import_module(file) - + #To be inserted as a parameter groupArray = numpy.array(groupList) -# groupArray = numpy.array([[0,1],[2,3]]) +# groupArray = numpy.array([[0,1],[2,3]]) self.dataOut.groupList = groupArray - + nGroups = groupArray.shape[0] nChannels = self.dataIn.nChannels nHeights=self.dataIn.heightList.size - + #Parameters Array self.dataOut.data_param = None - + #Set constants constants = self.dataOut.library.setConstants(self.dataIn) self.dataOut.constants = constants @@ -1644,24 +1605,24 @@ class SpectralFitting(Operation): ippSeconds = self.dataIn.ippSeconds K = self.dataIn.nIncohInt pairsArray = numpy.array(self.dataIn.pairsList) - + #List of possible combinations listComb = itertools.combinations(numpy.arange(groupArray.shape[1]),2) indCross = numpy.zeros(len(list(listComb)), dtype = 'int') - + if getSNR: listChannels = groupArray.reshape((groupArray.size)) listChannels.sort() noise = self.dataIn.getNoise() self.dataOut.data_SNR = self.__getSNR(self.dataIn.data_spc[listChannels,:,:], noise[listChannels]) - - for i in range(nGroups): + + for i in range(nGroups): coord = groupArray[i,:] - + #Input data array data = self.dataIn.data_spc[coord,:,:]/(M*N) data = data.reshape((data.shape[0]*data.shape[1],data.shape[2])) - + #Cross Spectra data array for Covariance Matrixes ind = 0 for pairs in listComb: @@ -1670,10 +1631,9 @@ class SpectralFitting(Operation): ind += 1 dataCross = self.dataIn.data_cspc[indCross,:,:]/(M*N) dataCross = dataCross**2/K - + for h in range(nHeights): -# print self.dataOut.heightList[h] - + #Input d = data[:,h] @@ -1682,7 +1642,7 @@ class SpectralFitting(Operation): ind = 0 for pairs in listComb: #Coordinates in Covariance Matrix - x = pairs[0] + x = pairs[0] y = pairs[1] #Channel Index S12 = dataCross[ind,:,h] @@ -1696,15 +1656,15 @@ class SpectralFitting(Operation): LT=L.T dp = numpy.dot(LT,d) - + #Initial values data_spc = self.dataIn.data_spc[coord,:,h] - + if (h>0)and(error1[3]<5): p0 = self.dataOut.data_param[i,:,h-1] else: p0 = numpy.array(self.dataOut.library.initialValuesFunction(data_spc, constants, i)) - + try: #Least Squares minp,covp,infodict,mesg,ier = optimize.leastsq(self.__residFunction,p0,args=(dp,LT,constants),full_output=True) @@ -1717,30 +1677,30 @@ class SpectralFitting(Operation): minp = p0*numpy.nan error0 = numpy.nan error1 = p0*numpy.nan - + #Save if self.dataOut.data_param is None: self.dataOut.data_param = numpy.zeros((nGroups, p0.size, nHeights))*numpy.nan self.dataOut.data_error = numpy.zeros((nGroups, p0.size + 1, nHeights))*numpy.nan - + self.dataOut.data_error[i,:,h] = numpy.hstack((error0,error1)) self.dataOut.data_param[i,:,h] = minp return - + def __residFunction(self, p, dp, LT, constants): fm = self.dataOut.library.modelFunction(p, constants) fmp=numpy.dot(LT,fm) - + return dp-fmp def __getSNR(self, z, noise): - + avg = numpy.average(z, axis=1) SNR = (avg.T-noise)/noise SNR = SNR.T return SNR - + def __chisq(p,chindex,hindex): #similar to Resid but calculates CHI**2 [LT,d,fm]=setupLTdfm(p,chindex,hindex) @@ -1748,53 +1708,53 @@ class SpectralFitting(Operation): fmp=numpy.dot(LT,fm) chisq=numpy.dot((dp-fmp).T,(dp-fmp)) return chisq - + class WindProfiler(Operation): - + __isConfig = False - + __initime = None __lastdatatime = None __integrationtime = None - + __buffer = None - + __dataReady = False - + __firstdata = None - + n = None - - def __init__(self, **kwargs): - Operation.__init__(self, **kwargs) - + + def __init__(self): + Operation.__init__(self) + def __calculateCosDir(self, elev, azim): zen = (90 - elev)*numpy.pi/180 azim = azim*numpy.pi/180 - cosDirX = numpy.sqrt((1-numpy.cos(zen)**2)/((1+numpy.tan(azim)**2))) + cosDirX = numpy.sqrt((1-numpy.cos(zen)**2)/((1+numpy.tan(azim)**2))) cosDirY = numpy.sqrt(1-numpy.cos(zen)**2-cosDirX**2) - + signX = numpy.sign(numpy.cos(azim)) signY = numpy.sign(numpy.sin(azim)) - + cosDirX = numpy.copysign(cosDirX, signX) cosDirY = numpy.copysign(cosDirY, signY) return cosDirX, cosDirY - + def __calculateAngles(self, theta_x, theta_y, azimuth): - + dir_cosw = numpy.sqrt(1-theta_x**2-theta_y**2) zenith_arr = numpy.arccos(dir_cosw) azimuth_arr = numpy.arctan2(theta_x,theta_y) + azimuth*math.pi/180 - + dir_cosu = numpy.sin(azimuth_arr)*numpy.sin(zenith_arr) dir_cosv = numpy.cos(azimuth_arr)*numpy.sin(zenith_arr) - + return azimuth_arr, zenith_arr, dir_cosu, dir_cosv, dir_cosw def __calculateMatA(self, dir_cosu, dir_cosv, dir_cosw, horOnly): - -# + +# if horOnly: A = numpy.c_[dir_cosu,dir_cosv] else: @@ -1808,37 +1768,37 @@ class WindProfiler(Operation): listPhi = phi.tolist() maxid = listPhi.index(max(listPhi)) minid = listPhi.index(min(listPhi)) - - rango = range(len(phi)) + + rango = list(range(len(phi))) # rango = numpy.delete(rango,maxid) - + heiRang1 = heiRang*math.cos(phi[maxid]) heiRangAux = heiRang*math.cos(phi[minid]) indOut = (heiRang1 < heiRangAux[0]).nonzero() heiRang1 = numpy.delete(heiRang1,indOut) - + velRadial1 = numpy.zeros([len(phi),len(heiRang1)]) SNR1 = numpy.zeros([len(phi),len(heiRang1)]) - + for i in rango: x = heiRang*math.cos(phi[i]) y1 = velRadial[i,:] f1 = interpolate.interp1d(x,y1,kind = 'cubic') - + x1 = heiRang1 y11 = f1(x1) - + y2 = SNR[i,:] f2 = interpolate.interp1d(x,y2,kind = 'cubic') y21 = f2(x1) - + velRadial1[i,:] = y11 SNR1[i,:] = y21 - + return heiRang1, velRadial1, SNR1 def __calculateVelUVW(self, A, velRadial): - + #Operacion Matricial # velUVW = numpy.zeros((velRadial.shape[1],3)) # for ind in range(velRadial.shape[1]): @@ -1846,42 +1806,42 @@ class WindProfiler(Operation): # velUVW = velUVW.transpose() velUVW = numpy.zeros((A.shape[0],velRadial.shape[1])) velUVW[:,:] = numpy.dot(A,velRadial) - - + + return velUVW - + # def techniqueDBS(self, velRadial0, dirCosx, disrCosy, azimuth, correct, horizontalOnly, heiRang, SNR0): - + def techniqueDBS(self, kwargs): """ Function that implements Doppler Beam Swinging (DBS) technique. - + Input: Radial velocities, Direction cosines (x and y) of the Beam, Antenna azimuth, Direction correction (if necessary), Ranges and SNR - + Output: Winds estimation (Zonal, Meridional and Vertical) - + Parameters affected: Winds, height range, SNR """ velRadial0 = kwargs['velRadial'] heiRang = kwargs['heightList'] SNR0 = kwargs['SNR'] - - if kwargs.has_key('dirCosx') and kwargs.has_key('dirCosy'): + + if 'dirCosx' in kwargs and 'dirCosy' in kwargs: theta_x = numpy.array(kwargs['dirCosx']) theta_y = numpy.array(kwargs['dirCosy']) else: elev = numpy.array(kwargs['elevation']) azim = numpy.array(kwargs['azimuth']) theta_x, theta_y = self.__calculateCosDir(elev, azim) - azimuth = kwargs['correctAzimuth'] - if kwargs.has_key('horizontalOnly'): + azimuth = kwargs['correctAzimuth'] + if 'horizontalOnly' in kwargs: horizontalOnly = kwargs['horizontalOnly'] else: horizontalOnly = False - if kwargs.has_key('correctFactor'): + if 'correctFactor' in kwargs: correctFactor = kwargs['correctFactor'] else: correctFactor = 1 - if kwargs.has_key('channelList'): + if 'channelList' in kwargs: channelList = kwargs['channelList'] if len(channelList) == 2: horizontalOnly = True @@ -1889,22 +1849,22 @@ class WindProfiler(Operation): param = param[arrayChannel,:,:] theta_x = theta_x[arrayChannel] theta_y = theta_y[arrayChannel] - - azimuth_arr, zenith_arr, dir_cosu, dir_cosv, dir_cosw = self.__calculateAngles(theta_x, theta_y, azimuth) - heiRang1, velRadial1, SNR1 = self.__correctValues(heiRang, zenith_arr, correctFactor*velRadial0, SNR0) + + azimuth_arr, zenith_arr, dir_cosu, dir_cosv, dir_cosw = self.__calculateAngles(theta_x, theta_y, azimuth) + heiRang1, velRadial1, SNR1 = self.__correctValues(heiRang, zenith_arr, correctFactor*velRadial0, SNR0) A = self.__calculateMatA(dir_cosu, dir_cosv, dir_cosw, horizontalOnly) - + #Calculo de Componentes de la velocidad con DBS winds = self.__calculateVelUVW(A,velRadial1) - + return winds, heiRang1, SNR1 - + def __calculateDistance(self, posx, posy, pairs_ccf, azimuth = None): - + nPairs = len(pairs_ccf) posx = numpy.asarray(posx) posy = numpy.asarray(posy) - + #Rotacion Inversa para alinear con el azimuth if azimuth!= None: azimuth = azimuth*math.pi/180 @@ -1913,126 +1873,126 @@ class WindProfiler(Operation): else: posx1 = posx posy1 = posy - + #Calculo de Distancias distx = numpy.zeros(nPairs) disty = numpy.zeros(nPairs) dist = numpy.zeros(nPairs) ang = numpy.zeros(nPairs) - + for i in range(nPairs): distx[i] = posx1[pairs_ccf[i][1]] - posx1[pairs_ccf[i][0]] - disty[i] = posy1[pairs_ccf[i][1]] - posy1[pairs_ccf[i][0]] + disty[i] = posy1[pairs_ccf[i][1]] - posy1[pairs_ccf[i][0]] dist[i] = numpy.sqrt(distx[i]**2 + disty[i]**2) ang[i] = numpy.arctan2(disty[i],distx[i]) - + return distx, disty, dist, ang - #Calculo de Matrices + #Calculo de Matrices # nPairs = len(pairs) # ang1 = numpy.zeros((nPairs, 2, 1)) # dist1 = numpy.zeros((nPairs, 2, 1)) -# +# # for j in range(nPairs): # dist1[j,0,0] = dist[pairs[j][0]] # dist1[j,1,0] = dist[pairs[j][1]] # ang1[j,0,0] = ang[pairs[j][0]] # ang1[j,1,0] = ang[pairs[j][1]] -# +# # return distx,disty, dist1,ang1 - + def __calculateVelVer(self, phase, lagTRange, _lambda): Ts = lagTRange[1] - lagTRange[0] velW = -_lambda*phase/(4*math.pi*Ts) - + return velW - + def __calculateVelHorDir(self, dist, tau1, tau2, ang): nPairs = tau1.shape[0] nHeights = tau1.shape[1] - vel = numpy.zeros((nPairs,3,nHeights)) + vel = numpy.zeros((nPairs,3,nHeights)) dist1 = numpy.reshape(dist, (dist.size,1)) - + angCos = numpy.cos(ang) angSin = numpy.sin(ang) - - vel0 = dist1*tau1/(2*tau2**2) + + vel0 = dist1*tau1/(2*tau2**2) vel[:,0,:] = (vel0*angCos).sum(axis = 1) vel[:,1,:] = (vel0*angSin).sum(axis = 1) - + ind = numpy.where(numpy.isinf(vel)) vel[ind] = numpy.nan - + return vel - + # def __getPairsAutoCorr(self, pairsList, nChannels): -# +# # pairsAutoCorr = numpy.zeros(nChannels, dtype = 'int')*numpy.nan -# -# for l in range(len(pairsList)): +# +# for l in range(len(pairsList)): # firstChannel = pairsList[l][0] # secondChannel = pairsList[l][1] -# -# #Obteniendo pares de Autocorrelacion +# +# #Obteniendo pares de Autocorrelacion # if firstChannel == secondChannel: # pairsAutoCorr[firstChannel] = int(l) -# +# # pairsAutoCorr = pairsAutoCorr.astype(int) -# +# # pairsCrossCorr = range(len(pairsList)) # pairsCrossCorr = numpy.delete(pairsCrossCorr,pairsAutoCorr) -# +# # return pairsAutoCorr, pairsCrossCorr - + # def techniqueSA(self, pairsSelected, pairsList, nChannels, tau, azimuth, _lambda, position_x, position_y, lagTRange, correctFactor): def techniqueSA(self, kwargs): - - """ + + """ Function that implements Spaced Antenna (SA) technique. - + Input: Radial velocities, Direction cosines (x and y) of the Beam, Antenna azimuth, Direction correction (if necessary), Ranges and SNR - + Output: Winds estimation (Zonal, Meridional and Vertical) - + Parameters affected: Winds """ position_x = kwargs['positionX'] position_y = kwargs['positionY'] azimuth = kwargs['azimuth'] - - if kwargs.has_key('correctFactor'): + + if 'correctFactor' in kwargs: correctFactor = kwargs['correctFactor'] else: correctFactor = 1 - + groupList = kwargs['groupList'] pairs_ccf = groupList[1] tau = kwargs['tau'] _lambda = kwargs['_lambda'] - + #Cross Correlation pairs obtained # pairsAutoCorr, pairsCrossCorr = self.__getPairsAutoCorr(pairssList, nChannels) # pairsArray = numpy.array(pairsList)[pairsCrossCorr] # pairsSelArray = numpy.array(pairsSelected) # pairs = [] -# +# # #Wind estimation pairs obtained # for i in range(pairsSelArray.shape[0]/2): # ind1 = numpy.where(numpy.all(pairsArray == pairsSelArray[2*i], axis = 1))[0][0] # ind2 = numpy.where(numpy.all(pairsArray == pairsSelArray[2*i + 1], axis = 1))[0][0] # pairs.append((ind1,ind2)) - + indtau = tau.shape[0]/2 tau1 = tau[:indtau,:] tau2 = tau[indtau:-1,:] # tau1 = tau1[pairs,:] # tau2 = tau2[pairs,:] phase1 = tau[-1,:] - + #--------------------------------------------------------------------- - #Metodo Directo + #Metodo Directo distx, disty, dist, ang = self.__calculateDistance(position_x, position_y, pairs_ccf,azimuth) winds = self.__calculateVelHorDir(dist, tau1, tau2, ang) winds = stats.nanmean(winds, axis=0) @@ -2048,100 +2008,97 @@ class WindProfiler(Operation): winds[2,:] = self.__calculateVelVer(phase1, lagTRange, _lambda) winds = correctFactor*winds return winds - + def __checkTime(self, currentTime, paramInterval, outputInterval): - + dataTime = currentTime + paramInterval deltaTime = dataTime - self.__initime - + if deltaTime >= outputInterval or deltaTime < 0: self.__dataReady = True - return - + return + def techniqueMeteors(self, arrayMeteor, meteorThresh, heightMin, heightMax): ''' Function that implements winds estimation technique with detected meteors. - + Input: Detected meteors, Minimum meteor quantity to wind estimation - + Output: Winds estimation (Zonal and Meridional) - + Parameters affected: Winds - ''' -# print arrayMeteor.shape + ''' #Settings nInt = (heightMax - heightMin)/2 -# print nInt nInt = int(nInt) -# print nInt - winds = numpy.zeros((2,nInt))*numpy.nan - + winds = numpy.zeros((2,nInt))*numpy.nan + #Filter errors error = numpy.where(arrayMeteor[:,-1] == 0)[0] finalMeteor = arrayMeteor[error,:] - + #Meteor Histogram finalHeights = finalMeteor[:,2] hist = numpy.histogram(finalHeights, bins = nInt, range = (heightMin,heightMax)) nMeteorsPerI = hist[0] heightPerI = hist[1] - + #Sort of meteors indSort = finalHeights.argsort() finalMeteor2 = finalMeteor[indSort,:] - + # Calculating winds ind1 = 0 - ind2 = 0 - + ind2 = 0 + for i in range(nInt): nMet = nMeteorsPerI[i] ind1 = ind2 ind2 = ind1 + nMet - + meteorAux = finalMeteor2[ind1:ind2,:] - + if meteorAux.shape[0] >= meteorThresh: vel = meteorAux[:, 6] zen = meteorAux[:, 4]*numpy.pi/180 azim = meteorAux[:, 3]*numpy.pi/180 - + n = numpy.cos(zen) # m = (1 - n**2)/(1 - numpy.tan(azim)**2) # l = m*numpy.tan(azim) l = numpy.sin(zen)*numpy.sin(azim) m = numpy.sin(zen)*numpy.cos(azim) - + A = numpy.vstack((l, m)).transpose() A1 = numpy.dot(numpy.linalg.inv( numpy.dot(A.transpose(),A) ),A.transpose()) windsAux = numpy.dot(A1, vel) - + winds[0,i] = windsAux[0] winds[1,i] = windsAux[1] - + return winds, heightPerI[:-1] - + def techniqueNSM_SA(self, **kwargs): metArray = kwargs['metArray'] heightList = kwargs['heightList'] timeList = kwargs['timeList'] - + rx_location = kwargs['rx_location'] groupList = kwargs['groupList'] azimuth = kwargs['azimuth'] dfactor = kwargs['dfactor'] k = kwargs['k'] - + azimuth1, dist = self.__calculateAzimuth1(rx_location, groupList, azimuth) d = dist*dfactor #Phase calculation metArray1 = self.__getPhaseSlope(metArray, heightList, timeList) - + metArray1[:,-2] = metArray1[:,-2]*metArray1[:,2]*1000/(k*d[metArray1[:,1].astype(int)]) #angles into velocities - + velEst = numpy.zeros((heightList.size,2))*numpy.nan azimuth1 = azimuth1*numpy.pi/180 - + for i in range(heightList.size): h = heightList[i] indH = numpy.where((metArray1[:,2] == h)&(numpy.abs(metArray1[:,-2]) < 100))[0] @@ -2154,71 +2111,71 @@ class WindProfiler(Operation): A = numpy.asmatrix(A) A1 = numpy.linalg.pinv(A.transpose()*A)*A.transpose() velHor = numpy.dot(A1,velAux) - + velEst[i,:] = numpy.squeeze(velHor) return velEst - + def __getPhaseSlope(self, metArray, heightList, timeList): meteorList = [] #utctime sec1 height SNR velRad ph0 ph1 ph2 coh0 coh1 coh2 #Putting back together the meteor matrix utctime = metArray[:,0] uniqueTime = numpy.unique(utctime) - + phaseDerThresh = 0.5 ippSeconds = timeList[1] - timeList[0] sec = numpy.where(timeList>1)[0][0] nPairs = metArray.shape[1] - 6 nHeights = len(heightList) - + for t in uniqueTime: metArray1 = metArray[utctime==t,:] # phaseDerThresh = numpy.pi/4 #reducir Phase thresh tmet = metArray1[:,1].astype(int) hmet = metArray1[:,2].astype(int) - + metPhase = numpy.zeros((nPairs, heightList.size, timeList.size - 1)) metPhase[:,:] = numpy.nan metPhase[:,hmet,tmet] = metArray1[:,6:].T - + #Delete short trails metBool = ~numpy.isnan(metPhase[0,:,:]) heightVect = numpy.sum(metBool, axis = 1) metBool[heightVect phaseDerThresh)) metPhase[phDerAux] = numpy.nan - + #--------------------------METEOR DETECTION ----------------------------------------- indMet = numpy.where(numpy.any(metBool,axis=1))[0] - + for p in numpy.arange(nPairs): phase = metPhase[p,:,:] phDer = metDer[p,:,:] - + for h in indMet: height = heightList[h] phase1 = phase[h,:] #82 phDer1 = phDer[h,:] - + phase1[~numpy.isnan(phase1)] = numpy.unwrap(phase1[~numpy.isnan(phase1)]) #Unwrap - + indValid = numpy.where(~numpy.isnan(phase1))[0] initMet = indValid[0] endMet = 0 - + for i in range(len(indValid)-1): - + #Time difference inow = indValid[i] inext = indValid[i+1] idiff = inext - inow #Phase difference - phDiff = numpy.abs(phase1[inext] - phase1[inow]) - + phDiff = numpy.abs(phase1[inext] - phase1[inow]) + if idiff>sec or phDiff>numpy.pi/4 or inext==indValid[-1]: #End of Meteor sizeTrail = inow - initMet + 1 if sizeTrail>3*sec: #Too short meteors @@ -2234,43 +2191,43 @@ class WindProfiler(Operation): vel = slope#*height*1000/(k*d) estAux = numpy.array([utctime,p,height, vel, rsq]) meteorList.append(estAux) - initMet = inext + initMet = inext metArray2 = numpy.array(meteorList) - + return metArray2 - + def __calculateAzimuth1(self, rx_location, pairslist, azimuth0): - + azimuth1 = numpy.zeros(len(pairslist)) dist = numpy.zeros(len(pairslist)) - + for i in range(len(rx_location)): ch0 = pairslist[i][0] ch1 = pairslist[i][1] - + diffX = rx_location[ch0][0] - rx_location[ch1][0] diffY = rx_location[ch0][1] - rx_location[ch1][1] azimuth1[i] = numpy.arctan2(diffY,diffX)*180/numpy.pi dist[i] = numpy.sqrt(diffX**2 + diffY**2) - + azimuth1 -= azimuth0 return azimuth1, dist - + def techniqueNSM_DBS(self, **kwargs): metArray = kwargs['metArray'] heightList = kwargs['heightList'] - timeList = kwargs['timeList'] + timeList = kwargs['timeList'] azimuth = kwargs['azimuth'] theta_x = numpy.array(kwargs['theta_x']) theta_y = numpy.array(kwargs['theta_y']) - + utctime = metArray[:,0] cmet = metArray[:,1].astype(int) hmet = metArray[:,3].astype(int) SNRmet = metArray[:,4] vmet = metArray[:,5] spcmet = metArray[:,6] - + nChan = numpy.max(cmet) + 1 nHeights = len(heightList) @@ -2286,20 +2243,20 @@ class WindProfiler(Operation): thisH = (h1met>=hmin) & (h1met8) & (vmet<50) & (spcmet<10) indthisH = numpy.where(thisH) - + if numpy.size(indthisH) > 3: - + vel_aux = vmet[thisH] chan_aux = cmet[thisH] cosu_aux = dir_cosu[chan_aux] cosv_aux = dir_cosv[chan_aux] cosw_aux = dir_cosw[chan_aux] - - nch = numpy.size(numpy.unique(chan_aux)) + + nch = numpy.size(numpy.unique(chan_aux)) if nch > 1: A = self.__calculateMatA(cosu_aux, cosv_aux, cosw_aux, True) velEst[i,:] = numpy.dot(A,vel_aux) - + return velEst def run(self, dataOut, technique, nHours=1, hmin=70, hmax=110, **kwargs): @@ -2310,39 +2267,39 @@ class WindProfiler(Operation): # noise = dataOut.noise heightList = dataOut.heightList SNR = dataOut.data_SNR - + if technique == 'DBS': - - kwargs['velRadial'] = param[:,1,:] #Radial velocity + + kwargs['velRadial'] = param[:,1,:] #Radial velocity kwargs['heightList'] = heightList kwargs['SNR'] = SNR - + dataOut.data_output, dataOut.heightList, dataOut.data_SNR = self.techniqueDBS(kwargs) #DBS Function dataOut.utctimeInit = dataOut.utctime dataOut.outputInterval = dataOut.paramInterval - + elif technique == 'SA': - + #Parameters # position_x = kwargs['positionX'] # position_y = kwargs['positionY'] # azimuth = kwargs['azimuth'] -# +# # if kwargs.has_key('crosspairsList'): # pairs = kwargs['crosspairsList'] # else: -# pairs = None -# +# pairs = None +# # if kwargs.has_key('correctFactor'): # correctFactor = kwargs['correctFactor'] # else: # correctFactor = 1 - + # tau = dataOut.data_param # _lambda = dataOut.C/dataOut.frequency # pairsList = dataOut.groupList # nChannels = dataOut.nChannels - + kwargs['groupList'] = dataOut.groupList kwargs['tau'] = dataOut.data_param kwargs['_lambda'] = dataOut.C/dataOut.frequency @@ -2350,30 +2307,30 @@ class WindProfiler(Operation): dataOut.data_output = self.techniqueSA(kwargs) dataOut.utctimeInit = dataOut.utctime dataOut.outputInterval = dataOut.timeInterval - - elif technique == 'Meteors': + + elif technique == 'Meteors': dataOut.flagNoData = True self.__dataReady = False - - if kwargs.has_key('nHours'): + + if 'nHours' in kwargs: nHours = kwargs['nHours'] - else: + else: nHours = 1 - - if kwargs.has_key('meteorsPerBin'): + + if 'meteorsPerBin' in kwargs: meteorThresh = kwargs['meteorsPerBin'] else: meteorThresh = 6 - - if kwargs.has_key('hmin'): + + if 'hmin' in kwargs: hmin = kwargs['hmin'] else: hmin = 70 - if kwargs.has_key('hmax'): + if 'hmax' in kwargs: hmax = kwargs['hmax'] else: hmax = 110 - + dataOut.outputInterval = nHours*3600 - + if self.__isConfig == False: # self.__initime = dataOut.datatime.replace(minute = 0, second = 0, microsecond = 03) #Get Initial LTC time @@ -2381,45 +2338,45 @@ class WindProfiler(Operation): self.__initime = (self.__initime.replace(minute = 0, second = 0, microsecond = 0) - datetime.datetime(1970, 1, 1)).total_seconds() self.__isConfig = True - + if self.__buffer is None: self.__buffer = dataOut.data_param self.__firstdata = copy.copy(dataOut) else: self.__buffer = numpy.vstack((self.__buffer, dataOut.data_param)) - + self.__checkTime(dataOut.utctime, dataOut.paramInterval, dataOut.outputInterval) #Check if the buffer is ready - + if self.__dataReady: dataOut.utctimeInit = self.__initime - + self.__initime += dataOut.outputInterval #to erase time offset - + dataOut.data_output, dataOut.heightList = self.techniqueMeteors(self.__buffer, meteorThresh, hmin, hmax) dataOut.flagNoData = False self.__buffer = None - + elif technique == 'Meteors1': dataOut.flagNoData = True self.__dataReady = False - - if kwargs.has_key('nMins'): + + if 'nMins' in kwargs: nMins = kwargs['nMins'] else: nMins = 20 - if kwargs.has_key('rx_location'): + if 'rx_location' in kwargs: rx_location = kwargs['rx_location'] else: rx_location = [(0,1),(1,1),(1,0)] - if kwargs.has_key('azimuth'): + if 'azimuth' in kwargs: azimuth = kwargs['azimuth'] else: azimuth = 51.06 - if kwargs.has_key('dfactor'): + if 'dfactor' in kwargs: dfactor = kwargs['dfactor'] - if kwargs.has_key('mode'): + if 'mode' in kwargs: mode = kwargs['mode'] - if kwargs.has_key('theta_x'): - theta_x = kwargs['theta_x'] - if kwargs.has_key('theta_y'): + if 'theta_x' in kwargs: + theta_x = kwargs['theta_x'] + if 'theta_y' in kwargs: theta_y = kwargs['theta_y'] else: mode = 'SA' @@ -2431,10 +2388,10 @@ class WindProfiler(Operation): freq = 50e6 lamb = C/freq k = 2*numpy.pi/lamb - + timeList = dataOut.abscissaList heightList = dataOut.heightList - + if self.__isConfig == False: dataOut.outputInterval = nMins*60 # self.__initime = dataOut.datatime.replace(minute = 0, second = 0, microsecond = 03) @@ -2445,20 +2402,20 @@ class WindProfiler(Operation): self.__initime = (initime.replace(minute = minuteNew, second = 0, microsecond = 0) - datetime.datetime(1970, 1, 1)).total_seconds() self.__isConfig = True - + if self.__buffer is None: self.__buffer = dataOut.data_param self.__firstdata = copy.copy(dataOut) else: self.__buffer = numpy.vstack((self.__buffer, dataOut.data_param)) - + self.__checkTime(dataOut.utctime, dataOut.paramInterval, dataOut.outputInterval) #Check if the buffer is ready - + if self.__dataReady: dataOut.utctimeInit = self.__initime self.__initime += dataOut.outputInterval #to erase time offset - + metArray = self.__buffer if mode == 'SA': dataOut.data_output = self.techniqueNSM_SA(rx_location=rx_location, groupList=groupList, azimuth=azimuth, dfactor=dfactor, k=k,metArray=metArray, heightList=heightList,timeList=timeList) @@ -2469,71 +2426,71 @@ class WindProfiler(Operation): self.__buffer = None return - + class EWDriftsEstimation(Operation): - - def __init__(self, **kwargs): - Operation.__init__(self, **kwargs) - + + def __init__(self): + Operation.__init__(self) + def __correctValues(self, heiRang, phi, velRadial, SNR): listPhi = phi.tolist() maxid = listPhi.index(max(listPhi)) minid = listPhi.index(min(listPhi)) - - rango = range(len(phi)) + + rango = list(range(len(phi))) # rango = numpy.delete(rango,maxid) - + heiRang1 = heiRang*math.cos(phi[maxid]) heiRangAux = heiRang*math.cos(phi[minid]) indOut = (heiRang1 < heiRangAux[0]).nonzero() heiRang1 = numpy.delete(heiRang1,indOut) - + velRadial1 = numpy.zeros([len(phi),len(heiRang1)]) SNR1 = numpy.zeros([len(phi),len(heiRang1)]) - + for i in rango: x = heiRang*math.cos(phi[i]) y1 = velRadial[i,:] f1 = interpolate.interp1d(x,y1,kind = 'cubic') - + x1 = heiRang1 y11 = f1(x1) - + y2 = SNR[i,:] f2 = interpolate.interp1d(x,y2,kind = 'cubic') y21 = f2(x1) - + velRadial1[i,:] = y11 SNR1[i,:] = y21 - + return heiRang1, velRadial1, SNR1 def run(self, dataOut, zenith, zenithCorrection): heiRang = dataOut.heightList velRadial = dataOut.data_param[:,3,:] SNR = dataOut.data_SNR - + zenith = numpy.array(zenith) - zenith -= zenithCorrection + zenith -= zenithCorrection zenith *= numpy.pi/180 - + heiRang1, velRadial1, SNR1 = self.__correctValues(heiRang, numpy.abs(zenith), velRadial, SNR) - + alp = zenith[0] bet = zenith[1] - + w_w = velRadial1[0,:] w_e = velRadial1[1,:] - - w = (w_w*numpy.sin(bet) - w_e*numpy.sin(alp))/(numpy.cos(alp)*numpy.sin(bet) - numpy.cos(bet)*numpy.sin(alp)) - u = (w_w*numpy.cos(bet) - w_e*numpy.cos(alp))/(numpy.sin(alp)*numpy.cos(bet) - numpy.sin(bet)*numpy.cos(alp)) - + + w = (w_w*numpy.sin(bet) - w_e*numpy.sin(alp))/(numpy.cos(alp)*numpy.sin(bet) - numpy.cos(bet)*numpy.sin(alp)) + u = (w_w*numpy.cos(bet) - w_e*numpy.cos(alp))/(numpy.sin(alp)*numpy.cos(bet) - numpy.sin(bet)*numpy.cos(alp)) + winds = numpy.vstack((u,w)) - + dataOut.heightList = heiRang1 dataOut.data_output = winds dataOut.data_SNR = SNR1 - + dataOut.utctimeInit = dataOut.utctime dataOut.outputInterval = dataOut.timeInterval return @@ -2546,11 +2503,11 @@ class NonSpecularMeteorDetection(Operation): data_acf = dataOut.data_pre[0] data_ccf = dataOut.data_pre[1] pairsList = dataOut.groupList[1] - + lamb = dataOut.C/dataOut.frequency tSamp = dataOut.ippSeconds*dataOut.nCohInt paramInterval = dataOut.paramInterval - + nChannels = data_acf.shape[0] nLags = data_acf.shape[1] nProfiles = data_acf.shape[2] @@ -2560,7 +2517,7 @@ class NonSpecularMeteorDetection(Operation): heightList = dataOut.heightList ippSeconds = dataOut.ippSeconds*dataOut.nCohInt*dataOut.nAvg utctime = dataOut.utctime - + dataOut.abscissaList = numpy.arange(0,paramInterval+ippSeconds,ippSeconds) #------------------------ SNR -------------------------------------- @@ -2572,7 +2529,7 @@ class NonSpecularMeteorDetection(Operation): SNR[i] = (power[i]-noise[i])/noise[i] SNRm = numpy.nanmean(SNR, axis = 0) SNRdB = 10*numpy.log10(SNR) - + if mode == 'SA': dataOut.groupList = dataOut.groupList[1] nPairs = data_ccf.shape[0] @@ -2580,22 +2537,22 @@ class NonSpecularMeteorDetection(Operation): phase = numpy.zeros(data_ccf[:,0,:,:].shape) # phase1 = numpy.copy(phase) coh1 = numpy.zeros(data_ccf[:,0,:,:].shape) - + for p in range(nPairs): ch0 = pairsList[p][0] ch1 = pairsList[p][1] ccf = data_ccf[p,0,:,:]/numpy.sqrt(data_acf[ch0,0,:,:]*data_acf[ch1,0,:,:]) - phase[p,:,:] = ndimage.median_filter(numpy.angle(ccf), size = (5,1)) #median filter -# phase1[p,:,:] = numpy.angle(ccf) #median filter - coh1[p,:,:] = ndimage.median_filter(numpy.abs(ccf), 5) #median filter -# coh1[p,:,:] = numpy.abs(ccf) #median filter + phase[p,:,:] = ndimage.median_filter(numpy.angle(ccf), size = (5,1)) #median filter +# phase1[p,:,:] = numpy.angle(ccf) #median filter + coh1[p,:,:] = ndimage.median_filter(numpy.abs(ccf), 5) #median filter +# coh1[p,:,:] = numpy.abs(ccf) #median filter coh = numpy.nanmax(coh1, axis = 0) # struc = numpy.ones((5,1)) # coh = ndimage.morphology.grey_dilation(coh, size=(10,1)) #---------------------- Radial Velocity ---------------------------- phaseAux = numpy.mean(numpy.angle(data_acf[:,1,:,:]), axis = 0) velRad = phaseAux*lamb/(4*numpy.pi*tSamp) - + if allData: boolMetFin = ~numpy.isnan(SNRm) # coh[:-1,:] = numpy.nanmean(numpy.abs(phase[:,1:,:] - phase[:,:-1,:]),axis=0) @@ -2603,31 +2560,31 @@ class NonSpecularMeteorDetection(Operation): #------------------------ Meteor mask --------------------------------- # #SNR mask # boolMet = (SNRdB>SNRthresh)#|(~numpy.isnan(SNRdB)) -# +# # #Erase small objects -# boolMet1 = self.__erase_small(boolMet, 2*sec, 5) -# +# boolMet1 = self.__erase_small(boolMet, 2*sec, 5) +# # auxEEJ = numpy.sum(boolMet1,axis=0) # indOver = auxEEJ>nProfiles*0.8 #Use this later # indEEJ = numpy.where(indOver)[0] # indNEEJ = numpy.where(~indOver)[0] -# +# # boolMetFin = boolMet1 -# +# # if indEEJ.size > 0: -# boolMet1[:,indEEJ] = False #Erase heights with EEJ -# +# boolMet1[:,indEEJ] = False #Erase heights with EEJ +# # boolMet2 = coh > cohThresh # boolMet2 = self.__erase_small(boolMet2, 2*sec,5) -# +# # #Final Meteor mask # boolMetFin = boolMet1|boolMet2 - + #Coherence mask boolMet1 = coh > 0.75 struc = numpy.ones((30,1)) boolMet1 = ndimage.morphology.binary_dilation(boolMet1, structure=struc) - + #Derivative mask derPhase = numpy.nanmean(numpy.abs(phase[:,1:,:] - phase[:,:-1,:]),axis=0) boolMet2 = derPhase < 0.2 @@ -2644,7 +2601,7 @@ class NonSpecularMeteorDetection(Operation): tmet = coordMet[0] hmet = coordMet[1] - + data_param = numpy.zeros((tmet.size, 6 + nPairs)) data_param[:,0] = utctime data_param[:,1] = tmet @@ -2653,7 +2610,7 @@ class NonSpecularMeteorDetection(Operation): data_param[:,4] = velRad[tmet,hmet] data_param[:,5] = coh[tmet,hmet] data_param[:,6:] = phase[:,tmet,hmet].T - + elif mode == 'DBS': dataOut.groupList = numpy.arange(nChannels) @@ -2661,7 +2618,7 @@ class NonSpecularMeteorDetection(Operation): phase = numpy.angle(data_acf[:,1,:,:]) # phase = ndimage.median_filter(numpy.angle(data_acf[:,1,:,:]), size = (1,5,1)) velRad = phase*lamb/(4*numpy.pi*tSamp) - + #Spectral width # acf1 = ndimage.median_filter(numpy.abs(data_acf[:,1,:,:]), size = (1,5,1)) # acf2 = ndimage.median_filter(numpy.abs(data_acf[:,2,:,:]), size = (1,5,1)) @@ -2676,24 +2633,24 @@ class NonSpecularMeteorDetection(Operation): #SNR boolMet1 = (SNRdB>SNRthresh) #SNR mask boolMet1 = ndimage.median_filter(boolMet1, size=(1,5,5)) - + #Radial velocity boolMet2 = numpy.abs(velRad) < 20 boolMet2 = ndimage.median_filter(boolMet2, (1,5,5)) - + #Spectral Width boolMet3 = spcWidth < 30 boolMet3 = ndimage.median_filter(boolMet3, (1,5,5)) # boolMetFin = self.__erase_small(boolMet1, 10,5) boolMetFin = boolMet1&boolMet2&boolMet3 - + #Creating data_param coordMet = numpy.where(boolMetFin) cmet = coordMet[0] tmet = coordMet[1] hmet = coordMet[2] - + data_param = numpy.zeros((tmet.size, 7)) data_param[:,0] = utctime data_param[:,1] = cmet @@ -2702,7 +2659,7 @@ class NonSpecularMeteorDetection(Operation): data_param[:,4] = SNR[cmet,tmet,hmet].T data_param[:,5] = velRad[cmet,tmet,hmet].T data_param[:,6] = spcWidth[cmet,tmet,hmet].T - + # self.dataOut.data_param = data_int if len(data_param) == 0: dataOut.flagNoData = True @@ -2712,21 +2669,21 @@ class NonSpecularMeteorDetection(Operation): def __erase_small(self, binArray, threshX, threshY): labarray, numfeat = ndimage.measurements.label(binArray) binArray1 = numpy.copy(binArray) - + for i in range(1,numfeat + 1): auxBin = (labarray==i) auxSize = auxBin.sum() - + x,y = numpy.where(auxBin) widthX = x.max() - x.min() widthY = y.max() - y.min() - + #width X: 3 seg -> 12.5*3 - #width Y: - + #width Y: + if (auxSize < 50) or (widthX < threshX) or (widthY < threshY): binArray1[auxBin] = False - + return binArray1 #--------------- Specular Meteor ---------------- @@ -2736,36 +2693,36 @@ class SMDetection(Operation): Function DetectMeteors() Project developed with paper: HOLDSWORTH ET AL. 2004 - + Input: self.dataOut.data_pre - + centerReceiverIndex: From the channels, which is the center receiver - + hei_ref: Height reference for the Beacon signal extraction tauindex: predefinedPhaseShifts: Predefined phase offset for the voltge signals - + cohDetection: Whether to user Coherent detection or not cohDet_timeStep: Coherent Detection calculation time step cohDet_thresh: Coherent Detection phase threshold to correct phases - + noise_timeStep: Noise calculation time step noise_multiple: Noise multiple to define signal threshold - + multDet_timeLimit: Multiple Detection Removal time limit in seconds multDet_rangeLimit: Multiple Detection Removal range limit in km - + phaseThresh: Maximum phase difference between receiver to be consider a meteor - SNRThresh: Minimum SNR threshold of the meteor signal to be consider a meteor - + SNRThresh: Minimum SNR threshold of the meteor signal to be consider a meteor + hmin: Minimum Height of the meteor to use it in the further wind estimations hmax: Maximum Height of the meteor to use it in the further wind estimations azimuth: Azimuth angle correction - + Affected: self.dataOut.data_param - + Rejection Criteria (Errors): 0: No error; analysis OK 1: SNR < SNR threshold @@ -2784,9 +2741,9 @@ class SMDetection(Operation): 14: height ambiguous echo: more then one possible height within 70 to 110 km 15: radial drift velocity or projected horizontal velocity exceeds 200 m/s 16: oscilatory echo, indicating event most likely not an underdense echo - + 17: phase difference in meteor Reestimation - + Data Storage: Meteors for Wind Estimation (8): Utc Time | Range Height @@ -2794,19 +2751,19 @@ class SMDetection(Operation): VelRad errorVelRad Phase0 Phase1 Phase2 Phase3 TypeError - - ''' - + + ''' + def run(self, dataOut, hei_ref = None, tauindex = 0, phaseOffsets = None, - cohDetection = False, cohDet_timeStep = 1, cohDet_thresh = 25, + cohDetection = False, cohDet_timeStep = 1, cohDet_thresh = 25, noise_timeStep = 4, noise_multiple = 4, multDet_timeLimit = 1, multDet_rangeLimit = 3, phaseThresh = 20, SNRThresh = 5, hmin = 50, hmax=150, azimuth = 0, channelPositions = None) : - - + + #Getting Pairslist if channelPositions is None: # channelPositions = [(2.5,0), (0,2.5), (0,0), (0,4.5), (-2,0)] #T @@ -2816,53 +2773,53 @@ class SMDetection(Operation): heiRang = dataOut.getHeiRange() #Get Beacon signal - No Beacon signal anymore # newheis = numpy.where(self.dataOut.heightList>self.dataOut.radarControllerHeaderObj.Taus[tauindex]) -# +# # if hei_ref != None: # newheis = numpy.where(self.dataOut.heightList>hei_ref) -# - - +# + + #****************REMOVING HARDWARE PHASE DIFFERENCES*************** # see if the user put in pre defined phase shifts voltsPShift = dataOut.data_pre.copy() - + # if predefinedPhaseShifts != None: # hardwarePhaseShifts = numpy.array(predefinedPhaseShifts)*numpy.pi/180 -# +# # # elif beaconPhaseShifts: # # #get hardware phase shifts using beacon signal # # hardwarePhaseShifts = self.__getHardwarePhaseDiff(self.dataOut.data_pre, pairslist, newheis, 10) # # hardwarePhaseShifts = numpy.insert(hardwarePhaseShifts,centerReceiverIndex,0) -# +# # else: -# hardwarePhaseShifts = numpy.zeros(5) -# +# hardwarePhaseShifts = numpy.zeros(5) +# # voltsPShift = numpy.zeros((self.dataOut.data_pre.shape[0],self.dataOut.data_pre.shape[1],self.dataOut.data_pre.shape[2]), dtype = 'complex') # for i in range(self.dataOut.data_pre.shape[0]): # voltsPShift[i,:,:] = self.__shiftPhase(self.dataOut.data_pre[i,:,:], hardwarePhaseShifts[i]) #******************END OF REMOVING HARDWARE PHASE DIFFERENCES********* - + #Remove DC voltsDC = numpy.mean(voltsPShift,1) voltsDC = numpy.mean(voltsDC,1) for i in range(voltsDC.shape[0]): voltsPShift[i] = voltsPShift[i] - voltsDC[i] - - #Don't considerate last heights, theyre used to calculate Hardware Phase Shift + + #Don't considerate last heights, theyre used to calculate Hardware Phase Shift # voltsPShift = voltsPShift[:,:,:newheis[0][0]] - + #************ FIND POWER OF DATA W/COH OR NON COH DETECTION (3.4) ********** #Coherent Detection if cohDetection: #use coherent detection to get the net power cohDet_thresh = cohDet_thresh*numpy.pi/180 voltsPShift = self.__coherentDetection(voltsPShift, cohDet_timeStep, dataOut.timeInterval, pairslist0, cohDet_thresh) - + #Non-coherent detection! powerNet = numpy.nansum(numpy.abs(voltsPShift[:,:,:])**2,0) #********** END OF COH/NON-COH POWER CALCULATION********************** - + #********** FIND THE NOISE LEVEL AND POSSIBLE METEORS **************** #Get noise noise, noise1 = self.__getNoise(powerNet, noise_timeStep, dataOut.timeInterval) @@ -2872,7 +2829,7 @@ class SMDetection(Operation): #Meteor echoes detection listMeteors = self.__findMeteors(powerNet, signalThresh) #******* END OF NOISE LEVEL AND POSSIBLE METEORS CACULATION ********** - + #************** REMOVE MULTIPLE DETECTIONS (3.5) *************************** #Parameters heiRange = dataOut.getHeiRange() @@ -2882,7 +2839,7 @@ class SMDetection(Operation): #Multiple detection removals listMeteors1 = self.__removeMultipleDetections(listMeteors, rangeLimit, timeLimit) #************ END OF REMOVE MULTIPLE DETECTIONS ********************** - + #********************* METEOR REESTIMATION (3.7, 3.8, 3.9, 3.10) ******************** #Parameters phaseThresh = phaseThresh*numpy.pi/180 @@ -2893,40 +2850,40 @@ class SMDetection(Operation): #Estimation of decay times (Errors N 7, 8, 11) listMeteors3 = self.__estimateDecayTime(listMeteors2, listMeteorsPower, dataOut.timeInterval, dataOut.frequency) #******************* END OF METEOR REESTIMATION ******************* - + #********************* METEOR PARAMETERS CALCULATION (3.11, 3.12, 3.13) ************************** #Calculating Radial Velocity (Error N 15) radialStdThresh = 10 - listMeteors4 = self.__getRadialVelocity(listMeteors3, listMeteorsVolts, radialStdThresh, pairslist0, dataOut.timeInterval) + listMeteors4 = self.__getRadialVelocity(listMeteors3, listMeteorsVolts, radialStdThresh, pairslist0, dataOut.timeInterval) if len(listMeteors4) > 0: #Setting New Array date = dataOut.utctime arrayParameters = self.__setNewArrays(listMeteors4, date, heiRang) - + #Correcting phase offset if phaseOffsets != None: phaseOffsets = numpy.array(phaseOffsets)*numpy.pi/180 arrayParameters[:,8:12] = numpy.unwrap(arrayParameters[:,8:12] + phaseOffsets) - + #Second Pairslist pairsList = [] pairx = (0,1) pairy = (2,3) pairsList.append(pairx) pairsList.append(pairy) - + jph = numpy.array([0,0,0,0]) h = (hmin,hmax) arrayParameters = meteorOps.getMeteorParams(arrayParameters, azimuth, h, pairsList, distances, jph) - + # #Calculate AOA (Error N 3, 4) # #JONES ET AL. 1998 # error = arrayParameters[:,-1] # AOAthresh = numpy.pi/8 # phases = -arrayParameters[:,9:13] # arrayParameters[:,4:7], arrayParameters[:,-1] = meteorOps.getAOA(phases, pairsList, error, AOAthresh, azimuth) -# +# # #Calculate Heights (Error N 13 and 14) # error = arrayParameters[:,-1] # Ranges = arrayParameters[:,2] @@ -2934,73 +2891,73 @@ class SMDetection(Operation): # arrayParameters[:,3], arrayParameters[:,-1] = meteorOps.getHeights(Ranges, zenith, error, hmin, hmax) # error = arrayParameters[:,-1] #********************* END OF PARAMETERS CALCULATION ************************** - - #***************************+ PASS DATA TO NEXT STEP ********************** + + #***************************+ PASS DATA TO NEXT STEP ********************** # arrayFinal = arrayParameters.reshape((1,arrayParameters.shape[0],arrayParameters.shape[1])) dataOut.data_param = arrayParameters - + if arrayParameters is None: dataOut.flagNoData = True else: dataOut.flagNoData = True - + return - + def __getHardwarePhaseDiff(self, voltage0, pairslist, newheis, n): - + minIndex = min(newheis[0]) maxIndex = max(newheis[0]) - + voltage = voltage0[:,:,minIndex:maxIndex+1] nLength = voltage.shape[1]/n nMin = 0 nMax = 0 phaseOffset = numpy.zeros((len(pairslist),n)) - + for i in range(n): nMax += nLength phaseCCF = -numpy.angle(self.__calculateCCF(voltage[:,nMin:nMax,:], pairslist, [0])) phaseCCF = numpy.mean(phaseCCF, axis = 2) - phaseOffset[:,i] = phaseCCF.transpose() + phaseOffset[:,i] = phaseCCF.transpose() nMin = nMax # phaseDiff, phaseArrival = self.estimatePhaseDifference(voltage, pairslist) - + #Remove Outliers factor = 2 wt = phaseOffset - signal.medfilt(phaseOffset,(1,5)) dw = numpy.std(wt,axis = 1) dw = dw.reshape((dw.size,1)) - ind = numpy.where(numpy.logical_or(wt>dw*factor,wt<-dw*factor)) + ind = numpy.where(numpy.logical_or(wt>dw*factor,wt<-dw*factor)) phaseOffset[ind] = numpy.nan - phaseOffset = stats.nanmean(phaseOffset, axis=1) - + phaseOffset = stats.nanmean(phaseOffset, axis=1) + return phaseOffset - + def __shiftPhase(self, data, phaseShift): #this will shift the phase of a complex number - dataShifted = numpy.abs(data) * numpy.exp((numpy.angle(data)+phaseShift)*1j) + dataShifted = numpy.abs(data) * numpy.exp((numpy.angle(data)+phaseShift)*1j) return dataShifted - + def __estimatePhaseDifference(self, array, pairslist): nChannel = array.shape[0] nHeights = array.shape[2] numPairs = len(pairslist) # phaseCCF = numpy.zeros((nChannel, 5, nHeights)) phaseCCF = numpy.angle(self.__calculateCCF(array, pairslist, [-2,-1,0,1,2])) - + #Correct phases derPhaseCCF = phaseCCF[:,1:,:] - phaseCCF[:,0:-1,:] indDer = numpy.where(numpy.abs(derPhaseCCF) > numpy.pi) - - if indDer[0].shape[0] > 0: + + if indDer[0].shape[0] > 0: for i in range(indDer[0].shape[0]): signo = -numpy.sign(derPhaseCCF[indDer[0][i],indDer[1][i],indDer[2][i]]) phaseCCF[indDer[0][i],indDer[1][i]+1:,:] += signo*2*numpy.pi - + # for j in range(numSides): # phaseCCFAux = self.calculateCCF(arrayCenter, arraySides[j,:,:], [-2,1,0,1,2]) # phaseCCF[j,:,:] = numpy.angle(phaseCCFAux) -# +# #Linear phaseInt = numpy.zeros((numPairs,1)) angAllCCF = phaseCCF[:,[0,1,3,4],0] @@ -3010,16 +2967,16 @@ class SMDetection(Operation): #Phase Differences phaseDiff = phaseInt - phaseCCF[:,2,:] phaseArrival = phaseInt.reshape(phaseInt.size) - + #Dealias phaseArrival = numpy.angle(numpy.exp(1j*phaseArrival)) # indAlias = numpy.where(phaseArrival > numpy.pi) # phaseArrival[indAlias] -= 2*numpy.pi # indAlias = numpy.where(phaseArrival < -numpy.pi) # phaseArrival[indAlias] += 2*numpy.pi - + return phaseDiff, phaseArrival - + def __coherentDetection(self, volts, timeSegment, timeInterval, pairslist, thresh): #this function will run the coherent detection used in Holdworth et al. 2004 and return the net power #find the phase shifts of each channel over 1 second intervals @@ -3029,25 +2986,25 @@ class SMDetection(Operation): numHeights = volts.shape[2] nChannel = volts.shape[0] voltsCohDet = volts.copy() - + pairsarray = numpy.array(pairslist) indSides = pairsarray[:,1] # indSides = numpy.array(range(nChannel)) # indSides = numpy.delete(indSides, indCenter) -# +# # listCenter = numpy.array_split(volts[indCenter,:,:], numBlocks, 0) listBlocks = numpy.array_split(volts, numBlocks, 1) - + startInd = 0 endInd = 0 - + for i in range(numBlocks): startInd = endInd - endInd = endInd + listBlocks[i].shape[1] - + endInd = endInd + listBlocks[i].shape[1] + arrayBlock = listBlocks[i] # arrayBlockCenter = listCenter[i] - + #Estimate the Phase Difference phaseDiff, aux = self.__estimatePhaseDifference(arrayBlock, pairslist) #Phase Difference RMS @@ -3059,21 +3016,21 @@ class SMDetection(Operation): for j in range(indSides.size): arrayBlock[indSides[j],:,indPhase] = self.__shiftPhase(arrayBlock[indSides[j],:,indPhase], phaseDiff[j,indPhase].transpose()) voltsCohDet[:,startInd:endInd,:] = arrayBlock - + return voltsCohDet - + def __calculateCCF(self, volts, pairslist ,laglist): - + nHeights = volts.shape[2] - nPoints = volts.shape[1] + nPoints = volts.shape[1] voltsCCF = numpy.zeros((len(pairslist), len(laglist), nHeights),dtype = 'complex') - + for i in range(len(pairslist)): volts1 = volts[pairslist[i][0]] - volts2 = volts[pairslist[i][1]] - + volts2 = volts[pairslist[i][1]] + for t in range(len(laglist)): - idxT = laglist[t] + idxT = laglist[t] if idxT >= 0: vStacked = numpy.vstack((volts2[idxT:,:], numpy.zeros((idxT, nHeights),dtype='complex'))) @@ -3081,10 +3038,10 @@ class SMDetection(Operation): vStacked = numpy.vstack((numpy.zeros((-idxT, nHeights),dtype='complex'), volts2[:(nPoints + idxT),:])) voltsCCF[i,t,:] = numpy.sum((numpy.conjugate(volts1)*vStacked),axis=0) - + vStacked = None return voltsCCF - + def __getNoise(self, power, timeSegment, timeInterval): numProfPerBlock = numpy.ceil(timeSegment/timeInterval) numBlocks = int(power.shape[0]/numProfPerBlock) @@ -3093,100 +3050,100 @@ class SMDetection(Operation): listPower = numpy.array_split(power, numBlocks, 0) noise = numpy.zeros((power.shape[0], power.shape[1])) noise1 = numpy.zeros((power.shape[0], power.shape[1])) - + startInd = 0 endInd = 0 - + for i in range(numBlocks): #split por canal startInd = endInd - endInd = endInd + listPower[i].shape[0] - + endInd = endInd + listPower[i].shape[0] + arrayBlock = listPower[i] noiseAux = numpy.mean(arrayBlock, 0) # noiseAux = numpy.median(noiseAux) # noiseAux = numpy.mean(arrayBlock) - noise[startInd:endInd,:] = noise[startInd:endInd,:] + noiseAux - + noise[startInd:endInd,:] = noise[startInd:endInd,:] + noiseAux + noiseAux1 = numpy.mean(arrayBlock) - noise1[startInd:endInd,:] = noise1[startInd:endInd,:] + noiseAux1 - + noise1[startInd:endInd,:] = noise1[startInd:endInd,:] + noiseAux1 + return noise, noise1 - + def __findMeteors(self, power, thresh): nProf = power.shape[0] nHeights = power.shape[1] listMeteors = [] - + for i in range(nHeights): powerAux = power[:,i] threshAux = thresh[:,i] - + indUPthresh = numpy.where(powerAux > threshAux)[0] indDNthresh = numpy.where(powerAux <= threshAux)[0] - + j = 0 - + while (j < indUPthresh.size - 2): if (indUPthresh[j + 2] == indUPthresh[j] + 2): indDNAux = numpy.where(indDNthresh > indUPthresh[j]) indDNthresh = indDNthresh[indDNAux] - + if (indDNthresh.size > 0): indEnd = indDNthresh[0] - 1 indInit = indUPthresh[j] - + meteor = powerAux[indInit:indEnd + 1] indPeak = meteor.argmax() + indInit FLA = sum(numpy.conj(meteor)*numpy.hstack((meteor[1:],0))) - + listMeteors.append(numpy.array([i,indInit,indPeak,indEnd,FLA])) #CHEQUEAR!!!!! j = numpy.where(indUPthresh == indEnd)[0] + 1 else: j+=1 else: j+=1 - + return listMeteors - + def __removeMultipleDetections(self,listMeteors, rangeLimit, timeLimit): - - arrayMeteors = numpy.asarray(listMeteors) + + arrayMeteors = numpy.asarray(listMeteors) listMeteors1 = [] - + while arrayMeteors.shape[0] > 0: FLAs = arrayMeteors[:,4] maxFLA = FLAs.argmax() listMeteors1.append(arrayMeteors[maxFLA,:]) - + MeteorInitTime = arrayMeteors[maxFLA,1] MeteorEndTime = arrayMeteors[maxFLA,3] MeteorHeight = arrayMeteors[maxFLA,0] - + #Check neighborhood maxHeightIndex = MeteorHeight + rangeLimit minHeightIndex = MeteorHeight - rangeLimit minTimeIndex = MeteorInitTime - timeLimit maxTimeIndex = MeteorEndTime + timeLimit - + #Check Heights indHeight = numpy.logical_and(arrayMeteors[:,0] >= minHeightIndex, arrayMeteors[:,0] <= maxHeightIndex) indTime = numpy.logical_and(arrayMeteors[:,3] >= minTimeIndex, arrayMeteors[:,1] <= maxTimeIndex) indBoth = numpy.where(numpy.logical_and(indTime,indHeight)) - + arrayMeteors = numpy.delete(arrayMeteors, indBoth, axis = 0) - + return listMeteors1 - + def __meteorReestimation(self, listMeteors, volts, pairslist, thresh, noise, timeInterval,frequency): numHeights = volts.shape[2] nChannel = volts.shape[0] - + thresholdPhase = thresh[0] thresholdNoise = thresh[1] thresholdDB = float(thresh[2]) - + thresholdDB1 = 10**(thresholdDB/10) pairsarray = numpy.array(pairslist) indSides = pairsarray[:,1] - + pairslist1 = list(pairslist) pairslist1.append((0,1)) pairslist1.append((3,4)) @@ -3195,31 +3152,31 @@ class SMDetection(Operation): listPowerSeries = [] listVoltageSeries = [] #volts has the war data - + if frequency == 30e6: timeLag = 45*10**-3 else: timeLag = 15*10**-3 lag = numpy.ceil(timeLag/timeInterval) - + for i in range(len(listMeteors)): - + ###################### 3.6 - 3.7 PARAMETERS REESTIMATION ######################### meteorAux = numpy.zeros(16) - + #Loading meteor Data (mHeight, mStart, mPeak, mEnd) mHeight = listMeteors[i][0] mStart = listMeteors[i][1] mPeak = listMeteors[i][2] mEnd = listMeteors[i][3] - + #get the volt data between the start and end times of the meteor meteorVolts = volts[:,mStart:mEnd+1,mHeight] meteorVolts = meteorVolts.reshape(meteorVolts.shape[0], meteorVolts.shape[1], 1) - + #3.6. Phase Difference estimation phaseDiff, aux = self.__estimatePhaseDifference(meteorVolts, pairslist) - + #3.7. Phase difference removal & meteor start, peak and end times reestimated #meteorVolts0.- all Channels, all Profiles meteorVolts0 = volts[:,:,mHeight] @@ -3227,15 +3184,15 @@ class SMDetection(Operation): meteorNoise = noise[:,mHeight] meteorVolts0[indSides,:] = self.__shiftPhase(meteorVolts0[indSides,:], phaseDiff) #Phase Shifting powerNet0 = numpy.nansum(numpy.abs(meteorVolts0)**2, axis = 0) #Power - + #Times reestimation mStart1 = numpy.where(powerNet0[:mPeak] < meteorThresh[:mPeak])[0] if mStart1.size > 0: mStart1 = mStart1[-1] + 1 - - else: + + else: mStart1 = mPeak - + mEnd1 = numpy.where(powerNet0[mPeak:] < meteorThresh[mPeak:])[0][0] + mPeak - 1 mEndDecayTime1 = numpy.where(powerNet0[mPeak:] < meteorNoise[mPeak:])[0] if mEndDecayTime1.size == 0: @@ -3243,7 +3200,7 @@ class SMDetection(Operation): else: mEndDecayTime1 = mEndDecayTime1[0] + mPeak - 1 # mPeak1 = meteorVolts0[mStart1:mEnd1 + 1].argmax() - + #meteorVolts1.- all Channels, from start to end meteorVolts1 = meteorVolts0[:,mStart1:mEnd1 + 1] meteorVolts2 = meteorVolts0[:,mPeak + lag:mEnd1 + 1] @@ -3252,17 +3209,17 @@ class SMDetection(Operation): meteorVolts1 = meteorVolts1.reshape(meteorVolts1.shape[0], meteorVolts1.shape[1], 1) meteorVolts2 = meteorVolts2.reshape(meteorVolts2.shape[0], meteorVolts2.shape[1], 1) ##################### END PARAMETERS REESTIMATION ######################### - + ##################### 3.8 PHASE DIFFERENCE REESTIMATION ######################## # if mEnd1 - mStart1 > 4: #Error Number 6: echo less than 5 samples long; too short for analysis - if meteorVolts2.shape[1] > 0: + if meteorVolts2.shape[1] > 0: #Phase Difference re-estimation phaseDiff1, phaseDiffint = self.__estimatePhaseDifference(meteorVolts2, pairslist1) #Phase Difference Estimation # phaseDiff1, phaseDiffint = self.estimatePhaseDifference(meteorVolts2, pairslist) meteorVolts2 = meteorVolts2.reshape(meteorVolts2.shape[0], meteorVolts2.shape[1]) phaseDiff11 = numpy.reshape(phaseDiff1, (phaseDiff1.shape[0],1)) meteorVolts2[indSides,:] = self.__shiftPhase(meteorVolts2[indSides,:], phaseDiff11[0:4]) #Phase Shifting - + #Phase Difference RMS phaseRMS1 = numpy.sqrt(numpy.mean(numpy.square(phaseDiff1))) powerNet1 = numpy.nansum(numpy.abs(meteorVolts1[:,:])**2,0) @@ -3277,27 +3234,27 @@ class SMDetection(Operation): #Vectorize meteorAux[0:7] = [mHeight, mStart1, mPeak1, mEnd1, mPeakPower1, mSNR1, phaseRMS1] meteorAux[7:11] = phaseDiffint[0:4] - + #Rejection Criterions if phaseRMS1 > thresholdPhase: #Error Number 17: Phase variation meteorAux[-1] = 17 elif mSNR1 < thresholdDB1: #Error Number 1: SNR < threshold dB meteorAux[-1] = 1 - - - else: + + + else: meteorAux[0:4] = [mHeight, mStart, mPeak, mEnd] meteorAux[-1] = 6 #Error Number 6: echo less than 5 samples long; too short for analysis PowerSeries = 0 - + listMeteors1.append(meteorAux) listPowerSeries.append(PowerSeries) listVoltageSeries.append(meteorVolts1) - - return listMeteors1, listPowerSeries, listVoltageSeries - + + return listMeteors1, listPowerSeries, listVoltageSeries + def __estimateDecayTime(self, listMeteors, listPower, timeInterval, frequency): - + threshError = 10 #Depending if it is 30 or 50 MHz if frequency == 30e6: @@ -3305,22 +3262,22 @@ class SMDetection(Operation): else: timeLag = 15*10**-3 lag = numpy.ceil(timeLag/timeInterval) - + listMeteors1 = [] - + for i in range(len(listMeteors)): meteorPower = listPower[i] meteorAux = listMeteors[i] - + if meteorAux[-1] == 0: - try: + try: indmax = meteorPower.argmax() indlag = indmax + lag - + y = meteorPower[indlag:] x = numpy.arange(0, y.size)*timeLag - + #first guess a = y[0] tau = timeLag @@ -3329,26 +3286,26 @@ class SMDetection(Operation): y1 = self.__exponential_function(x, *popt) #error estimation error = sum((y - y1)**2)/(numpy.var(y)*(y.size - popt.size)) - + decayTime = popt[1] riseTime = indmax*timeInterval meteorAux[11:13] = [decayTime, error] - + #Table items 7, 8 and 11 if (riseTime > 0.3): #Number 7: Echo rise exceeds 0.3s - meteorAux[-1] = 7 + meteorAux[-1] = 7 elif (decayTime < 2*riseTime) : #Number 8: Echo decay time less than than twice rise time meteorAux[-1] = 8 if (error > threshError): #Number 11: Poor fit to amplitude for estimation of decay time - meteorAux[-1] = 11 - - + meteorAux[-1] = 11 + + except: - meteorAux[-1] = 11 - - + meteorAux[-1] = 11 + + listMeteors1.append(meteorAux) - + return listMeteors1 #Exponential Function @@ -3356,9 +3313,9 @@ class SMDetection(Operation): def __exponential_function(self, x, a, tau): y = a*numpy.exp(-x/tau) return y - + def __getRadialVelocity(self, listMeteors, listVolts, radialStdThresh, pairslist, timeInterval): - + pairslist1 = list(pairslist) pairslist1.append((0,1)) pairslist1.append((3,4)) @@ -3368,33 +3325,33 @@ class SMDetection(Operation): c = 3e8 lag = numpy.ceil(timeLag/timeInterval) freq = 30e6 - + listMeteors1 = [] - + for i in range(len(listMeteors)): meteorAux = listMeteors[i] if meteorAux[-1] == 0: mStart = listMeteors[i][1] - mPeak = listMeteors[i][2] + mPeak = listMeteors[i][2] mLag = mPeak - mStart + lag - + #get the volt data between the start and end times of the meteor meteorVolts = listVolts[i] meteorVolts = meteorVolts.reshape(meteorVolts.shape[0], meteorVolts.shape[1], 1) #Get CCF allCCFs = self.__calculateCCF(meteorVolts, pairslist1, [-2,-1,0,1,2]) - + #Method 2 slopes = numpy.zeros(numPairs) time = numpy.array([-2,-1,1,2])*timeInterval angAllCCF = numpy.angle(allCCFs[:,[0,1,3,4],0]) - + #Correct phases derPhaseCCF = angAllCCF[:,1:] - angAllCCF[:,0:-1] indDer = numpy.where(numpy.abs(derPhaseCCF) > numpy.pi) - - if indDer[0].shape[0] > 0: + + if indDer[0].shape[0] > 0: for i in range(indDer[0].shape[0]): signo = -numpy.sign(derPhaseCCF[indDer[0][i],indDer[1][i]]) angAllCCF[indDer[0][i],indDer[1][i]+1:] += signo*2*numpy.pi @@ -3403,51 +3360,51 @@ class SMDetection(Operation): for j in range(numPairs): fit = stats.linregress(time, angAllCCF[j,:]) slopes[j] = fit[0] - + #Remove Outlier # indOut = numpy.argmax(numpy.abs(slopes - numpy.mean(slopes))) # slopes = numpy.delete(slopes,indOut) # indOut = numpy.argmax(numpy.abs(slopes - numpy.mean(slopes))) # slopes = numpy.delete(slopes,indOut) - + radialVelocity = -numpy.mean(slopes)*(0.25/numpy.pi)*(c/freq) radialError = numpy.std(slopes)*(0.25/numpy.pi)*(c/freq) meteorAux[-2] = radialError meteorAux[-3] = radialVelocity - + #Setting Error #Number 15: Radial Drift velocity or projected horizontal velocity exceeds 200 m/s - if numpy.abs(radialVelocity) > 200: + if numpy.abs(radialVelocity) > 200: meteorAux[-1] = 15 #Number 12: Poor fit to CCF variation for estimation of radial drift velocity elif radialError > radialStdThresh: meteorAux[-1] = 12 - + listMeteors1.append(meteorAux) return listMeteors1 - + def __setNewArrays(self, listMeteors, date, heiRang): - + #New arrays arrayMeteors = numpy.array(listMeteors) arrayParameters = numpy.zeros((len(listMeteors), 13)) - + #Date inclusion # date = re.findall(r'\((.*?)\)', date) # date = date[0].split(',') # date = map(int, date) -# +# # if len(date)<6: # date.append(0) -# +# # date = [date[0]*10000 + date[1]*100 + date[2], date[3]*10000 + date[4]*100 + date[5]] # arrayDate = numpy.tile(date, (len(listMeteors), 1)) arrayDate = numpy.tile(date, (len(listMeteors))) - + #Meteor array # arrayMeteors[:,0] = heiRang[arrayMeteors[:,0].astype(int)] # arrayMeteors = numpy.hstack((arrayDate, arrayMeteors)) - + #Parameters Array arrayParameters[:,0] = arrayDate #Date arrayParameters[:,1] = heiRang[arrayMeteors[:,0].astype(int)] #Range @@ -3455,13 +3412,13 @@ class SMDetection(Operation): arrayParameters[:,8:12] = arrayMeteors[:,7:11] #Phases arrayParameters[:,-1] = arrayMeteors[:,-1] #Error - + return arrayParameters - + class CorrectSMPhases(Operation): - + def run(self, dataOut, phaseOffsets, hmin = 50, hmax = 150, azimuth = 45, channelPositions = None): - + arrayParameters = dataOut.data_param pairsList = [] pairx = (0,1) @@ -3469,49 +3426,49 @@ class CorrectSMPhases(Operation): pairsList.append(pairx) pairsList.append(pairy) jph = numpy.zeros(4) - + phaseOffsets = numpy.array(phaseOffsets)*numpy.pi/180 # arrayParameters[:,8:12] = numpy.unwrap(arrayParameters[:,8:12] + phaseOffsets) arrayParameters[:,8:12] = numpy.angle(numpy.exp(1j*(arrayParameters[:,8:12] + phaseOffsets))) - + meteorOps = SMOperations() if channelPositions is None: # channelPositions = [(2.5,0), (0,2.5), (0,0), (0,4.5), (-2,0)] #T channelPositions = [(4.5,2), (2,4.5), (2,2), (2,0), (0,2)] #Estrella - + pairslist0, distances = meteorOps.getPhasePairs(channelPositions) h = (hmin,hmax) - + arrayParameters = meteorOps.getMeteorParams(arrayParameters, azimuth, h, pairsList, distances, jph) - + dataOut.data_param = arrayParameters return class SMPhaseCalibration(Operation): - + __buffer = None __initime = None __dataReady = False - + __isConfig = False - + def __checkTime(self, currentTime, initTime, paramInterval, outputInterval): - + dataTime = currentTime + paramInterval deltaTime = dataTime - initTime - + if deltaTime >= outputInterval or deltaTime < 0: return True - + return False - + def __getGammas(self, pairs, d, phases): gammas = numpy.zeros(2) - + for i in range(len(pairs)): - + pairi = pairs[i] phip3 = phases[:,pairi[0]] @@ -3525,7 +3482,7 @@ class SMPhaseCalibration(Operation): jgamma = numpy.angle(numpy.exp(1j*jgamma)) # jgamma[jgamma>numpy.pi] -= 2*numpy.pi # jgamma[jgamma<-numpy.pi] += 2*numpy.pi - + #Revised distribution jgammaArray = numpy.hstack((jgamma,jgamma+0.5*numpy.pi,jgamma-0.5*numpy.pi)) @@ -3534,39 +3491,39 @@ class SMPhaseCalibration(Operation): rmin = -0.5*numpy.pi rmax = 0.5*numpy.pi phaseHisto = numpy.histogram(jgammaArray, bins=nBins, range=(rmin,rmax)) - + meteorsY = phaseHisto[0] phasesX = phaseHisto[1][:-1] width = phasesX[1] - phasesX[0] phasesX += width/2 - + #Gaussian aproximation bpeak = meteorsY.argmax() peak = meteorsY.max() jmin = bpeak - 5 jmax = bpeak + 5 + 1 - + if jmin<0: jmin = 0 jmax = 6 elif jmax > meteorsY.size: jmin = meteorsY.size - 6 jmax = meteorsY.size - + x0 = numpy.array([peak,bpeak,50]) coeff = optimize.leastsq(self.__residualFunction, x0, args=(meteorsY[jmin:jmax], phasesX[jmin:jmax])) - + #Gammas gammas[i] = coeff[0][1] - + return gammas - + def __residualFunction(self, coeffs, y, t): - + return y - self.__gauss_function(t, coeffs) def __gauss_function(self, t, coeffs): - + return coeffs[0]*numpy.exp(-0.5*((t - coeffs[1]) / coeffs[2])**2) def __getPhases(self, azimuth, h, pairsList, d, gammas, meteorsArray): @@ -3587,16 +3544,16 @@ class SMPhaseCalibration(Operation): max_xangle = range_angle[iz]/2 + center_xangle min_yangle = -range_angle[iz]/2 + center_yangle max_yangle = range_angle[iz]/2 + center_yangle - + inc_x = (max_xangle-min_xangle)/nstepsx inc_y = (max_yangle-min_yangle)/nstepsy - + alpha_y = numpy.arange(nstepsy)*inc_y + min_yangle alpha_x = numpy.arange(nstepsx)*inc_x + min_xangle penalty = numpy.zeros((nstepsx,nstepsy)) jph_array = numpy.zeros((nchan,nstepsx,nstepsy)) jph = numpy.zeros(nchan) - + # Iterations looking for the offset for iy in range(int(nstepsy)): for ix in range(int(nstepsx)): @@ -3604,46 +3561,46 @@ class SMPhaseCalibration(Operation): d2 = d[pairsList[1][1]] d5 = d[pairsList[0][0]] d4 = d[pairsList[0][1]] - + alp2 = alpha_y[iy] #gamma 1 - alp4 = alpha_x[ix] #gamma 0 - + alp4 = alpha_x[ix] #gamma 0 + alp3 = -alp2*d3/d2 - gammas[1] alp5 = -alp4*d5/d4 - gammas[0] # jph[pairy[1]] = alpha_y[iy] -# jph[pairy[0]] = -gammas[1] - alpha_y[iy]*d[pairy[1]]/d[pairy[0]] - +# jph[pairy[0]] = -gammas[1] - alpha_y[iy]*d[pairy[1]]/d[pairy[0]] + # jph[pairx[1]] = alpha_x[ix] # jph[pairx[0]] = -gammas[0] - alpha_x[ix]*d[pairx[1]]/d[pairx[0]] jph[pairsList[0][1]] = alp4 jph[pairsList[0][0]] = alp5 jph[pairsList[1][0]] = alp3 - jph[pairsList[1][1]] = alp2 + jph[pairsList[1][1]] = alp2 jph_array[:,ix,iy] = jph # d = [2.0,2.5,2.5,2.0] - #falta chequear si va a leer bien los meteoros + #falta chequear si va a leer bien los meteoros meteorsArray1 = meteorOps.getMeteorParams(meteorsArray, azimuth, h, pairsList, d, jph) error = meteorsArray1[:,-1] ind1 = numpy.where(error==0)[0] penalty[ix,iy] = ind1.size - + i,j = numpy.unravel_index(penalty.argmax(), penalty.shape) phOffset = jph_array[:,i,j] - + center_xangle = phOffset[pairx[1]] center_yangle = phOffset[pairy[1]] - + phOffset = numpy.angle(numpy.exp(1j*jph_array[:,i,j])) - phOffset = phOffset*180/numpy.pi + phOffset = phOffset*180/numpy.pi return phOffset - - + + def run(self, dataOut, hmin, hmax, channelPositions=None, nHours = 1): - + dataOut.flagNoData = True - self.__dataReady = False + self.__dataReady = False dataOut.outputInterval = nHours*3600 - + if self.__isConfig == False: # self.__initime = dataOut.datatime.replace(minute = 0, second = 0, microsecond = 03) #Get Initial LTC time @@ -3651,19 +3608,19 @@ class SMPhaseCalibration(Operation): self.__initime = (self.__initime.replace(minute = 0, second = 0, microsecond = 0) - datetime.datetime(1970, 1, 1)).total_seconds() self.__isConfig = True - + if self.__buffer is None: self.__buffer = dataOut.data_param.copy() else: self.__buffer = numpy.vstack((self.__buffer, dataOut.data_param)) - + self.__dataReady = self.__checkTime(dataOut.utctime, self.__initime, dataOut.paramInterval, dataOut.outputInterval) #Check if the buffer is ready - + if self.__dataReady: dataOut.utctimeInit = self.__initime self.__initime += dataOut.outputInterval #to erase time offset - + freq = dataOut.frequency c = dataOut.C #m/s lamb = c/freq @@ -3685,13 +3642,13 @@ class SMPhaseCalibration(Operation): pairs.append((1,0)) else: pairs.append((0,1)) - + if distances[3] > distances[2]: pairs.append((3,2)) else: pairs.append((2,3)) # distances1 = [-distances[0]*lamb, distances[1]*lamb, -distances[2]*lamb, distances[3]*lamb] - + meteorsArray = self.__buffer error = meteorsArray[:,-1] boolError = (error==0)|(error==3)|(error==4)|(error==13)|(error==14) @@ -3699,7 +3656,7 @@ class SMPhaseCalibration(Operation): meteorsArray = meteorsArray[ind1,:] meteorsArray[:,-1] = 0 phases = meteorsArray[:,8:12] - + #Calculate Gammas gammas = self.__getGammas(pairs, distances, phases) # gammas = numpy.array([-21.70409463,45.76935864])*numpy.pi/180 @@ -3709,22 +3666,22 @@ class SMPhaseCalibration(Operation): dataOut.data_output = -phasesOff dataOut.flagNoData = False self.__buffer = None - - + + return - + class SMOperations(): - + def __init__(self): - + return - + def getMeteorParams(self, arrayParameters0, azimuth, h, pairsList, distances, jph): - + arrayParameters = arrayParameters0.copy() hmin = h[0] hmax = h[1] - + #Calculate AOA (Error N 3, 4) #JONES ET AL. 1998 AOAthresh = numpy.pi/8 @@ -3732,72 +3689,72 @@ class SMOperations(): phases = -arrayParameters[:,8:12] + jph # phases = numpy.unwrap(phases) arrayParameters[:,3:6], arrayParameters[:,-1] = self.__getAOA(phases, pairsList, distances, error, AOAthresh, azimuth) - + #Calculate Heights (Error N 13 and 14) error = arrayParameters[:,-1] Ranges = arrayParameters[:,1] zenith = arrayParameters[:,4] arrayParameters[:,2], arrayParameters[:,-1] = self.__getHeights(Ranges, zenith, error, hmin, hmax) - + #----------------------- Get Final data ------------------------------------ # error = arrayParameters[:,-1] # ind1 = numpy.where(error==0)[0] # arrayParameters = arrayParameters[ind1,:] - + return arrayParameters - + def __getAOA(self, phases, pairsList, directions, error, AOAthresh, azimuth): - + arrayAOA = numpy.zeros((phases.shape[0],3)) cosdir0, cosdir = self.__getDirectionCosines(phases, pairsList,directions) - + arrayAOA[:,:2] = self.__calculateAOA(cosdir, azimuth) cosDirError = numpy.sum(numpy.abs(cosdir0 - cosdir), axis = 1) arrayAOA[:,2] = cosDirError - + azimuthAngle = arrayAOA[:,0] zenithAngle = arrayAOA[:,1] - + #Setting Error indError = numpy.where(numpy.logical_or(error == 3, error == 4))[0] error[indError] = 0 #Number 3: AOA not fesible indInvalid = numpy.where(numpy.logical_and((numpy.logical_or(numpy.isnan(zenithAngle), numpy.isnan(azimuthAngle))),error == 0))[0] - error[indInvalid] = 3 + error[indInvalid] = 3 #Number 4: Large difference in AOAs obtained from different antenna baselines indInvalid = numpy.where(numpy.logical_and(cosDirError > AOAthresh,error == 0))[0] - error[indInvalid] = 4 + error[indInvalid] = 4 return arrayAOA, error - + def __getDirectionCosines(self, arrayPhase, pairsList, distances): - + #Initializing some variables ang_aux = numpy.array([-8,-7,-6,-5,-4,-3,-2,-1,0,1,2,3,4,5,6,7,8])*2*numpy.pi ang_aux = ang_aux.reshape(1,ang_aux.size) - + cosdir = numpy.zeros((arrayPhase.shape[0],2)) cosdir0 = numpy.zeros((arrayPhase.shape[0],2)) - - + + for i in range(2): ph0 = arrayPhase[:,pairsList[i][0]] ph1 = arrayPhase[:,pairsList[i][1]] d0 = distances[pairsList[i][0]] d1 = distances[pairsList[i][1]] - - ph0_aux = ph0 + ph1 + + ph0_aux = ph0 + ph1 ph0_aux = numpy.angle(numpy.exp(1j*ph0_aux)) # ph0_aux[ph0_aux > numpy.pi] -= 2*numpy.pi -# ph0_aux[ph0_aux < -numpy.pi] += 2*numpy.pi +# ph0_aux[ph0_aux < -numpy.pi] += 2*numpy.pi #First Estimation cosdir0[:,i] = (ph0_aux)/(2*numpy.pi*(d0 - d1)) - + #Most-Accurate Second Estimation phi1_aux = ph0 - ph1 phi1_aux = phi1_aux.reshape(phi1_aux.size,1) #Direction Cosine 1 cosdir1 = (phi1_aux + ang_aux)/(2*numpy.pi*(d0 + d1)) - + #Searching the correct Direction Cosine cosdir0_aux = cosdir0[:,i] cosdir0_aux = cosdir0_aux.reshape(cosdir0_aux.size,1) @@ -3806,59 +3763,59 @@ class SMOperations(): indcos = cosDiff.argmin(axis = 1) #Saving Value obtained cosdir[:,i] = cosdir1[numpy.arange(len(indcos)),indcos] - + return cosdir0, cosdir - + def __calculateAOA(self, cosdir, azimuth): cosdirX = cosdir[:,0] cosdirY = cosdir[:,1] - + zenithAngle = numpy.arccos(numpy.sqrt(1 - cosdirX**2 - cosdirY**2))*180/numpy.pi azimuthAngle = numpy.arctan2(cosdirX,cosdirY)*180/numpy.pi + azimuth#0 deg north, 90 deg east angles = numpy.vstack((azimuthAngle, zenithAngle)).transpose() - + return angles - + def __getHeights(self, Ranges, zenith, error, minHeight, maxHeight): - + Ramb = 375 #Ramb = c/(2*PRF) Re = 6371 #Earth Radius heights = numpy.zeros(Ranges.shape) - + R_aux = numpy.array([0,1,2])*Ramb R_aux = R_aux.reshape(1,R_aux.size) Ranges = Ranges.reshape(Ranges.size,1) - + Ri = Ranges + R_aux hi = numpy.sqrt(Re**2 + Ri**2 + (2*Re*numpy.cos(zenith*numpy.pi/180)*Ri.transpose()).transpose()) - Re - + #Check if there is a height between 70 and 110 km h_bool = numpy.sum(numpy.logical_and(hi > minHeight, hi < maxHeight), axis = 1) ind_h = numpy.where(h_bool == 1)[0] - + hCorr = hi[ind_h, :] ind_hCorr = numpy.where(numpy.logical_and(hi > minHeight, hi < maxHeight)) - + hCorr = hi[ind_hCorr][:len(ind_h)] heights[ind_h] = hCorr - + #Setting Error #Number 13: Height unresolvable echo: not valid height within 70 to 110 km - #Number 14: Height ambiguous echo: more than one possible height within 70 to 110 km + #Number 14: Height ambiguous echo: more than one possible height within 70 to 110 km indError = numpy.where(numpy.logical_or(error == 13, error == 14))[0] error[indError] = 0 - indInvalid2 = numpy.where(numpy.logical_and(h_bool > 1, error == 0))[0] + indInvalid2 = numpy.where(numpy.logical_and(h_bool > 1, error == 0))[0] error[indInvalid2] = 14 indInvalid1 = numpy.where(numpy.logical_and(h_bool == 0, error == 0))[0] - error[indInvalid1] = 13 - + error[indInvalid1] = 13 + return heights, error - + def getPhasePairs(self, channelPositions): chanPos = numpy.array(channelPositions) - listOper = list(itertools.combinations(range(5),2)) - + listOper = list(itertools.combinations(list(range(5)),2)) + distances = numpy.zeros(4) axisX = [] axisY = [] @@ -3866,15 +3823,15 @@ class SMOperations(): distY = numpy.zeros(3) ix = 0 iy = 0 - + pairX = numpy.zeros((2,2)) pairY = numpy.zeros((2,2)) - + for i in range(len(listOper)): pairi = listOper[i] - + posDif = numpy.abs(chanPos[pairi[0],:] - chanPos[pairi[1],:]) - + if posDif[0] == 0: axisY.append(pairi) distY[iy] = posDif[1] @@ -3883,7 +3840,7 @@ class SMOperations(): axisX.append(pairi) distX[ix] = posDif[0] ix += 1 - + for i in range(2): if i==0: dist0 = distX @@ -3891,7 +3848,7 @@ class SMOperations(): else: dist0 = distY axis0 = axisY - + side = numpy.argsort(dist0)[:-1] axis0 = numpy.array(axis0)[side,:] chanC = int(numpy.intersect1d(axis0[0,:], axis0[1,:])[0]) @@ -3899,7 +3856,7 @@ class SMOperations(): side = axis1[axis1 != chanC] diff1 = chanPos[chanC,i] - chanPos[side[0],i] diff2 = chanPos[chanC,i] - chanPos[side[1],i] - if diff1<0: + if diff1<0: chan2 = side[0] d2 = numpy.abs(diff1) chan1 = side[1] @@ -3909,7 +3866,7 @@ class SMOperations(): d2 = numpy.abs(diff2) chan1 = side[0] d1 = numpy.abs(diff1) - + if i==0: chanCX = chanC chan1X = chan1 @@ -3921,10 +3878,10 @@ class SMOperations(): chan2Y = chan2 distances[2:4] = numpy.array([d1,d2]) # axisXsides = numpy.reshape(axisX[ix,:],4) -# +# # channelCentX = int(numpy.intersect1d(pairX[0,:], pairX[1,:])[0]) # channelCentY = int(numpy.intersect1d(pairY[0,:], pairY[1,:])[0]) -# +# # ind25X = numpy.where(pairX[0,:] != channelCentX)[0][0] # ind20X = numpy.where(pairX[1,:] != channelCentX)[0][0] # channel25X = int(pairX[0,ind25X]) @@ -3933,59 +3890,59 @@ class SMOperations(): # ind20Y = numpy.where(pairY[1,:] != channelCentY)[0][0] # channel25Y = int(pairY[0,ind25Y]) # channel20Y = int(pairY[1,ind20Y]) - + # pairslist = [(channelCentX, channel25X),(channelCentX, channel20X),(channelCentY,channel25Y),(channelCentY, channel20Y)] - pairslist = [(chanCX, chan1X),(chanCX, chan2X),(chanCY,chan1Y),(chanCY, chan2Y)] - + pairslist = [(chanCX, chan1X),(chanCX, chan2X),(chanCY,chan1Y),(chanCY, chan2Y)] + return pairslist, distances # def __getAOA(self, phases, pairsList, error, AOAthresh, azimuth): -# +# # arrayAOA = numpy.zeros((phases.shape[0],3)) # cosdir0, cosdir = self.__getDirectionCosines(phases, pairsList) -# +# # arrayAOA[:,:2] = self.__calculateAOA(cosdir, azimuth) # cosDirError = numpy.sum(numpy.abs(cosdir0 - cosdir), axis = 1) # arrayAOA[:,2] = cosDirError -# +# # azimuthAngle = arrayAOA[:,0] # zenithAngle = arrayAOA[:,1] -# +# # #Setting Error # #Number 3: AOA not fesible # indInvalid = numpy.where(numpy.logical_and((numpy.logical_or(numpy.isnan(zenithAngle), numpy.isnan(azimuthAngle))),error == 0))[0] -# error[indInvalid] = 3 +# error[indInvalid] = 3 # #Number 4: Large difference in AOAs obtained from different antenna baselines # indInvalid = numpy.where(numpy.logical_and(cosDirError > AOAthresh,error == 0))[0] -# error[indInvalid] = 4 +# error[indInvalid] = 4 # return arrayAOA, error -# +# # def __getDirectionCosines(self, arrayPhase, pairsList): -# +# # #Initializing some variables # ang_aux = numpy.array([-8,-7,-6,-5,-4,-3,-2,-1,0,1,2,3,4,5,6,7,8])*2*numpy.pi # ang_aux = ang_aux.reshape(1,ang_aux.size) -# +# # cosdir = numpy.zeros((arrayPhase.shape[0],2)) # cosdir0 = numpy.zeros((arrayPhase.shape[0],2)) -# -# +# +# # for i in range(2): # #First Estimation # phi0_aux = arrayPhase[:,pairsList[i][0]] + arrayPhase[:,pairsList[i][1]] # #Dealias # indcsi = numpy.where(phi0_aux > numpy.pi) -# phi0_aux[indcsi] -= 2*numpy.pi +# phi0_aux[indcsi] -= 2*numpy.pi # indcsi = numpy.where(phi0_aux < -numpy.pi) -# phi0_aux[indcsi] += 2*numpy.pi +# phi0_aux[indcsi] += 2*numpy.pi # #Direction Cosine 0 # cosdir0[:,i] = -(phi0_aux)/(2*numpy.pi*0.5) -# +# # #Most-Accurate Second Estimation # phi1_aux = arrayPhase[:,pairsList[i][0]] - arrayPhase[:,pairsList[i][1]] # phi1_aux = phi1_aux.reshape(phi1_aux.size,1) # #Direction Cosine 1 # cosdir1 = -(phi1_aux + ang_aux)/(2*numpy.pi*4.5) -# +# # #Searching the correct Direction Cosine # cosdir0_aux = cosdir0[:,i] # cosdir0_aux = cosdir0_aux.reshape(cosdir0_aux.size,1) @@ -3994,51 +3951,50 @@ class SMOperations(): # indcos = cosDiff.argmin(axis = 1) # #Saving Value obtained # cosdir[:,i] = cosdir1[numpy.arange(len(indcos)),indcos] -# +# # return cosdir0, cosdir -# +# # def __calculateAOA(self, cosdir, azimuth): # cosdirX = cosdir[:,0] # cosdirY = cosdir[:,1] -# +# # zenithAngle = numpy.arccos(numpy.sqrt(1 - cosdirX**2 - cosdirY**2))*180/numpy.pi # azimuthAngle = numpy.arctan2(cosdirX,cosdirY)*180/numpy.pi + azimuth #0 deg north, 90 deg east # angles = numpy.vstack((azimuthAngle, zenithAngle)).transpose() -# +# # return angles -# +# # def __getHeights(self, Ranges, zenith, error, minHeight, maxHeight): -# +# # Ramb = 375 #Ramb = c/(2*PRF) # Re = 6371 #Earth Radius # heights = numpy.zeros(Ranges.shape) -# +# # R_aux = numpy.array([0,1,2])*Ramb # R_aux = R_aux.reshape(1,R_aux.size) -# +# # Ranges = Ranges.reshape(Ranges.size,1) -# +# # Ri = Ranges + R_aux # hi = numpy.sqrt(Re**2 + Ri**2 + (2*Re*numpy.cos(zenith*numpy.pi/180)*Ri.transpose()).transpose()) - Re -# +# # #Check if there is a height between 70 and 110 km # h_bool = numpy.sum(numpy.logical_and(hi > minHeight, hi < maxHeight), axis = 1) # ind_h = numpy.where(h_bool == 1)[0] -# +# # hCorr = hi[ind_h, :] # ind_hCorr = numpy.where(numpy.logical_and(hi > minHeight, hi < maxHeight)) -# -# hCorr = hi[ind_hCorr] +# +# hCorr = hi[ind_hCorr] # heights[ind_h] = hCorr -# +# # #Setting Error # #Number 13: Height unresolvable echo: not valid height within 70 to 110 km -# #Number 14: Height ambiguous echo: more than one possible height within 70 to 110 km -# -# indInvalid2 = numpy.where(numpy.logical_and(h_bool > 1, error == 0))[0] +# #Number 14: Height ambiguous echo: more than one possible height within 70 to 110 km +# +# indInvalid2 = numpy.where(numpy.logical_and(h_bool > 1, error == 0))[0] # error[indInvalid2] = 14 # indInvalid1 = numpy.where(numpy.logical_and(h_bool == 0, error == 0))[0] -# error[indInvalid1] = 13 -# -# return heights, error - \ No newline at end of file +# error[indInvalid1] = 13 +# +# return heights, error diff --git a/schainpy/model/proc/jroproc_spectra.py b/schainpy/model/proc/jroproc_spectra.py index 3371b55..719558f 100644 --- a/schainpy/model/proc/jroproc_spectra.py +++ b/schainpy/model/proc/jroproc_spectra.py @@ -1,17 +1,20 @@ +import time import itertools import numpy -from jroproc_base import ProcessingUnit, Operation +from schainpy.model.proc.jroproc_base import ProcessingUnit, MPDecorator, Operation from schainpy.model.data.jrodata import Spectra from schainpy.model.data.jrodata import hildebrand_sekhon +from schainpy.utils import log class SpectraProc(ProcessingUnit): - def __init__(self, **kwargs): - ProcessingUnit.__init__(self, **kwargs) + def __init__(self): + + ProcessingUnit.__init__(self) self.buffer = None self.firstdatatime = None @@ -19,6 +22,7 @@ class SpectraProc(ProcessingUnit): self.dataOut = Spectra() self.id_min = None self.id_max = None + self.setupReq = False #Agregar a todas las unidades de proc def __updateSpecFromVoltage(self): @@ -99,11 +103,11 @@ class SpectraProc(ProcessingUnit): (self.dataOut.nPairs, self.dataOut.nFFTPoints, self.dataOut.nHeights), dtype='complex') for pair in self.dataOut.pairsList: if pair[0] not in self.dataOut.channelList: - raise ValueError, "Error getting CrossSpectra: pair 0 of %s is not in channelList = %s" % ( - str(pair), str(self.dataOut.channelList)) + raise ValueError("Error getting CrossSpectra: pair 0 of %s is not in channelList = %s" % ( + str(pair), str(self.dataOut.channelList))) if pair[1] not in self.dataOut.channelList: - raise ValueError, "Error getting CrossSpectra: pair 1 of %s is not in channelList = %s" % ( - str(pair), str(self.dataOut.channelList)) + raise ValueError("Error getting CrossSpectra: pair 1 of %s is not in channelList = %s" % ( + str(pair), str(self.dataOut.channelList))) cspc[pairIndex, :, :] = fft_volt[pair[0], :, :] * \ numpy.conjugate(fft_volt[pair[1], :, :]) @@ -114,18 +118,12 @@ class SpectraProc(ProcessingUnit): self.dataOut.data_cspc = cspc self.dataOut.data_dc = dc self.dataOut.blockSize = blocksize - self.dataOut.flagShiftFFT = True + self.dataOut.flagShiftFFT = False def run(self, nProfiles=None, nFFTPoints=None, pairsList=[], ippFactor=None, shift_fft=False): - - self.dataOut.flagNoData = True - + if self.dataIn.type == "Spectra": self.dataOut.copy(self.dataIn) - # if not pairsList: - # pairsList = itertools.combinations(self.dataOut.channelList, 2) - # if self.dataOut.data_cspc is not None: - # self.__selectPairs(pairsList) if shift_fft: #desplaza a la derecha en el eje 2 determinadas posiciones shift = int(self.dataOut.nFFTPoints/2) @@ -134,13 +132,15 @@ class SpectraProc(ProcessingUnit): if self.dataOut.data_cspc is not None: #desplaza a la derecha en el eje 2 determinadas posiciones self.dataOut.data_cspc = numpy.roll(self.dataOut.data_cspc, shift, axis=1) - + return True if self.dataIn.type == "Voltage": + self.dataOut.flagNoData = True + if nFFTPoints == None: - raise ValueError, "This SpectraProc.run() need nFFTPoints input variable" + raise ValueError("This SpectraProc.run() need nFFTPoints input variable") if nProfiles == None: nProfiles = nFFTPoints @@ -160,9 +160,7 @@ class SpectraProc(ProcessingUnit): dtype='complex') if self.dataIn.flagDataAsBlock: - # data dimension: [nChannels, nProfiles, nSamples] nVoltProfiles = self.dataIn.data.shape[1] - # nVoltProfiles = self.dataIn.nProfiles if nVoltProfiles == nProfiles: self.buffer = self.dataIn.data.copy() @@ -180,8 +178,8 @@ class SpectraProc(ProcessingUnit): self.id_min += nVoltProfiles self.id_max += nVoltProfiles else: - raise ValueError, "The type object %s has %d profiles, it should just has %d profiles" % ( - self.dataIn.type, self.dataIn.data.shape[1], nProfiles) + raise ValueError("The type object %s has %d profiles, it should just has %d profiles" % ( + self.dataIn.type, self.dataIn.data.shape[1], nProfiles)) self.dataOut.flagNoData = True return 0 else: @@ -201,8 +199,8 @@ class SpectraProc(ProcessingUnit): return True - raise ValueError, "The type of input object '%s' is not valid" % ( - self.dataIn.type) + raise ValueError("The type of input object '%s' is not valid" % ( + self.dataIn.type)) def __selectPairs(self, pairsList): @@ -222,113 +220,28 @@ class SpectraProc(ProcessingUnit): self.dataOut.pairsList = pairs return - - def __selectPairsByChannel(self, channelList=None): - - if channelList == None: - return - - pairsIndexListSelected = [] - for pairIndex in self.dataOut.pairsIndexList: - # First pair - if self.dataOut.pairsList[pairIndex][0] not in channelList: - continue - # Second pair - if self.dataOut.pairsList[pairIndex][1] not in channelList: - continue - - pairsIndexListSelected.append(pairIndex) - - if not pairsIndexListSelected: - self.dataOut.data_cspc = None - self.dataOut.pairsList = [] - return - - self.dataOut.data_cspc = self.dataOut.data_cspc[pairsIndexListSelected] - self.dataOut.pairsList = [self.dataOut.pairsList[i] - for i in pairsIndexListSelected] - - return - - def selectChannels(self, channelList): - - channelIndexList = [] - - for channel in channelList: - if channel not in self.dataOut.channelList: - raise ValueError, "Error selecting channels, Channel %d is not valid.\nAvailable channels = %s" % ( - channel, str(self.dataOut.channelList)) - - index = self.dataOut.channelList.index(channel) - channelIndexList.append(index) - - self.selectChannelsByIndex(channelIndexList) - - def selectChannelsByIndex(self, channelIndexList): + + def selectFFTs(self, minFFT, maxFFT ): """ - Selecciona un bloque de datos en base a canales segun el channelIndexList - - Input: - channelIndexList : lista sencilla de canales a seleccionar por ej. [2,3,7] - - Affected: - self.dataOut.data_spc - self.dataOut.channelIndexList - self.dataOut.nChannels - - Return: - None + Selecciona un bloque de datos en base a un grupo de valores de puntos FFTs segun el rango + minFFT<= FFT <= maxFFT """ - - for channelIndex in channelIndexList: - if channelIndex not in self.dataOut.channelIndexList: - raise ValueError, "Error selecting channels: The value %d in channelIndexList is not valid.\nAvailable channel indexes = " % ( - channelIndex, self.dataOut.channelIndexList) - - data_spc = self.dataOut.data_spc[channelIndexList, :] - data_dc = self.dataOut.data_dc[channelIndexList, :] - - self.dataOut.data_spc = data_spc - self.dataOut.data_dc = data_dc - - # self.dataOut.channelList = [self.dataOut.channelList[i] for i in channelIndexList] - self.dataOut.channelList = range(len(channelIndexList)) - self.__selectPairsByChannel(channelIndexList) - return 1 - - def selectHeights(self, minHei, maxHei): - """ - Selecciona un bloque de datos en base a un grupo de valores de alturas segun el rango - minHei <= height <= maxHei - - Input: - minHei : valor minimo de altura a considerar - maxHei : valor maximo de altura a considerar - - Affected: - Indirectamente son cambiados varios valores a travez del metodo selectHeightsByIndex - - Return: - 1 si el metodo se ejecuto con exito caso contrario devuelve 0 - """ + if (minFFT > maxFFT): + raise ValueError("Error selecting heights: Height range (%d,%d) is not valid" % (minFFT, maxFFT)) - if (minHei > maxHei): - raise ValueError, "Error selecting heights: Height range (%d,%d) is not valid" % ( - minHei, maxHei) + if (minFFT < self.dataOut.getFreqRange()[0]): + minFFT = self.dataOut.getFreqRange()[0] - if (minHei < self.dataOut.heightList[0]): - minHei = self.dataOut.heightList[0] - - if (maxHei > self.dataOut.heightList[-1]): - maxHei = self.dataOut.heightList[-1] + if (maxFFT > self.dataOut.getFreqRange()[-1]): + maxFFT = self.dataOut.getFreqRange()[-1] minIndex = 0 maxIndex = 0 - heights = self.dataOut.heightList + FFTs = self.dataOut.getFreqRange() - inda = numpy.where(heights >= minHei) - indb = numpy.where(heights <= maxHei) + inda = numpy.where(FFTs >= minFFT) + indb = numpy.where(FFTs <= maxFFT) try: minIndex = inda[0][0] @@ -338,12 +251,12 @@ class SpectraProc(ProcessingUnit): try: maxIndex = indb[0][-1] except: - maxIndex = len(heights) + maxIndex = len(FFTs) - self.selectHeightsByIndex(minIndex, maxIndex) + self.selectFFTsByIndex(minIndex, maxIndex) return 1 - + def getBeaconSignal(self, tauindex=0, channelindex=0, hei_ref=None): newheis = numpy.where( self.dataOut.heightList > self.dataOut.radarControllerHeaderObj.Taus[tauindex]) @@ -386,52 +299,132 @@ class SpectraProc(ProcessingUnit): return 1 - def selectHeightsByIndex(self, minIndex, maxIndex): + def selectFFTsByIndex(self, minIndex, maxIndex): """ - Selecciona un bloque de datos en base a un grupo indices de alturas segun el rango - minIndex <= index <= maxIndex - - Input: - minIndex : valor de indice minimo de altura a considerar - maxIndex : valor de indice maximo de altura a considerar - - Affected: - self.dataOut.data_spc - self.dataOut.data_cspc - self.dataOut.data_dc - self.dataOut.heightList - - Return: - 1 si el metodo se ejecuto con exito caso contrario devuelve 0 + """ if (minIndex < 0) or (minIndex > maxIndex): - raise ValueError, "Error selecting heights: Index range (%d,%d) is not valid" % ( - minIndex, maxIndex) + raise ValueError("Error selecting heights: Index range (%d,%d) is not valid" % (minIndex, maxIndex)) - if (maxIndex >= self.dataOut.nHeights): - maxIndex = self.dataOut.nHeights - 1 + if (maxIndex >= self.dataOut.nProfiles): + maxIndex = self.dataOut.nProfiles-1 - # Spectra - data_spc = self.dataOut.data_spc[:, :, minIndex:maxIndex + 1] + #Spectra + data_spc = self.dataOut.data_spc[:,minIndex:maxIndex+1,:] data_cspc = None if self.dataOut.data_cspc is not None: - data_cspc = self.dataOut.data_cspc[:, :, minIndex:maxIndex + 1] + data_cspc = self.dataOut.data_cspc[:,minIndex:maxIndex+1,:] data_dc = None if self.dataOut.data_dc is not None: - data_dc = self.dataOut.data_dc[:, minIndex:maxIndex + 1] + data_dc = self.dataOut.data_dc[minIndex:maxIndex+1,:] self.dataOut.data_spc = data_spc self.dataOut.data_cspc = data_cspc self.dataOut.data_dc = data_dc + + self.dataOut.ippSeconds = self.dataOut.ippSeconds*(self.dataOut.nFFTPoints / numpy.shape(data_cspc)[1]) + self.dataOut.nFFTPoints = numpy.shape(data_cspc)[1] + self.dataOut.profilesPerBlock = numpy.shape(data_cspc)[1] + + return 1 + + def getNoise(self, minHei=None, maxHei=None, minVel=None, maxVel=None): + # validacion de rango + if minHei == None: + minHei = self.dataOut.heightList[0] + + if maxHei == None: + maxHei = self.dataOut.heightList[-1] + + if (minHei < self.dataOut.heightList[0]) or (minHei > maxHei): + print('minHei: %.2f is out of the heights range' % (minHei)) + print('minHei is setting to %.2f' % (self.dataOut.heightList[0])) + minHei = self.dataOut.heightList[0] + + if (maxHei > self.dataOut.heightList[-1]) or (maxHei < minHei): + print('maxHei: %.2f is out of the heights range' % (maxHei)) + print('maxHei is setting to %.2f' % (self.dataOut.heightList[-1])) + maxHei = self.dataOut.heightList[-1] + + # validacion de velocidades + velrange = self.dataOut.getVelRange(1) + + if minVel == None: + minVel = velrange[0] + + if maxVel == None: + maxVel = velrange[-1] + + if (minVel < velrange[0]) or (minVel > maxVel): + print('minVel: %.2f is out of the velocity range' % (minVel)) + print('minVel is setting to %.2f' % (velrange[0])) + minVel = velrange[0] + + if (maxVel > velrange[-1]) or (maxVel < minVel): + print('maxVel: %.2f is out of the velocity range' % (maxVel)) + print('maxVel is setting to %.2f' % (velrange[-1])) + maxVel = velrange[-1] + + # seleccion de indices para rango + minIndex = 0 + maxIndex = 0 + heights = self.dataOut.heightList + + inda = numpy.where(heights >= minHei) + indb = numpy.where(heights <= maxHei) + + try: + minIndex = inda[0][0] + except: + minIndex = 0 + + try: + maxIndex = indb[0][-1] + except: + maxIndex = len(heights) + + if (minIndex < 0) or (minIndex > maxIndex): + raise ValueError("some value in (%d,%d) is not valid" % ( + minIndex, maxIndex)) + + if (maxIndex >= self.dataOut.nHeights): + maxIndex = self.dataOut.nHeights - 1 + + # seleccion de indices para velocidades + indminvel = numpy.where(velrange >= minVel) + indmaxvel = numpy.where(velrange <= maxVel) + try: + minIndexVel = indminvel[0][0] + except: + minIndexVel = 0 + + try: + maxIndexVel = indmaxvel[0][-1] + except: + maxIndexVel = len(velrange) - self.dataOut.heightList = self.dataOut.heightList[minIndex:maxIndex + 1] + # seleccion del espectro + data_spc = self.dataOut.data_spc[:, + minIndexVel:maxIndexVel + 1, minIndex:maxIndex + 1] + # estimacion de ruido + noise = numpy.zeros(self.dataOut.nChannels) + + for channel in range(self.dataOut.nChannels): + daux = data_spc[channel, :, :] + sortdata = numpy.sort(daux, axis=None) + noise[channel] = hildebrand_sekhon(sortdata, self.dataOut.nIncohInt) + + self.dataOut.noise_estimation = noise.copy() return 1 - def removeDC(self, mode=2): +class removeDC(Operation): + + def run(self, dataOut, mode=2): + self.dataOut = dataOut jspectra = self.dataOut.data_spc jcspectra = self.dataOut.data_cspc @@ -444,11 +437,12 @@ class SpectraProc(ProcessingUnit): else: jcspectraExist = False - freq_dc = jspectra.shape[1] / 2 + freq_dc = int(jspectra.shape[1] / 2) ind_vel = numpy.array([-2, -1, 1, 2]) + freq_dc + ind_vel = ind_vel.astype(int) if ind_vel[0] < 0: - ind_vel[range(0, 1)] = ind_vel[range(0, 1)] + self.num_prof + ind_vel[list(range(0, 1))] = ind_vel[list(range(0, 1))] + self.num_prof if mode == 1: jspectra[:, freq_dc, :] = ( @@ -464,12 +458,12 @@ class SpectraProc(ProcessingUnit): xx = numpy.zeros([4, 4]) for fil in range(4): - xx[fil, :] = vel[fil]**numpy.asarray(range(4)) + xx[fil, :] = vel[fil]**numpy.asarray(list(range(4))) xx_inv = numpy.linalg.inv(xx) xx_aux = xx_inv[0, :] - for ich in range(num_chan): + for ich in range(num_chan): yy = jspectra[ich, ind_vel, :] jspectra[ich, freq_dc, :] = numpy.dot(xx_aux, yy) @@ -488,9 +482,34 @@ class SpectraProc(ProcessingUnit): self.dataOut.data_spc = jspectra self.dataOut.data_cspc = jcspectra - return 1 + return self.dataOut - def removeInterference(self, interf=2, hei_interf=None, nhei_interf=None, offhei_interf=None): +class removeInterference(Operation): + + def removeInterference2(self): + + cspc = self.dataOut.data_cspc + spc = self.dataOut.data_spc + Heights = numpy.arange(cspc.shape[2]) + realCspc = numpy.abs(cspc) + + for i in range(cspc.shape[0]): + LinePower= numpy.sum(realCspc[i], axis=0) + Threshold = numpy.amax(LinePower)-numpy.sort(LinePower)[len(Heights)-int(len(Heights)*0.1)] + SelectedHeights = Heights[ numpy.where( LinePower < Threshold ) ] + InterferenceSum = numpy.sum( realCspc[i,:,SelectedHeights], axis=0 ) + InterferenceThresholdMin = numpy.sort(InterferenceSum)[int(len(InterferenceSum)*0.98)] + InterferenceThresholdMax = numpy.sort(InterferenceSum)[int(len(InterferenceSum)*0.99)] + + + InterferenceRange = numpy.where( ([InterferenceSum > InterferenceThresholdMin]))# , InterferenceSum < InterferenceThresholdMax]) ) + #InterferenceRange = numpy.where( ([InterferenceRange < InterferenceThresholdMax])) + if len(InterferenceRange) maxHei): - print 'minHei: %.2f is out of the heights range' % (minHei) - print 'minHei is setting to %.2f' % (self.dataOut.heightList[0]) - minHei = self.dataOut.heightList[0] - - if (maxHei > self.dataOut.heightList[-1]) or (maxHei < minHei): - print 'maxHei: %.2f is out of the heights range' % (maxHei) - print 'maxHei is setting to %.2f' % (self.dataOut.heightList[-1]) - maxHei = self.dataOut.heightList[-1] - - # validacion de velocidades - velrange = self.dataOut.getVelRange(1) - - if minVel == None: - minVel = velrange[0] - - if maxVel == None: - maxVel = velrange[-1] - - if (minVel < velrange[0]) or (minVel > maxVel): - print 'minVel: %.2f is out of the velocity range' % (minVel) - print 'minVel is setting to %.2f' % (velrange[0]) - minVel = velrange[0] - - if (maxVel > velrange[-1]) or (maxVel < minVel): - print 'maxVel: %.2f is out of the velocity range' % (maxVel) - print 'maxVel is setting to %.2f' % (velrange[-1]) - maxVel = velrange[-1] - - # seleccion de indices para rango - minIndex = 0 - maxIndex = 0 - heights = self.dataOut.heightList - - inda = numpy.where(heights >= minHei) - indb = numpy.where(heights <= maxHei) - - try: - minIndex = inda[0][0] - except: - minIndex = 0 - - try: - maxIndex = indb[0][-1] - except: - maxIndex = len(heights) - - if (minIndex < 0) or (minIndex > maxIndex): - raise ValueError, "some value in (%d,%d) is not valid" % ( - minIndex, maxIndex) - - if (maxIndex >= self.dataOut.nHeights): - maxIndex = self.dataOut.nHeights - 1 - - # seleccion de indices para velocidades - indminvel = numpy.where(velrange >= minVel) - indmaxvel = numpy.where(velrange <= maxVel) - try: - minIndexVel = indminvel[0][0] - except: - minIndexVel = 0 - - try: - maxIndexVel = indmaxvel[0][-1] - except: - maxIndexVel = len(velrange) - - # seleccion del espectro - data_spc = self.dataOut.data_spc[:, - minIndexVel:maxIndexVel + 1, minIndex:maxIndex + 1] - # estimacion de ruido - noise = numpy.zeros(self.dataOut.nChannels) + def run(self, dataOut, interf = 2,hei_interf = None, nhei_interf = None, offhei_interf = None, mode=1): - for channel in range(self.dataOut.nChannels): - daux = data_spc[channel, :, :] - noise[channel] = hildebrand_sekhon(daux, self.dataOut.nIncohInt) + self.dataOut = dataOut - self.dataOut.noise_estimation = noise.copy() + if mode == 1: + self.removeInterference(interf = 2,hei_interf = None, nhei_interf = None, offhei_interf = None) + elif mode == 2: + self.removeInterference2() - return 1 + return self.dataOut class IncohInt(Operation): @@ -790,10 +724,9 @@ class IncohInt(Operation): n = None - def __init__(self, **kwargs): + def __init__(self): - Operation.__init__(self, **kwargs) -# self.isConfig = False + Operation.__init__(self) def setup(self, n=None, timeInterval=None, overlapping=False): """ @@ -819,12 +752,12 @@ class IncohInt(Operation): self.__byTime = False if n is None and timeInterval is None: - raise ValueError, "n or timeInterval should be specified ..." + raise ValueError("n or timeInterval should be specified ...") if n is not None: self.n = int(n) else: - # if (type(timeInterval)!=integer) -> change this line + self.__integrationtime = int(timeInterval) self.n = None self.__byTime = True @@ -924,8 +857,8 @@ class IncohInt(Operation): def run(self, dataOut, n=None, timeInterval=None, overlapping=False): if n == 1: - return - + return dataOut + dataOut.flagNoData = True if not self.isConfig: @@ -941,8 +874,9 @@ class IncohInt(Operation): dataOut.data_spc = avgdata_spc dataOut.data_cspc = avgdata_cspc - dataOut.data_dc = avgdata_dc - + dataOut.data_dc = avgdata_dc dataOut.nIncohInt *= self.n dataOut.utctime = avgdatatime dataOut.flagNoData = False + + return dataOut \ No newline at end of file diff --git a/schainpy/model/proc/jroproc_spectra_acf.py b/schainpy/model/proc/jroproc_spectra_acf.py index 4d316d2..709922d 100644 --- a/schainpy/model/proc/jroproc_spectra_acf.py +++ b/schainpy/model/proc/jroproc_spectra_acf.py @@ -1,6 +1,6 @@ import numpy -from jroproc_base import ProcessingUnit, Operation +from .jroproc_base import ProcessingUnit, Operation from schainpy.model.data.jrodata import Spectra from schainpy.model.data.jrodata import hildebrand_sekhon @@ -119,9 +119,9 @@ class SpectraAFCProc(ProcessingUnit): cspc = numpy.zeros((self.dataOut.nPairs, self.dataOut.nFFTPoints, self.dataOut.nHeights), dtype='complex') for pair in self.dataOut.pairsList: if pair[0] not in self.dataOut.channelList: - raise ValueError, "Error getting CrossSpectra: pair 0 of %s is not in channelList = %s" %(str(pair), str(self.dataOut.channelList)) + raise ValueError("Error getting CrossSpectra: pair 0 of %s is not in channelList = %s" %(str(pair), str(self.dataOut.channelList))) if pair[1] not in self.dataOut.channelList: - raise ValueError, "Error getting CrossSpectra: pair 1 of %s is not in channelList = %s" %(str(pair), str(self.dataOut.channelList)) + raise ValueError("Error getting CrossSpectra: pair 1 of %s is not in channelList = %s" %(str(pair), str(self.dataOut.channelList))) chan_index0 = self.dataOut.channelList.index(pair[0]) chan_index1 = self.dataOut.channelList.index(pair[1]) @@ -148,7 +148,7 @@ class SpectraAFCProc(ProcessingUnit): if self.dataIn.type == "Voltage": if nFFTPoints == None: - raise ValueError, "This SpectraProc.run() need nFFTPoints input variable" + raise ValueError("This SpectraProc.run() need nFFTPoints input variable") if nProfiles == None: nProfiles = nFFTPoints @@ -172,7 +172,7 @@ class SpectraAFCProc(ProcessingUnit): # self.profIndex += 1 else: - raise ValueError, "" + raise ValueError("") self.firstdatatime = self.dataIn.utctime @@ -186,7 +186,7 @@ class SpectraAFCProc(ProcessingUnit): return True - raise ValueError, "The type of input object '%s' is not valid"%(self.dataIn.type) + raise ValueError("The type of input object '%s' is not valid"%(self.dataIn.type)) def __selectPairs(self, pairsList): @@ -246,7 +246,7 @@ class SpectraAFCProc(ProcessingUnit): for channel in channelList: if channel not in self.dataOut.channelList: - raise ValueError, "Error selecting channels, Channel %d is not valid.\nAvailable channels = %s" %(channel, str(self.dataOut.channelList)) + raise ValueError("Error selecting channels, Channel %d is not valid.\nAvailable channels = %s" %(channel, str(self.dataOut.channelList))) index = self.dataOut.channelList.index(channel) channelIndexList.append(index) @@ -271,7 +271,7 @@ class SpectraAFCProc(ProcessingUnit): for channelIndex in channelIndexList: if channelIndex not in self.dataOut.channelIndexList: - raise ValueError, "Error selecting channels: The value %d in channelIndexList is not valid.\nAvailable channel indexes = " %(channelIndex, self.dataOut.channelIndexList) + raise ValueError("Error selecting channels: The value %d in channelIndexList is not valid.\nAvailable channel indexes = " %(channelIndex, self.dataOut.channelIndexList)) # nChannels = len(channelIndexList) @@ -305,7 +305,7 @@ class SpectraAFCProc(ProcessingUnit): """ if (minHei > maxHei): - raise ValueError, "Error selecting heights: Height range (%d,%d) is not valid" % (minHei, maxHei) + raise ValueError("Error selecting heights: Height range (%d,%d) is not valid" % (minHei, maxHei)) if (minHei < self.dataOut.heightList[0]): minHei = self.dataOut.heightList[0] @@ -394,7 +394,7 @@ class SpectraAFCProc(ProcessingUnit): """ if (minIndex < 0) or (minIndex > maxIndex): - raise ValueError, "Error selecting heights: Index range (%d,%d) is not valid" % (minIndex, maxIndex) + raise ValueError("Error selecting heights: Index range (%d,%d) is not valid" % (minIndex, maxIndex)) if (maxIndex >= self.dataOut.nHeights): maxIndex = self.dataOut.nHeights-1 @@ -435,7 +435,7 @@ class SpectraAFCProc(ProcessingUnit): ind_vel = numpy.array([-2,-1,1,2]) + freq_dc if ind_vel[0]<0: - ind_vel[range(0,1)] = ind_vel[range(0,1)] + self.num_prof + ind_vel[list(range(0,1))] = ind_vel[list(range(0,1))] + self.num_prof if mode == 1: jspectra[:,freq_dc,:] = (jspectra[:,ind_vel[1],:] + jspectra[:,ind_vel[2],:])/2 #CORRECCION @@ -449,7 +449,7 @@ class SpectraAFCProc(ProcessingUnit): xx = numpy.zeros([4,4]) for fil in range(4): - xx[fil,:] = vel[fil]**numpy.asarray(range(4)) + xx[fil,:] = vel[fil]**numpy.asarray(list(range(4))) xx_inv = numpy.linalg.inv(xx) xx_aux = xx_inv[0,:] @@ -489,7 +489,7 @@ class SpectraAFCProc(ProcessingUnit): #hei_interf if hei_interf is None: count_hei = num_hei/2 #Como es entero no importa - hei_interf = numpy.asmatrix(range(count_hei)) + num_hei - count_hei + hei_interf = numpy.asmatrix(list(range(count_hei))) + num_hei - count_hei hei_interf = numpy.asarray(hei_interf)[0] #nhei_interf if (nhei_interf == None): @@ -501,10 +501,10 @@ class SpectraAFCProc(ProcessingUnit): if (offhei_interf == None): offhei_interf = 0 - ind_hei = range(num_hei) + ind_hei = list(range(num_hei)) # mask_prof = numpy.asarray(range(num_prof - 2)) + 1 # mask_prof[range(num_prof/2 - 1,len(mask_prof))] += 1 - mask_prof = numpy.asarray(range(num_prof)) + mask_prof = numpy.asarray(list(range(num_prof))) num_mask_prof = mask_prof.size comp_mask_prof = [0, num_prof/2] @@ -523,7 +523,7 @@ class SpectraAFCProc(ProcessingUnit): psort = power.ravel().argsort() #Se estima la interferencia promedio en los Espectros de Potencia empleando - junkspc_interf = jspectra[ich,:,hei_interf[psort[range(offhei_interf, nhei_interf + offhei_interf)]]] + junkspc_interf = jspectra[ich,:,hei_interf[psort[list(range(offhei_interf, nhei_interf + offhei_interf))]]] if noise_exist: # tmp_noise = jnoise[ich] / num_prof @@ -576,7 +576,7 @@ class SpectraAFCProc(ProcessingUnit): xx = numpy.zeros([4,4]) for id1 in range(4): - xx[:,id1] = ind[id1]**numpy.asarray(range(4)) + xx[:,id1] = ind[id1]**numpy.asarray(list(range(4))) xx_inv = numpy.linalg.inv(xx) xx = xx_inv[:,0] @@ -602,14 +602,14 @@ class SpectraAFCProc(ProcessingUnit): cspower = cspower.sum(axis = 0) cspsort = cspower.ravel().argsort() - junkcspc_interf = jcspectra[ip,:,hei_interf[cspsort[range(offhei_interf, nhei_interf + offhei_interf)]]] + junkcspc_interf = jcspectra[ip,:,hei_interf[cspsort[list(range(offhei_interf, nhei_interf + offhei_interf))]]] junkcspc_interf = junkcspc_interf.transpose() jcspc_interf = junkcspc_interf.sum(axis = 1)/nhei_interf ind = numpy.abs(jcspc_interf[mask_prof]).ravel().argsort() - median_real = numpy.median(numpy.real(junkcspc_interf[mask_prof[ind[range(3*num_prof/4)]],:])) - median_imag = numpy.median(numpy.imag(junkcspc_interf[mask_prof[ind[range(3*num_prof/4)]],:])) + median_real = numpy.median(numpy.real(junkcspc_interf[mask_prof[ind[list(range(3*num_prof/4))]],:])) + median_imag = numpy.median(numpy.imag(junkcspc_interf[mask_prof[ind[list(range(3*num_prof/4))]],:])) junkcspc_interf[comp_mask_prof,:] = numpy.complex(median_real, median_imag) for iprof in range(num_prof): @@ -626,7 +626,7 @@ class SpectraAFCProc(ProcessingUnit): xx = numpy.zeros([4,4]) for id1 in range(4): - xx[:,id1] = ind[id1]**numpy.asarray(range(4)) + xx[:,id1] = ind[id1]**numpy.asarray(list(range(4))) xx_inv = numpy.linalg.inv(xx) xx = xx_inv[:,0] @@ -657,13 +657,13 @@ class SpectraAFCProc(ProcessingUnit): maxHei = self.dataOut.heightList[-1] if (minHei < self.dataOut.heightList[0]) or (minHei > maxHei): - print 'minHei: %.2f is out of the heights range'%(minHei) - print 'minHei is setting to %.2f'%(self.dataOut.heightList[0]) + print('minHei: %.2f is out of the heights range'%(minHei)) + print('minHei is setting to %.2f'%(self.dataOut.heightList[0])) minHei = self.dataOut.heightList[0] if (maxHei > self.dataOut.heightList[-1]) or (maxHei < minHei): - print 'maxHei: %.2f is out of the heights range'%(maxHei) - print 'maxHei is setting to %.2f'%(self.dataOut.heightList[-1]) + print('maxHei: %.2f is out of the heights range'%(maxHei)) + print('maxHei is setting to %.2f'%(self.dataOut.heightList[-1])) maxHei = self.dataOut.heightList[-1] # validacion de velocidades @@ -676,13 +676,13 @@ class SpectraAFCProc(ProcessingUnit): maxVel = velrange[-1] if (minVel < velrange[0]) or (minVel > maxVel): - print 'minVel: %.2f is out of the velocity range'%(minVel) - print 'minVel is setting to %.2f'%(velrange[0]) + print('minVel: %.2f is out of the velocity range'%(minVel)) + print('minVel is setting to %.2f'%(velrange[0])) minVel = velrange[0] if (maxVel > velrange[-1]) or (maxVel < minVel): - print 'maxVel: %.2f is out of the velocity range'%(maxVel) - print 'maxVel is setting to %.2f'%(velrange[-1]) + print('maxVel: %.2f is out of the velocity range'%(maxVel)) + print('maxVel is setting to %.2f'%(velrange[-1])) maxVel = velrange[-1] # seleccion de indices para rango @@ -704,7 +704,7 @@ class SpectraAFCProc(ProcessingUnit): maxIndex = len(heights) if (minIndex < 0) or (minIndex > maxIndex): - raise ValueError, "some value in (%d,%d) is not valid" % (minIndex, maxIndex) + raise ValueError("some value in (%d,%d) is not valid" % (minIndex, maxIndex)) if (maxIndex >= self.dataOut.nHeights): maxIndex = self.dataOut.nHeights-1 @@ -733,4 +733,4 @@ class SpectraAFCProc(ProcessingUnit): self.dataOut.noise_estimation = noise.copy() - return 1 + return 1 \ No newline at end of file diff --git a/schainpy/model/proc/jroproc_spectra_lags.py b/schainpy/model/proc/jroproc_spectra_lags.py index fa64eae..90b88e5 100644 --- a/schainpy/model/proc/jroproc_spectra_lags.py +++ b/schainpy/model/proc/jroproc_spectra_lags.py @@ -1,6 +1,6 @@ import numpy -from jroproc_base import ProcessingUnit, Operation +from .jroproc_base import ProcessingUnit, Operation from schainpy.model.data.jrodata import Spectra from schainpy.model.data.jrodata import hildebrand_sekhon @@ -125,9 +125,9 @@ class SpectraLagsProc(ProcessingUnit): cspc = numpy.zeros((self.dataOut.nPairs, self.dataOut.nFFTPoints, self.dataOut.nHeights), dtype='complex') for pair in self.dataOut.pairsList: if pair[0] not in self.dataOut.channelList: - raise ValueError, "Error getting CrossSpectra: pair 0 of %s is not in channelList = %s" %(str(pair), str(self.dataOut.channelList)) + raise ValueError("Error getting CrossSpectra: pair 0 of %s is not in channelList = %s" %(str(pair), str(self.dataOut.channelList))) if pair[1] not in self.dataOut.channelList: - raise ValueError, "Error getting CrossSpectra: pair 1 of %s is not in channelList = %s" %(str(pair), str(self.dataOut.channelList)) + raise ValueError("Error getting CrossSpectra: pair 1 of %s is not in channelList = %s" %(str(pair), str(self.dataOut.channelList))) chan_index0 = self.dataOut.channelList.index(pair[0]) chan_index1 = self.dataOut.channelList.index(pair[1]) @@ -158,7 +158,7 @@ class SpectraLagsProc(ProcessingUnit): if self.dataIn.type == "Voltage": if nFFTPoints == None: - raise ValueError, "This SpectraProc.run() need nFFTPoints input variable" + raise ValueError("This SpectraProc.run() need nFFTPoints input variable") if nProfiles == None: nProfiles = nFFTPoints @@ -189,7 +189,7 @@ class SpectraLagsProc(ProcessingUnit): return True - raise ValueError, "The type of input object '%s' is not valid"%(self.dataIn.type) + raise ValueError("The type of input object '%s' is not valid"%(self.dataIn.type)) def __selectPairs(self, pairsList): @@ -249,7 +249,7 @@ class SpectraLagsProc(ProcessingUnit): for channel in channelList: if channel not in self.dataOut.channelList: - raise ValueError, "Error selecting channels, Channel %d is not valid.\nAvailable channels = %s" %(channel, str(self.dataOut.channelList)) + raise ValueError("Error selecting channels, Channel %d is not valid.\nAvailable channels = %s" %(channel, str(self.dataOut.channelList))) index = self.dataOut.channelList.index(channel) channelIndexList.append(index) @@ -274,7 +274,7 @@ class SpectraLagsProc(ProcessingUnit): for channelIndex in channelIndexList: if channelIndex not in self.dataOut.channelIndexList: - raise ValueError, "Error selecting channels: The value %d in channelIndexList is not valid.\nAvailable channel indexes = " %(channelIndex, self.dataOut.channelIndexList) + raise ValueError("Error selecting channels: The value %d in channelIndexList is not valid.\nAvailable channel indexes = " %(channelIndex, self.dataOut.channelIndexList)) # nChannels = len(channelIndexList) @@ -308,7 +308,7 @@ class SpectraLagsProc(ProcessingUnit): """ if (minHei > maxHei): - raise ValueError, "Error selecting heights: Height range (%d,%d) is not valid" % (minHei, maxHei) + raise ValueError("Error selecting heights: Height range (%d,%d) is not valid" % (minHei, maxHei)) if (minHei < self.dataOut.heightList[0]): minHei = self.dataOut.heightList[0] @@ -397,7 +397,7 @@ class SpectraLagsProc(ProcessingUnit): """ if (minIndex < 0) or (minIndex > maxIndex): - raise ValueError, "Error selecting heights: Index range (%d,%d) is not valid" % (minIndex, maxIndex) + raise ValueError("Error selecting heights: Index range (%d,%d) is not valid" % (minIndex, maxIndex)) if (maxIndex >= self.dataOut.nHeights): maxIndex = self.dataOut.nHeights-1 @@ -438,7 +438,7 @@ class SpectraLagsProc(ProcessingUnit): ind_vel = numpy.array([-2,-1,1,2]) + freq_dc if ind_vel[0]<0: - ind_vel[range(0,1)] = ind_vel[range(0,1)] + self.num_prof + ind_vel[list(range(0,1))] = ind_vel[list(range(0,1))] + self.num_prof if mode == 1: jspectra[:,freq_dc,:] = (jspectra[:,ind_vel[1],:] + jspectra[:,ind_vel[2],:])/2 #CORRECCION @@ -452,7 +452,7 @@ class SpectraLagsProc(ProcessingUnit): xx = numpy.zeros([4,4]) for fil in range(4): - xx[fil,:] = vel[fil]**numpy.asarray(range(4)) + xx[fil,:] = vel[fil]**numpy.asarray(list(range(4))) xx_inv = numpy.linalg.inv(xx) xx_aux = xx_inv[0,:] @@ -492,7 +492,7 @@ class SpectraLagsProc(ProcessingUnit): #hei_interf if hei_interf is None: count_hei = num_hei/2 #Como es entero no importa - hei_interf = numpy.asmatrix(range(count_hei)) + num_hei - count_hei + hei_interf = numpy.asmatrix(list(range(count_hei))) + num_hei - count_hei hei_interf = numpy.asarray(hei_interf)[0] #nhei_interf if (nhei_interf == None): @@ -504,10 +504,10 @@ class SpectraLagsProc(ProcessingUnit): if (offhei_interf == None): offhei_interf = 0 - ind_hei = range(num_hei) + ind_hei = list(range(num_hei)) # mask_prof = numpy.asarray(range(num_prof - 2)) + 1 # mask_prof[range(num_prof/2 - 1,len(mask_prof))] += 1 - mask_prof = numpy.asarray(range(num_prof)) + mask_prof = numpy.asarray(list(range(num_prof))) num_mask_prof = mask_prof.size comp_mask_prof = [0, num_prof/2] @@ -526,7 +526,7 @@ class SpectraLagsProc(ProcessingUnit): psort = power.ravel().argsort() #Se estima la interferencia promedio en los Espectros de Potencia empleando - junkspc_interf = jspectra[ich,:,hei_interf[psort[range(offhei_interf, nhei_interf + offhei_interf)]]] + junkspc_interf = jspectra[ich,:,hei_interf[psort[list(range(offhei_interf, nhei_interf + offhei_interf))]]] if noise_exist: # tmp_noise = jnoise[ich] / num_prof @@ -579,7 +579,7 @@ class SpectraLagsProc(ProcessingUnit): xx = numpy.zeros([4,4]) for id1 in range(4): - xx[:,id1] = ind[id1]**numpy.asarray(range(4)) + xx[:,id1] = ind[id1]**numpy.asarray(list(range(4))) xx_inv = numpy.linalg.inv(xx) xx = xx_inv[:,0] @@ -605,14 +605,14 @@ class SpectraLagsProc(ProcessingUnit): cspower = cspower.sum(axis = 0) cspsort = cspower.ravel().argsort() - junkcspc_interf = jcspectra[ip,:,hei_interf[cspsort[range(offhei_interf, nhei_interf + offhei_interf)]]] + junkcspc_interf = jcspectra[ip,:,hei_interf[cspsort[list(range(offhei_interf, nhei_interf + offhei_interf))]]] junkcspc_interf = junkcspc_interf.transpose() jcspc_interf = junkcspc_interf.sum(axis = 1)/nhei_interf ind = numpy.abs(jcspc_interf[mask_prof]).ravel().argsort() - median_real = numpy.median(numpy.real(junkcspc_interf[mask_prof[ind[range(3*num_prof/4)]],:])) - median_imag = numpy.median(numpy.imag(junkcspc_interf[mask_prof[ind[range(3*num_prof/4)]],:])) + median_real = numpy.median(numpy.real(junkcspc_interf[mask_prof[ind[list(range(3*num_prof/4))]],:])) + median_imag = numpy.median(numpy.imag(junkcspc_interf[mask_prof[ind[list(range(3*num_prof/4))]],:])) junkcspc_interf[comp_mask_prof,:] = numpy.complex(median_real, median_imag) for iprof in range(num_prof): @@ -629,7 +629,7 @@ class SpectraLagsProc(ProcessingUnit): xx = numpy.zeros([4,4]) for id1 in range(4): - xx[:,id1] = ind[id1]**numpy.asarray(range(4)) + xx[:,id1] = ind[id1]**numpy.asarray(list(range(4))) xx_inv = numpy.linalg.inv(xx) xx = xx_inv[:,0] @@ -660,13 +660,13 @@ class SpectraLagsProc(ProcessingUnit): maxHei = self.dataOut.heightList[-1] if (minHei < self.dataOut.heightList[0]) or (minHei > maxHei): - print 'minHei: %.2f is out of the heights range'%(minHei) - print 'minHei is setting to %.2f'%(self.dataOut.heightList[0]) + print('minHei: %.2f is out of the heights range'%(minHei)) + print('minHei is setting to %.2f'%(self.dataOut.heightList[0])) minHei = self.dataOut.heightList[0] if (maxHei > self.dataOut.heightList[-1]) or (maxHei < minHei): - print 'maxHei: %.2f is out of the heights range'%(maxHei) - print 'maxHei is setting to %.2f'%(self.dataOut.heightList[-1]) + print('maxHei: %.2f is out of the heights range'%(maxHei)) + print('maxHei is setting to %.2f'%(self.dataOut.heightList[-1])) maxHei = self.dataOut.heightList[-1] # validacion de velocidades @@ -679,13 +679,13 @@ class SpectraLagsProc(ProcessingUnit): maxVel = velrange[-1] if (minVel < velrange[0]) or (minVel > maxVel): - print 'minVel: %.2f is out of the velocity range'%(minVel) - print 'minVel is setting to %.2f'%(velrange[0]) + print('minVel: %.2f is out of the velocity range'%(minVel)) + print('minVel is setting to %.2f'%(velrange[0])) minVel = velrange[0] if (maxVel > velrange[-1]) or (maxVel < minVel): - print 'maxVel: %.2f is out of the velocity range'%(maxVel) - print 'maxVel is setting to %.2f'%(velrange[-1]) + print('maxVel: %.2f is out of the velocity range'%(maxVel)) + print('maxVel is setting to %.2f'%(velrange[-1])) maxVel = velrange[-1] # seleccion de indices para rango @@ -707,7 +707,7 @@ class SpectraLagsProc(ProcessingUnit): maxIndex = len(heights) if (minIndex < 0) or (minIndex > maxIndex): - raise ValueError, "some value in (%d,%d) is not valid" % (minIndex, maxIndex) + raise ValueError("some value in (%d,%d) is not valid" % (minIndex, maxIndex)) if (maxIndex >= self.dataOut.nHeights): maxIndex = self.dataOut.nHeights-1 @@ -736,4 +736,4 @@ class SpectraLagsProc(ProcessingUnit): self.dataOut.noise_estimation = noise.copy() - return 1 + return 1 \ No newline at end of file diff --git a/schainpy/model/proc/jroproc_voltage.py b/schainpy/model/proc/jroproc_voltage.py index 6384680..a54b500 100644 --- a/schainpy/model/proc/jroproc_voltage.py +++ b/schainpy/model/proc/jroproc_voltage.py @@ -1,31 +1,31 @@ import sys -import numpy +import numpy,math from scipy import interpolate -from schainpy import cSchain -from jroproc_base import ProcessingUnit, Operation -from schainpy.model.data.jrodata import Voltage +from schainpy.model.proc.jroproc_base import ProcessingUnit, Operation, MPDecorator +from schainpy.model.data.jrodata import Voltage,hildebrand_sekhon +from schainpy.utils import log from time import time -class VoltageProc(ProcessingUnit): - def __init__(self, **kwargs): +class VoltageProc(ProcessingUnit): + + def __init__(self): - ProcessingUnit.__init__(self, **kwargs) + ProcessingUnit.__init__(self) - # self.objectDict = {} self.dataOut = Voltage() self.flip = 1 + self.setupReq = False def run(self): + if self.dataIn.type == 'AMISR': self.__updateObjFromAmisrInput() if self.dataIn.type == 'Voltage': self.dataOut.copy(self.dataIn) - # self.dataOut.copy(self.dataIn) - def __updateObjFromAmisrInput(self): self.dataOut.timeZone = self.dataIn.timeZone @@ -37,7 +37,7 @@ class VoltageProc(ProcessingUnit): self.dataOut.data = self.dataIn.data self.dataOut.utctime = self.dataIn.utctime self.dataOut.channelList = self.dataIn.channelList - # self.dataOut.timeInterval = self.dataIn.timeInterval + #self.dataOut.timeInterval = self.dataIn.timeInterval self.dataOut.heightList = self.dataIn.heightList self.dataOut.nProfiles = self.dataIn.nProfiles @@ -51,32 +51,22 @@ class VoltageProc(ProcessingUnit): self.dataOut.beam.codeList = self.dataIn.beam.codeList self.dataOut.beam.azimuthList = self.dataIn.beam.azimuthList self.dataOut.beam.zenithList = self.dataIn.beam.zenithList - # - # pass# - # - # def init(self): - # - # - # if self.dataIn.type == 'AMISR': - # self.__updateObjFromAmisrInput() - # - # if self.dataIn.type == 'Voltage': - # self.dataOut.copy(self.dataIn) - # # No necesita copiar en cada init() los atributos de dataIn - # # la copia deberia hacerse por cada nuevo bloque de datos - - def selectChannels(self, channelList): - channelIndexList = [] +class selectChannels(Operation): + + def run(self, dataOut, channelList): + + channelIndexList = [] + self.dataOut = dataOut for channel in channelList: if channel not in self.dataOut.channelList: - raise ValueError, "Channel %d is not in %s" %(channel, str(self.dataOut.channelList)) + raise ValueError("Channel %d is not in %s" %(channel, str(self.dataOut.channelList))) index = self.dataOut.channelList.index(channel) channelIndexList.append(index) - self.selectChannelsByIndex(channelIndexList) + return self.dataOut def selectChannelsByIndex(self, channelIndexList): """ @@ -99,24 +89,64 @@ class VoltageProc(ProcessingUnit): for channelIndex in channelIndexList: if channelIndex not in self.dataOut.channelIndexList: - print channelIndexList - raise ValueError, "The value %d in channelIndexList is not valid" %channelIndex + raise ValueError("The value %d in channelIndexList is not valid" %channelIndex) + + if self.dataOut.type == 'Voltage': + if self.dataOut.flagDataAsBlock: + """ + Si la data es obtenida por bloques, dimension = [nChannels, nProfiles, nHeis] + """ + data = self.dataOut.data[channelIndexList,:,:] + else: + data = self.dataOut.data[channelIndexList,:] - if self.dataOut.flagDataAsBlock: - """ - Si la data es obtenida por bloques, dimension = [nChannels, nProfiles, nHeis] - """ - data = self.dataOut.data[channelIndexList,:,:] - else: - data = self.dataOut.data[channelIndexList,:] + self.dataOut.data = data + # self.dataOut.channelList = [self.dataOut.channelList[i] for i in channelIndexList] + self.dataOut.channelList = range(len(channelIndexList)) + + elif self.dataOut.type == 'Spectra': + data_spc = self.dataOut.data_spc[channelIndexList, :] + data_dc = self.dataOut.data_dc[channelIndexList, :] + + self.dataOut.data_spc = data_spc + self.dataOut.data_dc = data_dc + + # self.dataOut.channelList = [self.dataOut.channelList[i] for i in channelIndexList] + self.dataOut.channelList = range(len(channelIndexList)) + self.__selectPairsByChannel(channelIndexList) - self.dataOut.data = data - # self.dataOut.channelList = [self.dataOut.channelList[i] for i in channelIndexList] - self.dataOut.channelList = range(len(channelIndexList)) - return 1 - def selectHeights(self, minHei=None, maxHei=None): + def __selectPairsByChannel(self, channelList=None): + + if channelList == None: + return + + pairsIndexListSelected = [] + for pairIndex in self.dataOut.pairsIndexList: + # First pair + if self.dataOut.pairsList[pairIndex][0] not in channelList: + continue + # Second pair + if self.dataOut.pairsList[pairIndex][1] not in channelList: + continue + + pairsIndexListSelected.append(pairIndex) + + if not pairsIndexListSelected: + self.dataOut.data_cspc = None + self.dataOut.pairsList = [] + return + + self.dataOut.data_cspc = self.dataOut.data_cspc[pairsIndexListSelected] + self.dataOut.pairsList = [self.dataOut.pairsList[i] + for i in pairsIndexListSelected] + + return + +class selectHeights(Operation): + + def run(self, dataOut, minHei=None, maxHei=None): """ Selecciona un bloque de datos en base a un grupo de valores de alturas segun el rango minHei <= height <= maxHei @@ -132,6 +162,8 @@ class VoltageProc(ProcessingUnit): 1 si el metodo se ejecuto con exito caso contrario devuelve 0 """ + self.dataOut = dataOut + if minHei == None: minHei = self.dataOut.heightList[0] @@ -163,8 +195,7 @@ class VoltageProc(ProcessingUnit): self.selectHeightsByIndex(minIndex, maxIndex) - return 1 - + return self.dataOut def selectHeightsByIndex(self, minIndex, maxIndex): """ @@ -183,81 +214,118 @@ class VoltageProc(ProcessingUnit): 1 si el metodo se ejecuto con exito caso contrario devuelve 0 """ - if (minIndex < 0) or (minIndex > maxIndex): - raise ValueError, "Height index range (%d,%d) is not valid" % (minIndex, maxIndex) + if self.dataOut.type == 'Voltage': + if (minIndex < 0) or (minIndex > maxIndex): + raise ValueError("Height index range (%d,%d) is not valid" % (minIndex, maxIndex)) - if (maxIndex >= self.dataOut.nHeights): - maxIndex = self.dataOut.nHeights + if (maxIndex >= self.dataOut.nHeights): + maxIndex = self.dataOut.nHeights - #voltage - if self.dataOut.flagDataAsBlock: - """ - Si la data es obtenida por bloques, dimension = [nChannels, nProfiles, nHeis] - """ - data = self.dataOut.data[:,:, minIndex:maxIndex] - else: - data = self.dataOut.data[:, minIndex:maxIndex] + #voltage + if self.dataOut.flagDataAsBlock: + """ + Si la data es obtenida por bloques, dimension = [nChannels, nProfiles, nHeis] + """ + data = self.dataOut.data[:,:, minIndex:maxIndex] + else: + data = self.dataOut.data[:, minIndex:maxIndex] + + # firstHeight = self.dataOut.heightList[minIndex] + + self.dataOut.data = data + self.dataOut.heightList = self.dataOut.heightList[minIndex:maxIndex] + + if self.dataOut.nHeights <= 1: + raise ValueError("selectHeights: Too few heights. Current number of heights is %d" %(self.dataOut.nHeights)) + elif self.dataOut.type == 'Spectra': + if (minIndex < 0) or (minIndex > maxIndex): + raise ValueError("Error selecting heights: Index range (%d,%d) is not valid" % ( + minIndex, maxIndex)) + + if (maxIndex >= self.dataOut.nHeights): + maxIndex = self.dataOut.nHeights - 1 + + # Spectra + data_spc = self.dataOut.data_spc[:, :, minIndex:maxIndex + 1] + + data_cspc = None + if self.dataOut.data_cspc is not None: + data_cspc = self.dataOut.data_cspc[:, :, minIndex:maxIndex + 1] - # firstHeight = self.dataOut.heightList[minIndex] + data_dc = None + if self.dataOut.data_dc is not None: + data_dc = self.dataOut.data_dc[:, minIndex:maxIndex + 1] - self.dataOut.data = data - self.dataOut.heightList = self.dataOut.heightList[minIndex:maxIndex] + self.dataOut.data_spc = data_spc + self.dataOut.data_cspc = data_cspc + self.dataOut.data_dc = data_dc - if self.dataOut.nHeights <= 1: - raise ValueError, "selectHeights: Too few heights. Current number of heights is %d" %(self.dataOut.nHeights) + self.dataOut.heightList = self.dataOut.heightList[minIndex:maxIndex + 1] return 1 - def filterByHeights(self, window): +class filterByHeights(Operation): - deltaHeight = self.dataOut.heightList[1] - self.dataOut.heightList[0] + def run(self, dataOut, window): + + deltaHeight = dataOut.heightList[1] - dataOut.heightList[0] if window == None: - window = (self.dataOut.radarControllerHeaderObj.txA/self.dataOut.radarControllerHeaderObj.nBaud) / deltaHeight + window = (dataOut.radarControllerHeaderObj.txA/dataOut.radarControllerHeaderObj.nBaud) / deltaHeight newdelta = deltaHeight * window - r = self.dataOut.nHeights % window - newheights = (self.dataOut.nHeights-r)/window + r = dataOut.nHeights % window + newheights = (dataOut.nHeights-r)/window if newheights <= 1: - raise ValueError, "filterByHeights: Too few heights. Current number of heights is %d and window is %d" %(self.dataOut.nHeights, window) + raise ValueError("filterByHeights: Too few heights. Current number of heights is %d and window is %d" %(dataOut.nHeights, window)) - if self.dataOut.flagDataAsBlock: + if dataOut.flagDataAsBlock: """ Si la data es obtenida por bloques, dimension = [nChannels, nProfiles, nHeis] """ - buffer = self.dataOut.data[:, :, 0:self.dataOut.nHeights-r] - buffer = buffer.reshape(self.dataOut.nChannels,self.dataOut.nProfiles,self.dataOut.nHeights/window,window) + buffer = dataOut.data[:, :, 0:int(dataOut.nHeights-r)] + buffer = buffer.reshape(dataOut.nChannels, dataOut.nProfiles, int(dataOut.nHeights/window), window) buffer = numpy.sum(buffer,3) else: - buffer = self.dataOut.data[:,0:self.dataOut.nHeights-r] - buffer = buffer.reshape(self.dataOut.nChannels,self.dataOut.nHeights/window,window) + buffer = dataOut.data[:,0:int(dataOut.nHeights-r)] + buffer = buffer.reshape(dataOut.nChannels,int(dataOut.nHeights/window),int(window)) buffer = numpy.sum(buffer,2) - self.dataOut.data = buffer - self.dataOut.heightList = self.dataOut.heightList[0] + numpy.arange( newheights )*newdelta - self.dataOut.windowOfFilter = window + dataOut.data = buffer + dataOut.heightList = dataOut.heightList[0] + numpy.arange( newheights )*newdelta + dataOut.windowOfFilter = window + + return dataOut - def setH0(self, h0, deltaHeight = None): + +class setH0(Operation): + + def run(self, dataOut, h0, deltaHeight = None): if not deltaHeight: - deltaHeight = self.dataOut.heightList[1] - self.dataOut.heightList[0] + deltaHeight = dataOut.heightList[1] - dataOut.heightList[0] - nHeights = self.dataOut.nHeights + nHeights = dataOut.nHeights newHeiRange = h0 + numpy.arange(nHeights)*deltaHeight - self.dataOut.heightList = newHeiRange + dataOut.heightList = newHeiRange + + return dataOut - def deFlip(self, channelList = []): - data = self.dataOut.data.copy() +class deFlip(Operation): - if self.dataOut.flagDataAsBlock: + def run(self, dataOut, channelList = []): + + data = dataOut.data.copy() + + if dataOut.flagDataAsBlock: flip = self.flip - profileList = range(self.dataOut.nProfiles) + profileList = list(range(dataOut.nProfiles)) if not channelList: for thisProfile in profileList: @@ -265,7 +333,7 @@ class VoltageProc(ProcessingUnit): flip *= -1.0 else: for thisChannel in channelList: - if thisChannel not in self.dataOut.channelList: + if thisChannel not in dataOut.channelList: continue for thisProfile in profileList: @@ -279,41 +347,74 @@ class VoltageProc(ProcessingUnit): data[:,:] = data[:,:]*self.flip else: for thisChannel in channelList: - if thisChannel not in self.dataOut.channelList: + if thisChannel not in dataOut.channelList: continue data[thisChannel,:] = data[thisChannel,:]*self.flip self.flip *= -1. - self.dataOut.data = data + dataOut.data = data - def setRadarFrequency(self, frequency=None): + return dataOut - if frequency != None: - self.dataOut.frequency = frequency - return 1 +class setAttribute(Operation): + ''' + Set an arbitrary attribute(s) to dataOut + ''' + + def __init__(self): + + Operation.__init__(self) + self._ready = False - def interpolateHeights(self, topLim, botLim): + def run(self, dataOut, **kwargs): + + for key, value in kwargs.items(): + setattr(dataOut, key, value) + + return dataOut + + +@MPDecorator +class printAttribute(Operation): + ''' + Print an arbitrary attribute of dataOut + ''' + + def __init__(self): + + Operation.__init__(self) + + def run(self, dataOut, attributes): + + for attr in attributes: + if hasattr(dataOut, attr): + log.log(getattr(dataOut, attr), attr) + + +class interpolateHeights(Operation): + + def run(self, dataOut, topLim, botLim): #69 al 72 para julia #82-84 para meteoros - if len(numpy.shape(self.dataOut.data))==2: - sampInterp = (self.dataOut.data[:,botLim-1] + self.dataOut.data[:,topLim+1])/2 + if len(numpy.shape(dataOut.data))==2: + sampInterp = (dataOut.data[:,botLim-1] + dataOut.data[:,topLim+1])/2 sampInterp = numpy.transpose(numpy.tile(sampInterp,(topLim-botLim + 1,1))) - #self.dataOut.data[:,botLim:limSup+1] = sampInterp - self.dataOut.data[:,botLim:topLim+1] = sampInterp + #dataOut.data[:,botLim:limSup+1] = sampInterp + dataOut.data[:,botLim:topLim+1] = sampInterp else: - nHeights = self.dataOut.data.shape[2] + nHeights = dataOut.data.shape[2] x = numpy.hstack((numpy.arange(botLim),numpy.arange(topLim+1,nHeights))) - y = self.dataOut.data[:,:,range(botLim)+range(topLim+1,nHeights)] + y = dataOut.data[:,:,list(range(botLim))+list(range(topLim+1,nHeights))] f = interpolate.interp1d(x, y, axis = 2) xnew = numpy.arange(botLim,topLim+1) ynew = f(xnew) + dataOut.data[:,:,botLim:topLim+1] = ynew - self.dataOut.data[:,:,botLim:topLim+1] = ynew + return dataOut - # import collections class CohInt(Operation): @@ -334,8 +435,6 @@ class CohInt(Operation): Operation.__init__(self, **kwargs) - # self.isConfig = False - def setup(self, n=None, timeInterval=None, stride=None, overlapping=False, byblock=False): """ Set the parameters of the integration class. @@ -355,7 +454,7 @@ class CohInt(Operation): self.stride = stride if n == None and timeInterval == None: - raise ValueError, "n or timeInterval should be specified ..." + raise ValueError("n or timeInterval should be specified ...") if n != None: self.n = n @@ -495,8 +594,8 @@ class CohInt(Operation): # print self.__bufferStride[self.__profIndexStride - 1] # raise return self.__bufferStride[self.__profIndexStride - 1] - - + + return None, None def integrate(self, data, datatime=None): @@ -518,7 +617,7 @@ class CohInt(Operation): avgdatatime = self.__initime deltatime = datatime - self.__lastdatatime - + if not self.__withOverlapping: self.__initime = datatime else: @@ -544,8 +643,9 @@ class CohInt(Operation): avgdatatime = (times - 1) * timeInterval + dataOut.utctime self.__dataReady = True return avgdata, avgdatatime - + def run(self, dataOut, n=None, timeInterval=None, stride=None, overlapping=False, byblock=False, **kwargs): + if not self.isConfig: self.setup(n=n, stride=stride, timeInterval=timeInterval, overlapping=overlapping, byblock=byblock, **kwargs) self.isConfig = True @@ -557,23 +657,26 @@ class CohInt(Operation): avgdata, avgdatatime = self.integrateByBlock(dataOut) dataOut.nProfiles /= self.n else: - if stride is None: + if stride is None: avgdata, avgdatatime = self.integrate(dataOut.data, dataOut.utctime) else: avgdata, avgdatatime = self.integrateByStride(dataOut.data, dataOut.utctime) - + # dataOut.timeInterval *= n dataOut.flagNoData = True if self.__dataReady: dataOut.data = avgdata - dataOut.nCohInt *= self.n + if not dataOut.flagCohInt: + dataOut.nCohInt *= self.n + dataOut.flagCohInt = True dataOut.utctime = avgdatatime # print avgdata, avgdatatime # raise # dataOut.timeInterval = dataOut.ippSeconds * dataOut.nCohInt dataOut.flagNoData = False + return dataOut class Decoder(Operation): @@ -593,7 +696,7 @@ class Decoder(Operation): self.osamp = None # self.__setValues = False self.isConfig = False - + self.setupReq = False def setup(self, code, osamp, dataOut): self.__profIndex = 0 @@ -613,7 +716,7 @@ class Decoder(Operation): self.__nHeis = dataOut.nHeights if self.__nHeis < self.nBaud: - raise ValueError, 'Number of heights (%d) should be greater than number of bauds (%d)' %(self.__nHeis, self.nBaud) + raise ValueError('Number of heights (%d) should be greater than number of bauds (%d)' %(self.__nHeis, self.nBaud)) #Frequency __codeBuffer = numpy.zeros((self.nCode, self.__nHeis), dtype=numpy.complex) @@ -661,21 +764,20 @@ class Decoder(Operation): def __convolutionByBlockInTime(self, data): - repetitions = self.__nProfiles / self.nCode - + repetitions = int(self.__nProfiles / self.nCode) junk = numpy.lib.stride_tricks.as_strided(self.code, (repetitions, self.code.size), (0, self.code.itemsize)) junk = junk.flatten() code_block = numpy.reshape(junk, (self.nCode*repetitions, self.nBaud)) - profilesList = xrange(self.__nProfiles) - - for i in range(self.__nChannels): - for j in profilesList: - self.datadecTime[i,j,:] = numpy.correlate(data[i,j,:], code_block[j,:], mode='full')[self.nBaud-1:] - return self.datadecTime + profilesList = range(self.__nProfiles) + + for i in range(self.__nChannels): + for j in profilesList: + self.datadecTime[i,j,:] = numpy.correlate(data[i,j,:], code_block[j,:], mode='full')[self.nBaud-1:] + return self.datadecTime def __convolutionByBlockInFreq(self, data): - raise NotImplementedError, "Decoder by frequency fro Blocks not implemented" + raise NotImplementedError("Decoder by frequency fro Blocks not implemented") fft_code = self.fft_code[self.__profIndex].reshape(1,-1) @@ -688,17 +790,17 @@ class Decoder(Operation): return data - + def run(self, dataOut, code=None, nCode=None, nBaud=None, mode = 0, osamp=None, times=None): if dataOut.flagDecodeData: - print "This data is already decoded, recoding again ..." + print("This data is already decoded, recoding again ...") if not self.isConfig: if code is None: if dataOut.code is None: - raise ValueError, "Code could not be read from %s instance. Enter a value in Code parameter" %dataOut.type + raise ValueError("Code could not be read from %s instance. Enter a value in Code parameter" %dataOut.type) code = dataOut.code else: @@ -714,12 +816,12 @@ class Decoder(Operation): sys.stderr.write("Decoder Warning: Argument 'times' in not used anymore\n") if self.code is None: - print "Fail decoding: Code is not defined." + print("Fail decoding: Code is not defined.") return self.__nProfiles = dataOut.nProfiles datadec = None - + if mode == 3: mode = 0 @@ -746,7 +848,7 @@ class Decoder(Operation): datadec = self.__convolutionInFreqOpt(dataOut.data) if datadec is None: - raise ValueError, "Codification mode selected is not valid: mode=%d. Try selecting 0 or 1" %mode + raise ValueError("Codification mode selected is not valid: mode=%d. Try selecting 0 or 1" %mode) dataOut.code = self.code dataOut.nCode = self.nCode @@ -760,11 +862,11 @@ class Decoder(Operation): if self.__profIndex == self.nCode-1: self.__profIndex = 0 - return 1 + return dataOut self.__profIndex += 1 - return 1 + return dataOut # dataOut.flagDeflipData = True #asumo q la data no esta sin flip @@ -795,7 +897,6 @@ class ProfileConcat(Operation): self.start_index = self.start_index + self.nHeights def run(self, dataOut, m): - dataOut.flagNoData = True if not self.isConfig: @@ -803,7 +904,7 @@ class ProfileConcat(Operation): self.isConfig = True if dataOut.flagDataAsBlock: - raise ValueError, "ProfileConcat can only be used when voltage have been read profile by profile, getBlock = False" + raise ValueError("ProfileConcat can only be used when voltage have been read profile by profile, getBlock = False") else: self.concat(dataOut.data) @@ -817,6 +918,7 @@ class ProfileConcat(Operation): xf = dataOut.heightList[0] + dataOut.nHeights * deltaHeight * m dataOut.heightList = numpy.arange(dataOut.heightList[0], xf, deltaHeight) dataOut.ippSeconds *= m + return dataOut class ProfileSelector(Operation): @@ -883,7 +985,7 @@ class ProfileSelector(Operation): if profileRangeList != None: minIndex = profileRangeList[0] maxIndex = profileRangeList[1] - profileList = range(minIndex, maxIndex+1) + profileList = list(range(minIndex, maxIndex+1)) dataOut.data = dataOut.data[:,minIndex:maxIndex+1,:] @@ -895,7 +997,7 @@ class ProfileSelector(Operation): minIndex = thisRange[0] maxIndex = thisRange[1] - profileList.extend(range(minIndex, maxIndex+1)) + profileList.extend(list(range(minIndex, maxIndex+1))) dataOut.data = dataOut.data[:,profileList,:] @@ -903,7 +1005,7 @@ class ProfileSelector(Operation): dataOut.profileIndex = dataOut.nProfiles - 1 dataOut.flagNoData = False - return True + return dataOut """ data dimension = [nChannels, nHeis] @@ -919,7 +1021,7 @@ class ProfileSelector(Operation): dataOut.flagNoData = False self.incProfileIndex() - return True + return dataOut if profileRangeList != None: @@ -934,7 +1036,7 @@ class ProfileSelector(Operation): dataOut.flagNoData = False self.incProfileIndex() - return True + return dataOut if rangeList != None: @@ -962,7 +1064,7 @@ class ProfileSelector(Operation): break - return True + return dataOut if beam != None: #beam is only for AMISR data @@ -972,11 +1074,10 @@ class ProfileSelector(Operation): self.incProfileIndex() - return True + return dataOut - raise ValueError, "ProfileSelector needs profileList, profileRangeList or rangeList parameter" + raise ValueError("ProfileSelector needs profileList, profileRangeList or rangeList parameter") - return False class Reshaper(Operation): @@ -1015,21 +1116,21 @@ class Reshaper(Operation): def __checkInputs(self, dataOut, shape, nTxs): if shape is None and nTxs is None: - raise ValueError, "Reshaper: shape of factor should be defined" + raise ValueError("Reshaper: shape of factor should be defined") if nTxs: if nTxs < 0: - raise ValueError, "nTxs should be greater than 0" + raise ValueError("nTxs should be greater than 0") if nTxs < 1 and dataOut.nProfiles % (1./nTxs) != 0: - raise ValueError, "nProfiles= %d is not divisibled by (1./nTxs) = %f" %(dataOut.nProfiles, (1./nTxs)) + raise ValueError("nProfiles= %d is not divisibled by (1./nTxs) = %f" %(dataOut.nProfiles, (1./nTxs))) shape = [dataOut.nChannels, dataOut.nProfiles*nTxs, dataOut.nHeights/nTxs] return shape, nTxs if len(shape) != 2 and len(shape) != 3: - raise ValueError, "shape dimension should be equal to 2 or 3. shape = (nProfiles, nHeis) or (nChannels, nProfiles, nHeis). Actually shape = (%d, %d, %d)" %(dataOut.nChannels, dataOut.nProfiles, dataOut.nHeights) + raise ValueError("shape dimension should be equal to 2 or 3. shape = (nProfiles, nHeis) or (nChannels, nProfiles, nHeis). Actually shape = (%d, %d, %d)" %(dataOut.nChannels, dataOut.nProfiles, dataOut.nHeights)) if len(shape) == 2: shape_tuple = [dataOut.nChannels] @@ -1069,7 +1170,7 @@ class Reshaper(Operation): profileIndex = dataOut.profileIndex*nTxs else: - raise ValueError, "nTxs should be greater than 0 and lower than 1, or use VoltageReader(..., getblock=True)" + raise ValueError("nTxs should be greater than 0 and lower than 1, or use VoltageReader(..., getblock=True)") deltaHeight = dataOut.heightList[1] - dataOut.heightList[0] @@ -1081,6 +1182,8 @@ class Reshaper(Operation): dataOut.ippSeconds /= self.__nTxs + return dataOut + class SplitProfiles(Operation): def __init__(self, **kwargs): @@ -1098,9 +1201,9 @@ class SplitProfiles(Operation): shape = dataOut.data.shape if shape[2] % n != 0: - raise ValueError, "Could not split the data, n=%d has to be multiple of %d" %(n, shape[2]) + raise ValueError("Could not split the data, n=%d has to be multiple of %d" %(n, shape[2])) - new_shape = shape[0], shape[1]*n, shape[2]/n + new_shape = shape[0], shape[1]*n, int(shape[2]/n) dataOut.data = numpy.reshape(dataOut.data, new_shape) dataOut.flagNoData = False @@ -1109,7 +1212,7 @@ class SplitProfiles(Operation): else: - raise ValueError, "Could not split the data when is read Profile by Profile. Use VoltageReader(..., getblock=True)" + raise ValueError("Could not split the data when is read Profile by Profile. Use VoltageReader(..., getblock=True)") deltaHeight = dataOut.heightList[1] - dataOut.heightList[0] @@ -1121,6 +1224,8 @@ class SplitProfiles(Operation): dataOut.ippSeconds /= n + return dataOut + class CombineProfiles(Operation): def __init__(self, **kwargs): @@ -1141,7 +1246,7 @@ class CombineProfiles(Operation): new_shape = shape[0], shape[1]/n, shape[2]*n if shape[1] % n != 0: - raise ValueError, "Could not split the data, n=%d has to be multiple of %d" %(n, shape[1]) + raise ValueError("Could not split the data, n=%d has to be multiple of %d" %(n, shape[1])) dataOut.data = numpy.reshape(dataOut.data, new_shape) dataOut.flagNoData = False @@ -1182,6 +1287,209 @@ class CombineProfiles(Operation): dataOut.ippSeconds *= n + return dataOut + +class PulsePairVoltage(Operation): + ''' + Function PulsePair(Signal Power, Velocity) + The real component of Lag[0] provides Intensity Information + The imag component of Lag[1] Phase provides Velocity Information + + Configuration Parameters: + nPRF = Number of Several PRF + theta = Degree Azimuth angel Boundaries + + Input: + self.dataOut + lag[N] + Affected: + self.dataOut.spc + ''' + isConfig = False + __profIndex = 0 + __initime = None + __lastdatatime = None + __buffer = None + noise = None + __dataReady = False + n = None + __nch = 0 + __nHeis = 0 + removeDC = False + ipp = None + lambda_ = 0 + + def __init__(self,**kwargs): + Operation.__init__(self,**kwargs) + + def setup(self, dataOut, n = None, removeDC=False): + ''' + n= Numero de PRF's de entrada + ''' + self.__initime = None + self.__lastdatatime = 0 + self.__dataReady = False + self.__buffer = 0 + self.__profIndex = 0 + self.noise = None + self.__nch = dataOut.nChannels + self.__nHeis = dataOut.nHeights + self.removeDC = removeDC + self.lambda_ = 3.0e8/(9345.0e6) + self.ippSec = dataOut.ippSeconds + self.nCohInt = dataOut.nCohInt + print("IPPseconds",dataOut.ippSeconds) + + print("ELVALOR DE n es:", n) + if n == None: + raise ValueError("n should be specified.") + + if n != None: + if n<2: + raise ValueError("n should be greater than 2") + + self.n = n + self.__nProf = n + + self.__buffer = numpy.zeros((dataOut.nChannels, + n, + dataOut.nHeights), + dtype='complex') + + def putData(self,data): + ''' + Add a profile to he __buffer and increase in one the __profiel Index + ''' + self.__buffer[:,self.__profIndex,:]= data + self.__profIndex += 1 + return + + def pushData(self,dataOut): + ''' + Return the PULSEPAIR and the profiles used in the operation + Affected : self.__profileIndex + ''' + #················· Remove DC··································· + if self.removeDC==True: + mean = numpy.mean(self.__buffer,1) + tmp = mean.reshape(self.__nch,1,self.__nHeis) + dc= numpy.tile(tmp,[1,self.__nProf,1]) + self.__buffer = self.__buffer - dc + #··················Calculo de Potencia ························ + pair0 = self.__buffer*numpy.conj(self.__buffer) + pair0 = pair0.real + lag_0 = numpy.sum(pair0,1) + #··················Calculo de Ruido x canal···················· + self.noise = numpy.zeros(self.__nch) + for i in range(self.__nch): + daux = numpy.sort(pair0[i,:,:],axis= None) + self.noise[i]=hildebrand_sekhon( daux ,self.nCohInt) + + self.noise = self.noise.reshape(self.__nch,1) + self.noise = numpy.tile(self.noise,[1,self.__nHeis]) + noise_buffer = self.noise.reshape(self.__nch,1,self.__nHeis) + noise_buffer = numpy.tile(noise_buffer,[1,self.__nProf,1]) + #·················· Potencia recibida= P , Potencia senal = S , Ruido= N·· + #·················· P= S+N ,P=lag_0/N ································· + #···················· Power ·················································· + data_power = lag_0/(self.n*self.nCohInt) + #------------------ Senal ··················································· + data_intensity = pair0 - noise_buffer + data_intensity = numpy.sum(data_intensity,axis=1)*(self.n*self.nCohInt)#*self.nCohInt) + #data_intensity = (lag_0-self.noise*self.n)*(self.n*self.nCohInt) + for i in range(self.__nch): + for j in range(self.__nHeis): + if data_intensity[i][j] < 0: + data_intensity[i][j] = numpy.min(numpy.absolute(data_intensity[i][j])) + + #················· Calculo de Frecuencia y Velocidad doppler········ + pair1 = self.__buffer[:,:-1,:]*numpy.conjugate(self.__buffer[:,1:,:]) + lag_1 = numpy.sum(pair1,1) + data_freq = (-1/(2.0*math.pi*self.ippSec*self.nCohInt))*numpy.angle(lag_1) + data_velocity = (self.lambda_/2.0)*data_freq + + #················ Potencia promedio estimada de la Senal··········· + lag_0 = lag_0/self.n + S = lag_0-self.noise + + #················ Frecuencia Doppler promedio ····················· + lag_1 = lag_1/(self.n-1) + R1 = numpy.abs(lag_1) + + #················ Calculo del SNR·································· + data_snrPP = S/self.noise + for i in range(self.__nch): + for j in range(self.__nHeis): + if data_snrPP[i][j] < 1.e-20: + data_snrPP[i][j] = 1.e-20 + + #················· Calculo del ancho espectral ······················ + L = S/R1 + L = numpy.where(L<0,1,L) + L = numpy.log(L) + tmp = numpy.sqrt(numpy.absolute(L)) + data_specwidth = (self.lambda_/(2*math.sqrt(2)*math.pi*self.ippSec*self.nCohInt))*tmp*numpy.sign(L) + n = self.__profIndex + + self.__buffer = numpy.zeros((self.__nch, self.__nProf,self.__nHeis), dtype='complex') + self.__profIndex = 0 + return data_power,data_intensity,data_velocity,data_snrPP,data_specwidth,n + + + def pulsePairbyProfiles(self,dataOut): + + self.__dataReady = False + data_power = None + data_intensity = None + data_velocity = None + data_specwidth = None + data_snrPP = None + self.putData(data=dataOut.data) + if self.__profIndex == self.n: + data_power,data_intensity, data_velocity,data_snrPP,data_specwidth, n = self.pushData(dataOut=dataOut) + self.__dataReady = True + + return data_power, data_intensity, data_velocity, data_snrPP, data_specwidth + + + def pulsePairOp(self, dataOut, datatime= None): + + if self.__initime == None: + self.__initime = datatime + data_power, data_intensity, data_velocity, data_snrPP, data_specwidth = self.pulsePairbyProfiles(dataOut) + self.__lastdatatime = datatime + + if data_power is None: + return None, None, None,None,None,None + + avgdatatime = self.__initime + deltatime = datatime - self.__lastdatatime + self.__initime = datatime + + return data_power, data_intensity, data_velocity, data_snrPP, data_specwidth, avgdatatime + + def run(self, dataOut,n = None,removeDC= False, overlapping= False,**kwargs): + + if not self.isConfig: + self.setup(dataOut = dataOut, n = n , removeDC=removeDC , **kwargs) + self.isConfig = True + data_power, data_intensity, data_velocity,data_snrPP,data_specwidth, avgdatatime = self.pulsePairOp(dataOut, dataOut.utctime) + dataOut.flagNoData = True + + if self.__dataReady: + dataOut.nCohInt *= self.n + dataOut.dataPP_POW = data_intensity # S + dataOut.dataPP_POWER = data_power # P + dataOut.dataPP_DOP = data_velocity + dataOut.dataPP_SNR = data_snrPP + dataOut.dataPP_WIDTH = data_specwidth + dataOut.PRFbyAngle = self.n #numero de PRF*cada angulo rotado que equivale a un tiempo. + dataOut.utctime = avgdatatime + dataOut.flagNoData = False + return dataOut + + + # import collections # from scipy.stats import mode # diff --git a/schainpy/model/proc/pxproc_parameters.py b/schainpy/model/proc/pxproc_parameters.py index 5d5e9d8..9ba0611 100644 --- a/schainpy/model/proc/pxproc_parameters.py +++ b/schainpy/model/proc/pxproc_parameters.py @@ -11,7 +11,7 @@ from time import gmtime from numpy import transpose -from jroproc_base import ProcessingUnit, Operation +from .jroproc_base import ProcessingUnit, Operation from schainpy.model.data.jrodata import Parameters diff --git a/schainpy/model/serializer/__init__.py b/schainpy/model/serializer/__init__.py deleted file mode 100644 index e69de29..0000000 --- a/schainpy/model/serializer/__init__.py +++ /dev/null diff --git a/schainpy/model/serializer/data.py b/schainpy/model/serializer/data.py deleted file mode 100644 index 1212559..0000000 --- a/schainpy/model/serializer/data.py +++ /dev/null @@ -1,114 +0,0 @@ -''' -Created on Jul 15, 2014 - -@author: Miguel Urco -''' -from serializer import DynamicSerializer - -DEFAULT_SERIALIZER = None #'cPickle', 'msgpack', "yaml" - -from schainpy.model.data.jrodata import * - -CLASSNAME_KEY = 'classname__' - -def isObject(myObj): - - return hasattr(myObj,'__dict__') - -def isDictFormat(thisValue): - - if type(thisValue) != type({}): - return False - - if CLASSNAME_KEY not in thisValue.keys(): - return False - - return True - -def obj2Dict(myObj, keyList=[]): - - if not keyList: - keyList = myObj.__dict__.keys() - - myDict = {} - - myDict[CLASSNAME_KEY] = myObj.__class__.__name__ - - for thisKey, thisValue in myObj.__dict__.items(): - - if thisKey not in keyList: - continue - - if not isObject(thisValue): - myDict[thisKey] = thisValue - continue - - ## If this value is another class instance - myDict[thisKey] = obj2Dict(thisValue) - - return myDict - -def dict2Obj(myDict): - ''' - ''' - - if CLASSNAME_KEY not in myDict.keys(): - return None - - className = eval(myDict[CLASSNAME_KEY]) - - myObj = className() - - for thisKey, thisValue in myDict.items(): - - if thisKey == CLASSNAME_KEY: - continue - - if not isDictFormat(thisValue): - setattr(myObj, thisKey, thisValue) - continue - - myNewObj = dict2Obj(thisValue) - setattr(myObj, thisKey, myNewObj) - - return myObj - -def dict2Serial(myDict, serializer=DEFAULT_SERIALIZER): - - SERIALIZER = DynamicSerializer(serializer) - - mySerial = SERIALIZER.dumps(myDict) - - return mySerial - -def serial2Dict(mySerial, serializer=DEFAULT_SERIALIZER): - - SERIALIZER = DynamicSerializer(serializer) - - myDict = SERIALIZER.loads(mySerial) - - return myDict - -def obj2Serial(myObj, serializer=DEFAULT_SERIALIZER, **kwargs): - - SERIALIZER = DynamicSerializer(serializer) - - myDict = obj2Dict(myObj, **kwargs) - mySerial = dict2Serial(myDict, serializer) - - return mySerial - -def serial2Obj(mySerial, metadataDict = {}, serializer=DEFAULT_SERIALIZER): - - SERIALIZER = DynamicSerializer(serializer) - - myDataDict = serial2Dict(mySerial, serializer) - - if not metadataDict: - myObj = dict2Obj(myDataDict) - return myObj - - metadataDict.update(myDataDict) - myObj = dict2Obj(metadataDict) - - return myObj diff --git a/schainpy/model/serializer/serializer.py b/schainpy/model/serializer/serializer.py deleted file mode 100644 index 82e5f46..0000000 --- a/schainpy/model/serializer/serializer.py +++ /dev/null @@ -1,108 +0,0 @@ -''' -Created on Jul 17, 2014 - -@author: roj-idl71 -''' - -DEFAULT_SERIALIZER = None - -try: - import cPickle - DEFAULT_SERIALIZER = 'cPickle' -except: - pass - -try: - import msgpack_numpy - DEFAULT_SERIALIZER = 'msgpack' -except: - pass - -# import jsonpickle -# import yaml - -class Serializer(object): - - def __init__(self): - - self.serializer = None - - def dumps(self, obj, **kwargs): - - return self.serializer.dumps(obj, **kwargs) - - def loads(self, obj, **kwargs): - return self.serializer.loads(obj, **kwargs) - -class cPickleSerializer(Serializer): - - def __init__(self): - self.serializer = cPickle - - def dumps(self, obj, **kwargs): - return self.serializer.dumps(obj, 2) - - def loads(self, obj, **kwargs): - return self.serializer.loads(obj) - -class msgpackSerializer(Serializer): - - def __init__(self): - - self.serializer = msgpack_numpy - - def dumps(self, obj, **kwargs): - return self.serializer.packb(obj) - - def loads(self, obj, **kwargs): - return self.serializer.unpackb(obj) - -# class jsonpickleSerializer(Serializer): -# -# def __init__(self): -# -# self.serializer = jsonpickle -# -# def dumps(self, obj, **kwargs): -# return self.serializer.encode(obj, **kwargs) -# -# def loads(self, obj, **kwargs): -# return self.serializer.decode(obj, **kwargs) -# -# class yamlSerializer(Serializer): -# -# def __init__(self): -# -# self.serializer = yaml -# -# def dumps(self, obj, **kwargs): -# return self.serializer.dump(obj, **kwargs) -# -# def loads(self, obj, **kwargs): -# return self.serializer.load(obj, **kwargs) - -class DynamicSerializer(Serializer): - - def __init__(self, module = None): - - if not DEFAULT_SERIALIZER: - raise ImportError, "Install a python serializer like cPickle or msgpack" - - if not module: - module == DEFAULT_SERIALIZER - - if module == 'cPickle': - self.serializer = cPickleSerializer() -# -# if module == 'jsonpickle': -# self.serializer = jsonpickleSerializer() -# -# if module == 'yaml': -# self.serializer = yamlSerializer() - - if module == 'msgpack': - self.serializer = msgpackSerializer() - - -if __name__ == '__main__': - pass \ No newline at end of file diff --git a/schainpy/model/serializer/test/DynamicObject.py b/schainpy/model/serializer/test/DynamicObject.py deleted file mode 100644 index 5045e1f..0000000 --- a/schainpy/model/serializer/test/DynamicObject.py +++ /dev/null @@ -1,816 +0,0 @@ -''' -The DynamicObject module supports dynamic loading of YAML -defined objects into Python class objects. Object can -be sub-classed to allow direct binding of methods having -matching signatures. - -$Id$ -''' - -import urllib -import os -import re -import yaml # YAML Ain't Markup Language -import numpy as np -import copy -import inspect -import PrecisionTime -import time -import sys -import datetime -import collections - -# Replacement Loader for PyYAML to keep dictionaries in-order: -import OrderedYAML -#OrderedYAML.collections - -class Object(object): - """ Loads a YAML defined python class dynamically using the supplied URI, - which may be a file, directory, web hyper-link, or hyper-linked directory. """ - - # Dictionary containing all known Object class names and corresponding class objects - dynamicClasses = collections.OrderedDict() - - def __init__(self, object_uri=None, revision=None, recursive=False): - if isinstance(object_uri, file): - # URI is a yaml file - read it. - self.yaml = file.read() - elif object_uri == None: - self.yaml = None - elif isinstance(object_uri, str): - if object_uri.endswith('.yml'): - # URI is a web hyper-linked yaml file - read it. - self.yaml = urllib.urlopen(object_uri).read() - else: - # URI is a (hyper-linked?) directory - try reading it. - #print "URI is a directory." - try: - self.files = self.__parseLink(object_uri, recursive) - except IOError: - # URI is a local directory - get a list of YAML files in it - self.files = self.__getYamlFiles(object_uri, recursive) - - # For each YAML file found, create a new DynamicObject of it: - self.yaml = [] - for fn in self.files: - self.yaml.append(Object(fn)) - else: - print "Invalid URI supplied: %s"%(object_uri,) - - def __parseLink(self, object_uri, recursive): - """ Returns a listing of all YAML files located in the - hyper-link directory given by page. """ - page = urllib.urlopen(object_uri).read() - #print "URI is a URL directory: %s"%(object_uri,) - pattern = re.compile(r'') - - # List of files contained in the directory at the given URL, ignoring - # any "?" / GET query-string locations given: - files = [x[9:-2] for x in pattern.findall(page) if not x[9:-2].startswith('?')] - #print files - - yamlFiles = [] - dirs = [] - for fn in files: - if not fn.startswith('/'): # Ignore absolute paths... - path = os.path.join(object_uri, fn) - #print path - - # Keep list of YAML files found... - if fn.endswith('.yml'): - yamlFiles.append(path) - - # Keep list of directories found... - elif recursive and fn.endswith('/'): - dirs.append(path) - - if recursive: - #print dirs - for path in dirs: - yamlFiles += self.__parseLink(path,recursive) - - return yamlFiles - - def __getYamlFiles(self, local_dir, recursive): - """ Returns a listing of all YAML files located in the given - directory, recursing if requested. """ - yamlFiles = [] - dirs = [] - for fn in os.listdir(local_dir): - path = os.path.join(local_dir, fn) - - # List of YAML files found... - if fn.endswith('.yml'): - yamlFiles.append(path) - - # List of directories found... - elif recursive and os.path.isdir(path): - dirs.append(path) - - # Recurse if desired: - if recursive: - for path in dirs: - yamlFiles += self.__getYamlFiles(path,recursive) - - return yamlFiles - - def equals(self, obj, compare_time_created=True): - """ Returns True iff self has identical attributes - (numerically) to obj (no extras) """ - - if not isinstance(obj, Object): return False - - self_keys = self.__dict__.keys() - obj_keys = obj.__dict__.keys() - if not self_keys == obj_keys: - return False - for key in self_keys: - obj_keys.remove(key) - - self_value, obj_value = self.__dict__[key], obj.__dict__[key] - if isinstance(self_value, Object): - if not self_value.equals(obj_value, compare_time_created): - return False - elif isinstance(self_value, np.ndarray): - m1 = map(repr,self_value.flat) - m2 = map(repr,obj_value.flat) - ret = m1 == m2 - if not ret: - return False - else: - if not self_value == obj_value: - # Return False iff the different times are important - return key == '__time_created' and not compare_time_created - - return obj_keys == [] # no more keys --> the objects are identical - - def sizeof(self): - """ Recursively computes the size in bytes of the given Dynamic Object """ - sz = 0 - values = self.__dict__.values() - for val in values: - if isinstance(val, Object): sz += val.sizeof() - elif isinstance(val, np.ndarray): sz += val.nbytes - elif hasattr(val, 'dtype') and hasattr(val.dtype, 'itemsize'): sz += val.dtype.itemsize - else: sz += sys.getsizeof(val) - return sz - - # Automatic methods for accessing meta-data - getters = ['__object_name', '__revision_number', '__revision_id', '__revision_source', '__revision_tag', '__time_created'] - def getObjectName(self): return self.__class__.meta_attributes['__object_name'] - def getRevisionNumber(self): return self.__class__.meta_attributes['__revision_number'] - def getRevisionId(self): return self.__class__.meta_attributes['__revision_id'] - def getRevisionSource(self): return self.__class__.meta_attributes['__revision_source'] - def getRevisionTag(self): return self.__class__.meta_attributes['__revision_tag'] - def getTimeCreated(self): return getattr(self, "__time_created") - - """ - __getters = [('ObjectName', getObjectName), ('RevisionNumber', getRevisionNumber), - ('RevisionId', getRevisionId), ('RevisionSource', getRevisionSource), - ('RevisionTag', getRevisionTag)] - def __repr__(self): - meta_atts = repr([(x[0], x[1](self)) for x in Object.__getters]) - atts = repr(self.__dict__) - return "Object(%s, %s)"%(atts, meta_atts) - """ - - -class SignatureException(Exception): - """ Exception thrown when a data or method signature is unknown or invalid - for a particular Object. """ - def __init__(self, value): self.value = value - def __str__(self): return repr(self.value) - -class _IDLTag(object): - """ IDLTag (aka Interface Definition Language Tag) is an abstract helper class - used by the Factory to define built-in tags used - specifically for our IDL """ - def __init__(self, yamlString): - self.yamlString = yamlString - def __repr__(self): - return self.yamlString - -class _Reference(_IDLTag): - """ Helper class for Factory: Objects can be composed - of other objects, requiring a Reference to the other object. """ - def __repr__(self): - return "Ref(%s)"%(self.yamlString,) - -class _Method(_IDLTag): - """ Helper class for Factory: Objects have methods - associated with them - this tag tells the Factory that a method - signature follows (in dict format) """ - def __repr__(self): - return "Method(%r)"%(self.yamlString,) - -class Binary(Object): - def __init__(self, binary_type, value=None): - self.binary_type = binary_type - self.value = value - -import Lookup - -class BuiltinDtype(_IDLTag): - """ Helper class for Factory: Object parameters each - have a certain data type (either dtype.xxxx for numpy compatible data - types, or one of the generic python data types (i.e. int, bool, str...) - - __addYamlConstructor in Factory registers all of the tags - listed as keys in the dtypes dictionary.""" - - def __init__(self, yamlString, tag=None): - self.tag = tag[1:] - super(BuiltinDtype, self).__init__(yamlString) - #print self.tag - try: self.dtype = Lookup.numpy_dtypes[self.tag] - except KeyError: self.dtype = Lookup.builtin_objects[self.tag] - - def __repr__(self): - return "_BuiltinType(%s,%s)"%(self.yamlString, self.tag) - -# Register hexadecimal representation of numpy dtypes in YAML - -class _Parameter: - """ Helper class for Factory: Contains the name, default - value, and length (if an array) of an object initialization parameter. """ - - def __init__(self, name, hasDefault=False, default=None, length=None, classType=None): - self.name = name - self.hasDefault = hasDefault - self.default = default - self.length = length - if isinstance(classType, None.__class__) and not isinstance(default, None.__class__): - self.classType = default.__class__ - else: - self.classType = classType - -class _UnresolvedType: - """ Used to indicate a data type which has not yet been parsed (i.e. for - recursive data-types. """ - - def __init__(self, yamlObject): - # Either the name of the class we couldn't resolve, or a dictionary - # containing the name and a default value - self.yamlObject = yamlObject - -class UnresolvedTypeException(Exception): - """ Raised when a !ref tag is used, but the reference cannot be resolved """ - pass - -def get_class(kls): - """ Returns a pointer to the class instance with the name kls - Function acquired from http://stackoverflow.com/questions/452969/ """ - parts = kls.split('.') - module = ".".join(parts[:-1]) - m = __import__( module ) - for comp in parts[1:]: - m = getattr(m, comp) - return m - -# Aliased constructor & representer adders for easily swapping between Ordered and non-Ordered: -def add_constructor(tag, constructor): - #yaml.add_constructor(tag, constructor) - OrderedYAML.Loader.add_constructor(tag, constructor) -def add_representer(cls, representer): - #yaml.add_representer(cls, representer) - OrderedYAML.Dumper.add_representer(cls, representer) - -# Implicit constructor for _Reference objects using the !ref tag: -def __ref_constructor(loader, node): - if isinstance(node, yaml.nodes.MappingNode): - return _Reference(loader.construct_mapping(node)) - else: - return _Reference(loader.construct_scalar(node)) -add_constructor(u'!ref', __ref_constructor) - -# Method constructor using !method tag: -def __method_constructor(loader, node): - if isinstance(node, yaml.nodes.MappingNode): - return _Method(loader.construct_mapping(node)) - else: - return _Method(loader.construct_scalar(node)) -add_constructor(u'!method', __method_constructor) - -# Generic constructor for any _BuiltinDtype -def __dtype_constructor(loader, node): - if isinstance(node, yaml.nodes.SequenceNode): - ret = BuiltinDtype(loader.construct_sequence(node), tag=node.tag) - elif isinstance(node, yaml.nodes.MappingNode): - ret = BuiltinDtype(loader.construct_mapping(node), tag=node.tag) - else: - ret = BuiltinDtype(loader.construct_scalar(node), tag=node.tag) - return ret - -# Register YAML constructors for each builtin type: -for dtype in Lookup.numpy_dtypes.keys() + Lookup.builtin_objects.keys(): - add_constructor(u'!%s'%(dtype,), __dtype_constructor) - -class FactoryLoader(OrderedYAML.Loader): - """ A YAML Loader specifically designed to load YAML object definitions - (as opposed to actual instances of the objects) """ - - def construct_yaml_timestamp(self, node): - """ Make empty timestamps (None/null) acceptable, otherwise parse the timestamp """ - if node.value == u'': - name = 'YAML_DEFN_LOADED_INCORRECTLY' # in case we forget to fix the name... - return _Parameter(name, hasDefault=False, classType=datetime.datetime) - else: - return yaml.constructor.SafeConstructor.construct_yaml_timestamp(self, node) - -# Override default timestamp constructor: -FactoryLoader.add_constructor( - u'tag:yaml.org,2002:timestamp', - FactoryLoader.construct_yaml_timestamp -) - -import DynamicYAML -class Factory: - """ Load a YAML defined python class and create a class with initialization - provided by this factory. This is intended as an abstract class to be sub-classed - to enable complex initialization on object instantiation. - - Factory subclasses should override __buildClass().""" - - def __init__(self, dynamic_object=None, yaml=None, typeCheck='strong', parse=True, revision_dict=None): - if revision_dict != None: self.revision_dict = revision_dict # Remember for when we build each individual class - else: - self.revision_dict = {\ - "__revision_number": 0, - "__revision_id": 'unknown', - "__revision_source": 'unknown', - "__revision_tag": 'unknown'} - if parse: - if dynamic_object: - self.parse(dynamic_object, typeCheck=typeCheck) - else: - dyno = Object() - dyno.yaml = yaml - self.parse(dyno, typeCheck=typeCheck) - - def parse(self, dynamic_object, typeCheck='strong'): - """ - Initializer for a Factory, converting the given dynamic_object - containing a (text) YAML object definition into the corresponding class-type - with initializer. - - typeCheck parameter can be one of 'strong' or 'cast': - 'strong': Class initializer should raise a TypeError when given - anything but the correct type - 'cast': Class initializer should attempt to cast any input to the correct type - """ - - # Remember what kind of type-checking to do: - if typeCheck not in ['strong', 'cast']: - raise Exception('Incorrect input for typeCheck: %s\nExpected "strong" or "cast"'%(typeCheck)) - self.typeCheck = typeCheck - - # Get a list of the objects to build: - if isinstance(dynamic_object.yaml, list): - objects = dynamic_object.yaml - else: - objects = [dynamic_object] - - # Generate a dictionary of classes from the DynamicObjects given: - self.classes = dict() - for obj in objects: - - # This loader breaks nothing anymore #everything currently - loader = FactoryLoader(obj.yaml) - #loader = yaml.Loader(obj.yaml) - - # Dictionary with method and data signatures for the current object: - objDefn = [] - while loader.check_data(): - objDefn.append(loader.get_data()) - loader.dispose() - - # Parse the dictionary into a class definition: - objClass = self.__buildClass(objDefn) - self.classes.update(objClass) - - def parseMethodSignature(self, sigName, methDict): - """ Returns the python method corresponding to the given signature - (given signature should be in the loaded YAML dict format. - - Override this method for recognizing complex method signatures. """ - - raise SignatureException("Object abstract base class doesn't support any method signatures.") - - def parseDataSignature(self, sigName, sig): - """ Returns the Parameter object corresponding to the given signature. - - This method should be overridden for recognizing complex data signatures - (don't forget to call super(sig) for built-in data types though!) """ - - # Is the object an array with explicit default elements?: - if isinstance(sig.yamlString, list): - #length = len(sig.yamlString) - if 'dtype' in sig.tag: - default = np.array(sig.yamlString, dtype=sig.dtype) - elif 'binary' == sig.tag: - default = Binary(sig.yamlString["type"]) - else: - default = sig.yamlString - return _Parameter(sigName, True, default, length=None) - - # Is the object an array with length and default value given?: - if isinstance(sig.yamlString, dict) and "len" in sig.yamlString.keys(): - length = sig.yamlString["len"] - - # Shape is given as something like [[],[]], not [2,2] - convert - if isinstance(length, list): - - def get_shape(lst): - """ Gets the shape of a list recursively filled with empty lists """ - if lst == []: return [0] - return [len(lst)] + get_shape(lst[0]) - - if len(length) > 0: - if isinstance(length[0], list): - length = get_shape(length) - else: - pass - else: - length = [0] # convert [] to [0] (numpy interprets [] as [1] for shapes) - - - if 'complex' in sig.tag: - imag = sig.yamlString["default"]["imag"] - real = sig.yamlString["default"]["real"] - default = sig.dtype(real) + sig.dtype(imag*1j) - elif 'binary' == sig.tag: - default = Binary(sig.yamlString["type"]) - else: - default = sig.dtype(sig.yamlString["default"]) - - return _Parameter(sigName, True, default, length) - - # The object is singular, with a given value: - if 'complex' in sig.tag: - imag = sig.yamlString["imag"] - real = sig.yamlString["real"] - default = sig.dtype(real) + sig.dtype(imag*1j) - return _Parameter(sigName, True, default) - elif 'binary' == sig.tag: - default = Binary(sig.yamlString["type"]) - return _Parameter(sigName, False, default, classType=Binary) - elif 'timestamp' in sig.tag: - if isinstance(sig.yamlString, dict): - if sig.tag in ['timestamp_picosecond', 'timestamp_ps']: - try: s = sig.yamlString['second'] - except KeyError: s = sig.yamlString['s'] - try: ps = sig.yamlString['picosecond'] - except KeyError: ps = sig.yamlString['ps'] - return _Parameter(sigName, True, PrecisionTime.psTime(s, ps)) - elif sig.tag in ['timestamp_nanosecond', 'timestamp_ns']: - try: s = sig.yamlString['second'] - except KeyError: s = sig.yamlString['s'] - try: ns = sig.yamlString['nanosecond'] - except KeyError: ns = sig.yamlString['ns'] - return _Parameter(sigName, True, PrecisionTime.nsTime(s, ns)) - else: - if sig.tag in ['timestamp_picosecond', 'timestamp_ps']: - return _Parameter(sigName, False, classType=PrecisionTime.psTime) - elif sig.tag in ['timestamp_nanosecond', 'timestamp_ns']: - return _Parameter(sigName, False, classType=PrecisionTime.nsTime) - else: - default = sig.dtype(sig.yamlString) - return _Parameter(sigName, True, default) # not binary - - - - def __parsePythonType(self, sigName, sig): - """ Returns a _Parameter object, similar to parseDataSignature, but - for a basic python type. """ - - if isinstance(sig, collections.OrderedDict): - default = dict(sig) # Type-check user-defined !!maps as dicts, not OrderedDicts. - else: - default = sig # The signature sig is the default value itself - return _Parameter(sigName, True, default) - - def __parseReferenceSignature(self, sigName, ref_object, objClasses): - """ Takes a reference object ref_object to be named sigName, and - produces a _Parameter object with default value of None. """ - - # List of names of classes we've created so far: - #print [x for x in objClasses] - names = objClasses.keys() - - if ref_object.yamlString in names: - defaultType = objClasses[ref_object.yamlString] - return _Parameter(sigName, classType=defaultType) - else: - try: - # Try to find the class type in globals: - className = objClasses[str(ref_object.yamlString)] - defaultType = get_class(className) - except (ValueError, KeyError): - defaultType = _UnresolvedType(ref_object.yamlString) - #raise NameError("Invalid reference to module %s"%(className,)) - - return _Parameter(sigName, classType=defaultType) - - def __buildInitializer(self, className, classData): - """ Constructs the initializer for an object which expects parameters - listed in classData as input upon initialization. """ - - # Type of type-checking to use: - strong = (self.typeCheck == 'strong') - #cast = (self.typeCheck == 'cast') - - def typeCheck(param, arg): - """ - Checks to see if the type of arg matches that of the corresponding param, - casting arg to the correct type if desired. - """ - if isinstance(arg, param.classType): return arg - if isinstance(arg, np.ndarray) and arg.dtype.type == param.classType: - if not param.hasDefault: return arg - if param.default.shape == (): return arg - if param.default.shape[-1] == 0: return arg - if arg.shape == param.default.shape: return arg - if isinstance(arg, None.__class__): return arg - if strong: - raise TypeError("Incorrect input type on strong type-checking."+\ - " Expected %s - got %s"%(param.classType,arg.__class__)) - else: - # If the parameter corresponding to the given argument has a non-NoneType default - # value, then attempt to cast the argument into the correct parameter type - if param.hasDefault and param.default != None: - if isinstance(param.default, np.ndarray): - return np.array(arg, dtype=param.default.dtype) - else: - return param.default.__class__(arg) - else: - return param.classType(arg) - - """ - attributes = {"__object_name": className, - "__revision_number": self.svn_revision_number, - "__revision_id": 'unknown', - "__revision_source": 'unknown', - "__revision_tag": 'unknown'} - """ - attributes = {} # Create new attributes dict for this particular class object - attributes.update(self.revision_dict) # Revision info now passed into the factory - attributes['__object_name'] = className - - def init(_self, *args, **kwargs): - """ Dynamically generated initializer. """ - - # meta-data goes in the class, not the objects (commented the following out): - """ - # Initialize automatic class data - for attr,value in attributes.items(): - try: - value = kwargs[attr] # Are we given a value to over-ride with? - del kwargs[attr] # Ignore the meta attribute later - except KeyError: - pass - setattr(_self, attr, value) - """ - - # Set default values first (assume no parameters): - for param in classData: - if param.length: - if isinstance(param.length, int): param.length = [param.length] - default = np.empty(param.length, dtype=param.classType) - if param.hasDefault: - # Initialize array with default array value given: - flatIter = default.flat - for i in range(len(flatIter)): - flatIter[i] = copy.deepcopy(param.default) - else: - # Initialize to None if no default given: - default.fill(None) - else: - default = param.default - setattr(_self, param.name, copy.deepcopy(default)) - - # Set attributes given by standard args: - for i in range(len(args)): - arg = typeCheck(classData[i], args[i]) - setattr(_self, classData[i].name, arg) - - # Set named attributes (given by dictionary kwargs): - for key,value in kwargs.items(): - - try: keyIndex = [param.name for param in classData].index(key) - except ValueError: - raise TypeError("'%s' is an invalid keyword argument"%(key,)) - arg = typeCheck(classData[keyIndex],value) - #setattr(_self, key, value) - setattr(_self, key, arg) - - - # Object instantiation / creation time (if not already present): - if not kwargs.has_key('__time_created'): - setattr(_self, "__time_created", np.float64(time.time())) - - return init, attributes - - def __findClass(self, className, localClasses): - """ Looks for the given className first in the given dictionary of localClasses - then in the global definitions, returning the corresponding class object. Raises - a KeyError if the class cannot be found. """ - - # If class definition was in the YAML file, extend that one: - if className in localClasses.keys(): - return localClasses[className] - - # Else try finding the class definition in our global scope: - try: classObj = get_class(className) - except KeyError: - raise KeyError("Class '%s' not found in given YAML scope or global scope."%(className,)) - return classObj - - def __buildClass(self, objDefn): - """ Takes an object definition list / dictionary objDefn (loaded from a YAML - object definition file) and creates a class, dynamically binding - method and data signatures to the new class. - - This method only performs a basic binding of method and data signatures to - the new class. Object(s) having more complex initialization requirements - should be given their own Factory subclass, overriding this - and other methods.""" - - # objDefn is a list of dictionaries found in the YAML file - build each one... - objClasses = dict() - objClassesRev = dict() - - # A list of all _Parameter objects created, used to resolve recursive - # or "tangled" data structures - allClassData = [] - - for document in objDefn: - # Each document can contain multiple objects - build each one. - # (NOTE: objects can cross reference each other in the same document - # need to resolve Reference objects as last step) - for objClassName in document.keys(): - - # The dictionary containing method & data signatures: - objDict = document[objClassName] - - # Extract data / attribute definitions (signatures) from the YAML dictionary - # as well as method signatures and which classes this class extends: - classData = [] - classMethods = dict() - classBases = [Object] - - # List structured documents result in a list of dicts each with one key: - if isinstance(objDict, list): keys = [param.keys()[0] for param in objDict] - # Otherwise the parameter names are just the keys of the dict - else: keys = objDict.keys() # if key not found, raises AttributeError - - for sigName in keys: - #print sigName - sig = objDict[sigName] - #for f in _BuiltinDtype.python_dtypes: print f.__class__ - if sigName == '__extends': - if isinstance(sig, str): - sig = [sig] - if isinstance(sig, list): - for className in sig: - newBase = self.__findClass(className, objClasses) - - # Remove Object extension if newBase extends it already: - if Object in classBases and Object in inspect.getmro(newBase): - classBases.remove(Object) - classBases += [newBase] - else: - raise TypeError("Incorrect format for extending classes - %s"%(sig,)) - elif isinstance(sig, BuiltinDtype): - classData.append(self.parseDataSignature(sigName, sig)) - elif isinstance(sig, Lookup.python_dtypes): - classData.append(self.__parsePythonType(sigName, sig)) - elif isinstance(sig, _Reference): - classData.append(self.__parseReferenceSignature(sigName, sig, objClasses)) - elif isinstance(sig, _Method): - classMethods[sigName] = self.parseMethodSignature(sigName, sig.yamlString) - elif isinstance(sig, (PrecisionTime.nsTime, PrecisionTime.psTime)): - classData.append(_Parameter(sigName, True, sig)) - elif isinstance(sig, _Parameter): # sig is already a parameter (we skipped a step) - sig.name = sigName # we didn't know the name during load time - fill that in now - classData.append(sig) - else: - msg = "Factory abstract base class doesn't " +\ - "support the following signature: %r \"%s\""%(sig.__class__,str(sig)) - print sig.__class__ - raise SignatureException(msg) - - # Built-in attribute for all Dynamic Objects: - classData.append(_Parameter('__time_created', classType=np.float64)) - - # Turn the object data / attributes into a usable __init__ method: - classMethods["__init__"], meta_attributes = self.__buildInitializer(objClassName, classData) - - # Keep a record of the _Parameters created for later type resolution - allClassData.extend(classData) - - """ - __automaticMethods = { - "getObjectName": lambda _self: getattr(_self, '__object_name'), - "getRevisionNumber": lambda _self: getattr(_self, '__revision_number'), - "getRevisionId": lambda _self: getattr(_self, '__revision_id'), - "getRevisionSource": lambda _self: getattr(_self, '__revision_source'), - "getRevisionTag": lambda _self: getattr(_self, '__revision_tag') - } - classMethods.update(__automaticMethods) - """ - - # Put the method signatures into a namespace for the new class, - # then dynamically build the class from this namespace. - classNamespace = classMethods - classNamespace["meta_attributes"] = meta_attributes - cls = type(str(objClassName), tuple(classBases), classNamespace) - objClasses[objClassName] = cls - objClassesRev['%s.%s'%(objClassName,cls.meta_attributes["__revision_number"])] = cls - - # Create and register a constructor (loading) and representer (dumping) for the new class cls - def construct_dynamic_object(loader, node): - kwargs = loader.construct_mapping(node) - # Remove revision control from loaded objects (info is in the class object!) - for arg in kwargs.keys(): - if arg in getattr(Object, 'getters') and arg != '__time_created': - del kwargs[arg] - return cls(**kwargs) - revision = cls.meta_attributes["__revision_number"] - DynamicYAML.Loader.add_constructor(u'!%s.%s'%(str(objClassName),revision), construct_dynamic_object) - - represent_dynamic_object = DynamicYAML.Dumper.represent_dynamic_object - DynamicYAML.Dumper.add_representer(cls, represent_dynamic_object) - - def findClass(className): - """ Search for the most recently added class object with given className """ - try: - return objClasses[className] # Look for reference to object in same YAML defn file: - except KeyError: - # Now look for reference to class object loaded from any YAML defn file, loading the - # most recent version / revision (number) of the definition - for dynClass in Object.dynamicClasses.keys()[::-1]: - if dynClass.startswith(className): - return Object.dynamicClasses[dynClass] - - # Still unresolved - raise exception: - allDynamicClasses = repr(objClasses.keys() + Object.dynamicClasses.keys()) - raise UnresolvedTypeException("Cannot resolve type '%s': Name not found in %s"%(className,allDynamicClasses)) - - - def resolve(param): - - # Reference is just a string - that's the class name: - if isinstance(param.classType.yamlObject, (str, unicode)): - className = str(param.classType.yamlObject) - param.classType = findClass(className) - return - - # Reference is a dict containing class name and / or default values: - if not isinstance(param.classType.yamlObject, dict): - raise UnresolvedTypeException("Cannot resolve reference of type '%s'"%(param.classType.yamlObject.__class__,)) - - # Definitely a dict: - refDict = param.classType.yamlObject - - # Determine the name of the class being referenced - try: - className = refDict["type"] - except KeyError: - raise KeyError("No 'type' key in reference dictionary for parameter '%s'"%(param.name,)) - - # Determine the class object corresponding to the class name - param.classType = findClass(className) - - try: - defaultParams = refDict["default"] - except KeyError: - defaultParams = None - - if defaultParams != None: - for sub_param in defaultParams: - if isinstance(sub_param.classType, _UnresolvedType): - resolve(sub_param) - param.default = param.classType( **defaultParams ) # Create the default object - param.hasDefault = True - else: - param.hasDefault = False # for good measure - - # Is it an object array?: - if "len" in refDict.keys(): - param.length = refDict["len"] - - # Resolve any unresolved data-types: - for param in allClassData: - if isinstance(param.classType, _UnresolvedType): - resolve(param) - - Object.dynamicClasses.update(objClassesRev) - return objClasses - -def load_defn(yaml): - """ Shortcut for producing a single DynamicObject class object from - the provided yaml definition in string format """ - return Factory(yaml=yaml).classes.values()[0] - - - diff --git a/schainpy/model/serializer/test/DynamicSerializer.py b/schainpy/model/serializer/test/DynamicSerializer.py deleted file mode 100644 index 34b0444..0000000 --- a/schainpy/model/serializer/test/DynamicSerializer.py +++ /dev/null @@ -1,69 +0,0 @@ -# -# rps 6/9/2014 -# mit haystack obs -# -# wrapper for Karl's code - -import DynamicObject # used for serial/deserial of complex python objects -import Serializer # used for serial/deserial of complex python - -# -class DynamicSerializer: - # - #------------------------------------------------------ - # - def __init__(self,which='yaml'): - # - # choices are: yaml, msgpack, hdf5, json - # - self.err_f = False - self.whichList = ['yaml', 'msgpack', 'hdf5', 'json'] # from Serialzer.py - self.err_f,self.serializer = self.initSerializer(which) - # - #------------------------------------------------------ - # - def initSerializer(self,which): - # - # calls REU student code that works but hasn't been walked-through - # it's a dynamic serializer not strictly a yaml serializer - # - err_f = False - match_f = False - serializer = None - ii = 0 - while ii < len(self.whichList): - if (self.whichList[ii] == which): - match_f = True - break - ii = ii + 1 - # end while - if not match_f: - err_f = True - else: - serializer = which - serializer = Serializer.serializers[serializer]() - - return err_f,serializer - # end initSerializer - # - # -------------------------------------------------- - # - def loads(self,element): # borrows name from json module (json - to - python) - retval = self.serializer.fromSerial(element) # de-serialize - return retval - # end loads - # - # -------------------------------------------------- - # - def dumps(self,element): # borrows name from json module (python - to - json) - retval = self.serializer.toSerial(element) # serialize - return retval - # end dumps - # - # -------------------------------------------------- - # -# end class DynamicSerializer - -if __name__ == "__main__": - DynamicSerializer() - print "DynamicSerializer ran" \ No newline at end of file diff --git a/schainpy/model/serializer/test/DynamicYAML.py b/schainpy/model/serializer/test/DynamicYAML.py deleted file mode 100644 index f74cf98..0000000 --- a/schainpy/model/serializer/test/DynamicYAML.py +++ /dev/null @@ -1,221 +0,0 @@ -''' -Module containing YAML Loader and Dumper for DynamicObjects -as well as built-in data types (numpy, PrecisionTime, datetime, Binary, ...) - -$Id$ -''' - -import yaml -import OrderedYAML -import DynamicObject -import binascii -import numpy as np -import PrecisionTime -import Lookup -import pysvn - -def load_defn(source, rev='head', repo=""): - """ Import YAML definition(s) from given 'source' SVN location - with specific revision number 'rev'. Returns a dict of the object - names -> class object instances. - - NOTE: Object defns with same name & revision number will conflict / - cause issues (regardless of svn location). """ - client = pysvn.Client() - - if rev == 'head': - #yaml = client.cat(source) - rev = client.info(repo).revision.number - - if source.startswith('http'): - yaml = client.cat("%s?p=%d"%(source, rev)) - else: - pysvn_rev = pysvn.Revision(pysvn.opt_revision_kind.number, rev) - yaml = client.cat(source, pysvn_rev) - - revision_dict = {\ - "__revision_number": rev, - "__revision_id": 'unknown', - "__revision_source": source, - "__revision_tag": 'unknown'} - - return DynamicObject.Factory(yaml=yaml, revision_dict=revision_dict).classes - -class Loader(OrderedYAML.Loader): - - def __init__(self, stream): - OrderedYAML.Loader.__init__(self, stream) - - def construct_object(self, node, deep=False): - """ Unresolved tags on mapping nodes come from un-imported YAML definitions - import it """ - resolved = node.tag in self.yaml_constructors - resolved = resolved or any([node.tag.startswith(x) for x in self.yaml_multi_constructors]) - if isinstance(node, yaml.nodes.MappingNode) and not resolved: - data = self.construct_mapping(self, node) - self.constructed_objects[node] = data - del self.recursive_objects[node] - if data.has_key('__revision_source'): - # TODO: Handle password authentication - client = pysvn.Client() - source = data['__revision_source'] - if source.startswith('http'): - rev = data['__revision_number'] - defn = client.cat("%s?p=%d"%(source, rev)) - else: - rev = pysvn.Revision(pysvn.opt_revision_kind.number, data['__revision_number']) - defn = client.cat(source, revision=rev) - DynamicObject.Factory(yaml=defn) # Register the object - - constructor = self.yaml_constructors["%s.%s"%(data['__revision_name'], data['__revision_number'])] - return constructor(node) - else: - raise Exception("Cannot load object with tag '%s' - cannot find YAML object definition (no __revision_source included)") - else: - return yaml.Loader.construct_object(self, node, deep=deep) - -class Dumper(OrderedYAML.Dumper): - - def __init__(self, stream, *args, **kwargs): - OrderedYAML.Dumper.__init__(self, stream, *args, **kwargs) - - def represent_dynamic_object(self, obj): - """ - Override the !!python/object:__main__.xxx syntax with - !ObjectName.zzz where zzz is the revision number of the Object obj - """ - - state = {} - state.update(obj.__dict__.items()) - state.update(obj.__class__.meta_attributes.items()) - name = obj.getObjectName() # obj.__class__.__name__ - revision = obj.getRevisionNumber() - return self.represent_mapping(u'!%s.%s' % (name, revision), state) - -# Dtypes to be stored as hex in YAML streams / strings -hex_dtypes = ['float', 'complex', 'half', 'single', 'double'] - -# Register hex constructors for the numpy / built-in dtypes: -dtypes = Lookup.numpy_dtypes - -# Inverse lookup for accessing tags given a class instance: -cls_dtypes = dict([(v,k) for (k,v) in dtypes.items()]) - -# Representer for numpy arrays: -def ndarray_representer(dumper, obj): - #if isinstance(obj, np.ndarray): - tag = 'dtype.'+obj.dtype.type.__name__ - hexlify = any([x in tag for x in hex_dtypes]) - np_ary = obj - #hex_ary = np.empty(np_ary.shape, dtype=yaml.nodes.ScalarNode) - np_flat, hex_flat = np_ary.flat, [] #hex_ary.flat - hex_flat.append(dumper.represent_sequence(u'tag:yaml.org,2002:seq', list(np_ary.shape), flow_style=True)) - if hexlify: - lst = [] - for i in range(len(np_flat)): - value = u'%s'%(np_flat[i],) - node = dumper.represent_scalar(u'tag:yaml.org,2002:str', value, style='') - lst.append(node) - hex_flat.append(yaml.nodes.SequenceNode(u'tag:yaml.org,2002:seq', lst, flow_style=True)) - lst = [] - for i in range(len(np_flat)): - if hexlify: value = u'%s'%(binascii.hexlify(np_flat[i]),) - else: value = u'%s'%(np_flat[i],) - node = dumper.represent_scalar(u'tag:yaml.org,2002:str', value, style='') - if hexlify: lst.append(node) - else: hex_flat.append(node) - if hexlify: hex_flat.append(yaml.nodes.SequenceNode(u'tag:yaml.org,2002:seq', lst, flow_style=True)) - return yaml.nodes.SequenceNode(u'!%s'%(tag,), hex_flat, flow_style=True) -Dumper.add_representer(np.ndarray, ndarray_representer) - -# Constructor for ndarrays with arbitrary (specified) dtype: -def ndarray_constructor(loader, node, dtype, hexlify=False): - shape = loader.construct_sequence(node.value.pop(0)) - np_ary = np.empty(shape, dtype=dtype) - np_flat = np_ary.flat # Flat iterator - if hexlify: - node.value[1].tag = node.tag - node = node.value[1] # only look at hexlified values - for i in range(len(node.value)): - # Over-ride the 'tag:yaml.org,2002:str' tag with correct data type - node.value[i].tag = node.tag - value = loader.construct_object(node.value[i]) - #if hexlify: - # value = binascii.unhexlify(value) - # value = np.frombuffer(value, dtype=dtype) - np_flat[i] = value - return np_ary - -class __dtype_con: - - def __init__(self, tag): - # Whether or not to convert to hex: - hexlify = any([x in tag for x in hex_dtypes]) - dtype = dtypes[tag] - - # Mutable list containing constructor & representer info - self.fncn_attributes = [tag, hexlify, dtype] - - def dtype_constructor(loader, node): - tag, hexlify, dtype = self.fncn_attributes - if isinstance(node, yaml.nodes.SequenceNode): - return ndarray_constructor(loader, node, dtype, hexlify=hexlify) - else: # isinstance(node, yaml.nodes.ScalarNode): - value = loader.construct_scalar(node) - dtype = dtypes[node.tag[1:]] - if hexlify: - value = binascii.unhexlify(value) - value = np.frombuffer(value, dtype=dtype)[0] - else: - value = dtype(value) - return value - - def dtype_representer(dumper, obj): - tag, hexlify, dtype = self.fncn_attributes - if isinstance(obj, float): obj = np.float64(obj) - if hexlify: value = u'%s'%(binascii.hexlify(obj),) - else: value = u'%s'%(obj,) - try: tag = u'!%s'%(cls_dtypes[obj.__class__]) # 'dtype.'+obj.__class__.__name__ # bullshit... - except KeyError: tag = '' - node = dumper.represent_scalar(tag, value, style='') - return node - - self.dtype_constructor = dtype_constructor - self.dtype_representer = dtype_representer - -keys = [x for x in dtypes.keys() if x != 'dtype.int' and x != 'dtype.bool'] -print keys - -n = len(keys) -print n -i=0 - -for tag in keys: - dtype = __dtype_con(tag) - dtype_constructor = dtype.dtype_constructor - dtype_representer = dtype.dtype_representer - Loader.add_constructor(u'!%s'%(tag,), dtype_constructor) - Dumper.add_representer(dtypes[tag], dtype_representer) - -# Precision time constructors & representers: -def ns_rep(dumper, obj): - state = {'second': obj.__dict__['second'], 'nanosecond': obj.__dict__['nanosecond']} - return dumper.represent_mapping(u'!timestamp_ns', state) -def ps_rep(dumper, obj): - state = {'second': obj.__dict__['second'], 'picosecond': obj.__dict__['picosecond']} - return dumper.represent_mapping(u'!timestamp_ps', state) -def ns_con(loader, node): return PrecisionTime.nsTime(**loader.construct_mapping(node)) -def ps_con(loader, node): return PrecisionTime.psTime(**loader.construct_mapping(node)) - -Dumper.add_representer(PrecisionTime.nsTime, ns_rep) -Dumper.add_representer(PrecisionTime.psTime, ps_rep) -Loader.add_constructor(u'!timestamp_ns', ns_con) -Loader.add_constructor(u'!timestamp_nanosecond', ns_con) -Loader.add_constructor(u'!timestamp_ps', ps_con) -Loader.add_constructor(u'!timestamp_picosecond', ps_con) - -# Binary object constructor & representer: -def bin_rep(dumper, obj): return dumper.represent_mapping(u'!binary', obj.__dict__) -def bin_con(loader, node): return DynamicObject.Binary(**loader.construct_mapping(node)) -Dumper.add_representer(DynamicObject.Binary, bin_rep) -Loader.add_constructor(u'!binary', bin_con) - diff --git a/schainpy/model/serializer/test/Lookup.py b/schainpy/model/serializer/test/Lookup.py deleted file mode 100644 index 53ab278..0000000 --- a/schainpy/model/serializer/test/Lookup.py +++ /dev/null @@ -1,62 +0,0 @@ -''' -Helper module for DynamicObject module - contains dictionaries -of data types built-in to our YAML IDL, converting backing and forth between -strings / YAML tags and python class instances. - -$Id$ -''' - -import datetime -import numpy as np -import PrecisionTime -import DynamicObject -Binary = DynamicObject.Binary -import platform -import collections - -# Implicit Types: -python_dtypes = tuple([bool,int,long,float,str,datetime.datetime,list, - set,dict,tuple,unicode]) - -# Numpy Data-types: -numpy_dtypes = {'dtype.bool': bool, 'dtype.int': np.int, 'dtype.int8': np.int8, - 'dtype.int16': np.int16, 'dtype.int32': np.int32, 'dtype.int64': np.int64, - 'dtype.uint8': np.uint8, 'dtype.uint16': np.uint16, 'dtype.uint32': np.uint32, - 'dtype.uint64': np.uint64, 'dtype.float': np.float, 'dtype.float16': np.float16, - 'dtype.float32': np.float32, 'dtype.float64': np.float64, 'dtype.complex': np.complex, - 'dtype.complex64': np.complex64, 'dtype.complex128': np.complex128, - 'dtype.byte': np.byte, 'dtype.short': np.short, 'dtype.intc': np.intc, - 'dtype.longlong': np.longlong, 'dtype.intp': np.intp, 'dtype.ubyte': np.ubyte, - 'dtype.ushort': np.ushort, 'dtype.uintc': np.uintc, 'dtype.uint': np.uint, - 'dtype.uintc': np.uintc, 'dtype.uint': np.uint, 'dtype.ulonglong': np.ulonglong, - 'dtype.uintp': np.uintp, 'dtype.half': np.half, 'dtype.single': np.single, - 'dtype.double': np.double, 'dtype.longfloat': np.longfloat, - 'dtype.csingle': np.csingle, 'dtype.clongfloat': np.clongfloat, 'dtype.long': np.long} - -if platform.architecture()[0] != '32bit': # 64bit - certain numpy types exist - numpy_dtypes.update({'dtype.float128': np.float128, 'dtype.complex256': np.complex256}) - bit32 = False -else: - bit32 = True -#else: # 32 bit - fix 32 bit integer issue. -# np.int32 = np.int -# bit32 = True - -# Built-in objects: -builtin_objects = {'binary': Binary, 'nsTime': PrecisionTime.nsTime, 'psTime': PrecisionTime.psTime, - 'timestamp_ns': PrecisionTime.nsTime, 'timestamp_ps': PrecisionTime.psTime, - 'timestamp_nanosecond': PrecisionTime.nsTime, 'timestamp_picosecond': PrecisionTime.psTime, - 'datetime': datetime.datetime, 'Binary': Binary} - -builtin_objects_simple = {'nsTime': PrecisionTime.nsTime, 'psTime': PrecisionTime.psTime, - 'binary': Binary, 'datetime': datetime.datetime, - 'Binary': Binary} - -# Inverse lookup for accessing tags given a class instance: -cls_dtypes = dict([(v,k) for (k,v) in numpy_dtypes.items()]) -obj_dtypes = dict([(v,k) for (k,v) in builtin_objects_simple.items()]) - -# Pointer to the list of all Object classes created, as located in the Object module / class: -dynamicClasses = DynamicObject.Object.dynamicClasses - - diff --git a/schainpy/model/serializer/test/OrderedYAML.py b/schainpy/model/serializer/test/OrderedYAML.py deleted file mode 100644 index 711fd4f..0000000 --- a/schainpy/model/serializer/test/OrderedYAML.py +++ /dev/null @@ -1,89 +0,0 @@ -''' -A YAML Loader and Dumper which provide ordered dictionaries in place -of dictionaries (to keep the order of attributes as -found in the original YAML object file). - -This module is modified from a submission on pyyaml.org: -http://pyyaml.org/attachment/ticket/161/use_ordered_dict.py - -$Id$ -''' - -import yaml -import collections - -class Loader(yaml.loader.Loader): - """ YAML Loader producing OrderedDicts in place of dicts """ - - def __init__(self, stream): - yaml.loader.Loader.__init__(self, stream) - - def construct_ordered_mapping(self, node, deep=False): - """ Replacement mapping constructor producing an OrderedDict """ - if not isinstance(node, yaml.MappingNode): - raise yaml.constructor.ConstructorError(None, None, - "expected a mapping node, but found %s" % node.id, - node.start_mark) - mapping = collections.OrderedDict() - for key_node, value_node in node.value: - key = self.construct_object(key_node, deep=deep) - if not isinstance(key, collections.Hashable): - raise yaml.constructor.ConstructorError("while constructing a mapping", node.start_mark, - "found unhashable key", key_node.start_mark) - value = self.construct_object(value_node, deep=deep) - mapping[key] = value - return mapping - # yaml.constructor.BaseConstructor.construct_mapping = construct_ordered_mapping - - def construct_mapping(self, node, deep=False): - return self.construct_ordered_mapping(node, deep=deep) - - def construct_yaml_map_with_ordered_dict(self, node): - data = collections.OrderedDict() - yield data - value = self.construct_mapping(node) - data.update(value) - -class Dumper(yaml.dumper.Dumper): - """ YAML Dumper producing documents from OrderedDicts """ - - def __init__(self, stream, *args, **kwargs): - yaml.dumper.Dumper.__init__(self, stream, *args, **kwargs) - - def represent_ordered_mapping(self, tag, mapping, flow_style=None): - """ Replacement mapping representer for OrderedDicts """ - value = [] - node = yaml.MappingNode(tag, value, flow_style=flow_style) - if self.alias_key is not None: - self.represented_objects[self.alias_key] = node - best_style = True - if hasattr(mapping, 'items'): - mapping = list(mapping.items()) - for item_key, item_value in mapping: - node_key = self.represent_data(item_key) - node_value = self.represent_data(item_value) - if not (isinstance(node_key, yaml.ScalarNode) and not node_key.style): - best_style = False - if not (isinstance(node_value, yaml.ScalarNode) and not node_value.style): - best_style = False - value.append((node_key, node_value)) - if flow_style is None: - if self.default_flow_style is not None: - node.flow_style = self.default_flow_style - else: - node.flow_style = best_style - return node - # yaml.representer.BaseRepresenter.represent_mapping = represent_ordered_mapping - - def represent_mapping(self, tag, mapping, flow_style=None): - return self.represent_ordered_mapping(tag, mapping, flow_style=flow_style) - -# Loader.add_constructor( -# u'tag:yaml.org,2002:map', -# Loader.construct_yaml_map_with_ordered_dict -# ) -# -# Dumper.add_representer( -# collections.OrderedDict, -# yaml.representer.SafeRepresenter.represent_dict -# ) diff --git a/schainpy/model/serializer/test/PrecisionTime.py b/schainpy/model/serializer/test/PrecisionTime.py deleted file mode 100644 index 205cf5b..0000000 --- a/schainpy/model/serializer/test/PrecisionTime.py +++ /dev/null @@ -1,211 +0,0 @@ -#!/usr/local/midas/bin/python - -"""PrecisionTime.py is a collection of python classes to manipulate times with high -precision using integer logic. - -Written by "Bill Rideout":mailto:wrideout@haystack.mit.edu May 24, 2007 - -$Id$ -""" -import types - -class nsTime: - """nsTime is a class to handle times given as UT second (integer) and nanosecond (integer) - - If nanosecond > 1E9, seconds will be added to second - """ - - def __init__(self, second, nanosecond): - self.second = int(second) - if self.second < 0: - raise ValueError, 'seconds must be greater than 0, not %i' % (self.second) - nanosecond = long(nanosecond) - if nanosecond < 0: - raise ValueError, 'nanoseconds must be greater 0, not %i' % (nanosecond) - addSec = nanosecond / 1000000000 - if addSec > 0: - self.second += addSec - self.nanosecond = nanosecond % 1000000000 - self.totalNS = long(self.nanosecond) + long(self.second) * 1000000000 - - - def __add__(self, other): - """__add__ another nsTime to this one and return a new one as result - """ - nsResult = self.nanosecond + other.nanosecond - addSec = int(nsResult / 1000000000) - newSec = self.second + other.second + addSec - newNS = nsResult % 1000000000 - return(nsTime(newSec, newNS)) - - - def increase(self, other): - """increase adds other to self, changing self (rather than creating a new object) - """ - nsResult = self.nanosecond + other.nanosecond - addSec = int(nsResult / 1000000000) - self.second = self.second + other.second + addSec - self.nanosecond = nsResult % 1000000000 - self.totalNS = long(self.nanosecond) + long(self.second) * 1000000000 - - - def __sub__(self, other): - """__sub__ another nsTime from this one and return a new one as result - """ - nsResult = self.nanosecond - other.nanosecond - if nsResult < 0: - addSec = 1 - nsResult += 1000000000 - else: - addSec = 0 - newSec = (self.second - other.second) - addSec - return(nsTime(newSec, nsResult)) - - - def multiply(self, factor): - """multiply this nsTime times an integer - """ - if type(factor) not in (types.IntType, types.LongType): - raise ValueError, 'Illegal type %s passed into nsTime.multiply' % (str(type(factor))) - newTotalNS = self.totalNS * factor - newSeconds = int(newTotalNS / 1000000000) - newNanoseconds = int(newTotalNS - (newSeconds * 1000000000)) - return(nsTime(newSeconds, newNanoseconds)) - - - def integerDivision(self, other): - """integerDivision returns the total number of other nsTimes that fit in self - """ - return(int(self.totalNS / other.totalNS)) - - def getUnixTime(self): - """ getUnixTime() returns a Unix style time as a float. """ - return(float(self.second) + float(self.nanosecond)/1.0e9) - - def __mod__(self, other): - """__mod__ implements self % other. - """ - if type(other) in (types.IntType, types.LongType): - return self.totalNS % other - else: - return self.totalNS % other.totalNS - - def __eq__(self, other): - """ equality of two nsTime objects """ - if not (hasattr(other, 'second') and hasattr(other, 'nanosecond')): return False - return self.__cmp__(other) == 0 - - def __cmp__(self, other): - """compare two nsTime objects - """ - result = cmp(self.second, other.second) - if result != 0: - return(result) - - return(cmp(self.nanosecond, other.nanosecond)) - - - def __str__(self): - return '%d.%09d' % (self.second, self.nanosecond) - - -class psTime: - """psTime is a class to handle times given as UT second (integer) and picosecond (integer) - - If picosecond > 1E12, seconds will be added to second - """ - - def __init__(self, second, picosecond): - self.second = int(second) - if self.second < 0: - raise ValueError, 'seconds must be greater than 0, not %i' % (self.second) - picosecond = long(picosecond) - if picosecond < 0: - raise ValueError, 'picoseconds must be greater 0, not %i' % (picosecond) - addSec = picosecond / 1000000000000 - if addSec > 0: - self.second += addSec - self.picosecond = picosecond % 1000000000000 - self.totalPS = long(self.picosecond) + long(self.second) * 1000000000000 - - - def __add__(self, other): - """__add__ another psTime to this one and return a new one as result - """ - psResult = self.picosecond + other.picosecond - addSec = int(psResult / 1000000000000) - newSec = self.second + other.second + addSec - newPS = psResult % 1000000000000 - return(psTime(newSec, newPS)) - - - def increase(self, other): - """increase adds other to self, changing self (rather than creating a new object) - """ - psResult = self.picosecond + other.picosecond - addSec = int(psResult / 1000000000000) - self.second = self.second + other.second + addSec - self.picosecond = psResult % 1000000000000 - self.totalPS = long(self.picosecond) + long(self.second) * 1000000000000 - - - def __sub__(self, other): - """__sub__ another psTime from this one and return a new one as result - """ - psResult = self.picosecond - other.picosecond - if psResult < 0: - addSec = 1 - psResult += 1000000000000 - else: - addSec = 0 - newSec = (self.second - other.second) - addSec - return(psTime(newSec, psResult)) - - - def multiply(self, factor): - """multiply this psTime times an integer - """ - if type(factor) not in (types.IntType, types.LongType): - raise ValueError, 'Illegal type %s passed into psTime.multiply' % (str(type(factor))) - newTotalPS = self.totalPS * factor - newSeconds = int(newTotalPS / 1000000000000) - newPicoseconds = int(newTotalPS - (newSeconds * 1000000000000)) - return(psTime(newSeconds, newPicoseconds)) - - - def integerDivision(self, other): - """integerDivision returns the total number of other psTimes that fit in self - """ - return(int(self.totalPS / other.totalPS)) - - def getUnixTime(self): - """ getUnixTime() returns a Unix style time as a float. """ - return(float(self.second) + float(self.picosecond)/1.0e12) - - def __mod__(self, other): - """__mod__ implements self % other. - """ - if type(other) in (types.IntType, types.LongType): - return self.totalPS % other - else: - return self.totalPS % other.totalPS - - def __eq__(self, other): - """ equality of two psTime objects """ - if not (hasattr(other, 'second') and hasattr(other, 'picosecond')): return False - return self.__cmp__(other) == 0 - - def __cmp__(self, other): - """compare two psTime objects - """ - result = cmp(self.second, other.second) - if result != 0: - return(result) - - return(cmp(self.picosecond, other.picosecond)) - - - def __str__(self): - return '%d.%12d' % (self.second, self.picosecond) - - diff --git a/schainpy/model/serializer/test/Serializer.py b/schainpy/model/serializer/test/Serializer.py deleted file mode 100644 index 9201f6c..0000000 --- a/schainpy/model/serializer/test/Serializer.py +++ /dev/null @@ -1,373 +0,0 @@ -''' -Module containing classes with serialization and de-serialization services. - -$Id$ -''' - -import Lookup -import numpy as np -import zlib -import binascii -import yaml -import DynamicObject -import DynamicYAML -import PrecisionTime -import datetime -import re -import os -#import json -import jsonpickle -import jpickle -import h5py -import msgpack - -class CompressionException(Exception): pass - -class Serializer: - """ Base class for pickle-like serialization - of DynamicObjects (with compression available) """ - - def __init__(self): - pass - - def dump(self, obj, file_name, compression=None): - """ Dumps obj to file_name, serializing the obj with toSerial() """ - string = self.dumps(obj, compression) - open(file_name, 'w').write(string) - - def dumps(self, obj, compression=None): - """ Returns serialized string representing obj, using toSerial() to serialize """ - if compression == 'gzip': - return zlib.compress(self.toSerial(obj)) - elif compression in [None, '']: - return self.toSerial(obj) - else: - raise CompressionException("Invalid decompression type '%r'"%(compression,)) - - def load(self, file_name, compression=None): - """ Returns the Object located in file_name, using fromSerial() to deserialize """ - string = open(file_name, 'r').read() - return self.loads(string, compression) - - def loads(self, string, compression=None): - """ Returns the Object serialized as the given string """ - if compression == 'gzip': - return self.fromSerial(zlib.decompress(string)) - elif compression in [None, '']: - return self.fromSerial(string) - else: - raise CompressionException("Invalid compression type '%r'"%(compression,)) - - def fromSerial(self, string): - """ Deserializes the given string """ - return string - - def toSerial(self, obj): - """ Serializes the given object """ - return repr(obj) - -class YAMLSerializer(Serializer): - """ Serializes a Object to/from YAML format """ - - def __init__(self): - Serializer.__init__(self) - - def fromSerial(self, string): - loader = DynamicYAML.Loader - return yaml.load(string, Loader=loader) - - def toSerial(self, obj): - dumper = DynamicYAML.Dumper - return yaml.dump(obj, Dumper=dumper) - -# Regular expression taken from yaml.constructor.py -timestamp_regexp_str = str(\ - ur'^(?P[0-9][0-9][0-9][0-9])' - ur'-(?P[0-9][0-9]?)' - ur'-(?P[0-9][0-9]?)' - ur'(?:(?:[Tt]|[ \t]+)' - ur'(?P[0-9][0-9]?)' - ur':(?P[0-9][0-9])' - ur':(?P[0-9][0-9])' - ur'(?:\.(?P[0-9]*))?' - ur'(?:[ \t]*(?PZ|(?P[-+])(?P[0-9][0-9]?)' - ur'(?::(?P[0-9][0-9]))?))?)?$') -timestamp_regexp = re.compile(timestamp_regexp_str, re.X) - -def construct_timestamp(value): - """ Taken & modified from yaml.constructor.py """ - - match = timestamp_regexp.match(value) - #print "&%s&"%(value,) - #print timestamp_regexp_str - values = match.groupdict() - year = int(values['year']) - month = int(values['month']) - day = int(values['day']) - if not values['hour']: - return datetime.date(year, month, day) - hour = int(values['hour']) - minute = int(values['minute']) - second = int(values['second']) - fraction = 0 - if values['fraction']: - fraction = values['fraction'][:6] - while len(fraction) < 6: - fraction += '0' - fraction = int(fraction) - delta = None - if values['tz_sign']: - tz_hour = int(values['tz_hour']) - tz_minute = int(values['tz_minute'] or 0) - delta = datetime.timedelta(hours=tz_hour, minutes=tz_minute) - if values['tz_sign'] == '-': - delta = -delta - data = datetime.datetime(year, month, day, hour, minute, second, fraction) - if delta: - data -= delta - return data - -class MessagePackSerializer(Serializer): - """ Serializes a Object to/from MessagePack format """ - - def __fromSerial(self, msg_dict): - if not isinstance(msg_dict, (dict, list, tuple)): - return msg_dict # msg_dict is a value - return it - if isinstance(msg_dict, dict) and msg_dict.has_key('__meta_attributes'): - meta_attr = msg_dict['__meta_attributes'] - msg_dict.pop('__meta_attributes') - if meta_attr.has_key('type'): - if meta_attr['type'] == 'datetime': - return construct_timestamp(str(msg_dict['ts'])) - elif meta_attr['type'] == 'nsTime': - msg_dict.pop('totalNS') - elif meta_attr['type'] == 'psTime': - msg_dict.pop('totalPS') - try: dtype = Lookup.cls_dtypes[meta_attr['type']] - except KeyError: dtype = Lookup.builtin_objects[meta_attr['type']] - return dtype(**msg_dict) - else: - for key in msg_dict.keys(): - msg_dict[key] = self.__fromSerial(msg_dict[key]) - cls = Lookup.dynamicClasses['%s.%s'%(meta_attr['__object_name'],meta_attr['__revision_number'])] - return cls(**msg_dict) - elif msg_dict == (): - return [] - elif isinstance(msg_dict[0], str) and msg_dict[1] in Lookup.numpy_dtypes and\ - isinstance(msg_dict, tuple) and len(msg_dict) == 2: - value = binascii.unhexlify(msg_dict[0]) - return np.frombuffer(value, dtype=Lookup.numpy_dtypes[msg_dict[1]])[0] - - tup = isinstance(msg_dict, tuple) - if tup and len(msg_dict) > 1 and msg_dict[0] in Lookup.numpy_dtypes.keys(): - msg_flat = list(msg_dict) - dtypeName = msg_flat.pop(0) - dtype = Lookup.numpy_dtypes[dtypeName] - shape = msg_flat.pop(0) - obj = np.empty(shape, dtype=dtype) - np_flat = obj.flat - for i in range(len(np_flat)): - if isinstance(msg_flat[i], float): - value = msg_flat[i] - else: - value = self.__fromSerial((msg_flat[i], dtypeName)) - np_flat[i] = value - return obj - else: - return msg_dict - - def fromSerial(self, string): - msg_dict = msgpack.unpackb(string) - return self.__fromSerial(msg_dict) - - def __toSerial(self, obj): - - if isinstance(obj, (PrecisionTime.nsTime, PrecisionTime.psTime, DynamicObject.Binary, datetime.datetime)): - msg_dict = {} - if isinstance(obj, datetime.datetime): - msg_dict['ts'] = obj.isoformat(' ') - else: - msg_dict.update(obj.__dict__) - msg_dict['__meta_attributes'] = {'type': obj.__class__.__name__} - return msg_dict - elif isinstance(obj, DynamicObject.Object): - msg_dict = {} - for key, value in obj.__dict__.items(): - msg_dict[key] = self.__toSerial(value) - - msg_dict['__meta_attributes'] = obj.__class__.meta_attributes - return msg_dict - elif isinstance(obj, np.ndarray): - np_flat = obj.flat - msg_flat = [] - msg_flat.append(Lookup.cls_dtypes[obj.dtype.type]) # dtype is first element - msg_flat.append(obj.shape) # shape of array is second element - for i in range(len(np_flat)): - toSer = self.__toSerial(np_flat[i]) - if isinstance(toSer, tuple): - msg_flat.append(toSer[0]) - else: - msg_flat.append(toSer) - return list(msg_flat) - - is_builtin = obj.__class__ in Lookup.numpy_dtypes.values() - #is_python = isinstance(obj, Lookup.python_dtypes) - if is_builtin: # and not is_python: - try: - #print obj.__class__ - msg_dict = (binascii.hexlify(obj), Lookup.cls_dtypes[obj.__class__]) - return msg_dict - except TypeError: # numpy dtype is a built-in python type... force the hexlify: - if not Lookup.bit32: - if obj.__class__ == int: return (binascii.hexlify(np.int64(obj)), 'dtype.int64') - elif obj.__class__ == float: return (binascii.hexlify(np.float64(obj)), 'dtype.float64') - else: - #print np.int32(obj).__class__, obj.__class__ - if obj.__class__ == int: return (binascii.hexlify(np.int32(obj)), 'dtype.int32') - elif obj.__class__ == float: return (binascii.hexlify(np.float32(obj)), 'dtype.float32') - raise - else: - return obj - - def toSerial(self, obj): - #if Lookup.bit32 and np.int32 != np.int: np.int32 = np.int - toSer = self.__toSerial(obj) - #print toSer - value = msgpack.packb(toSer) - return value - -class HDF5Serializer(Serializer): - """ Serializes a Object to/from HDF5 format """ - - tmp_num = 0 - - def __fromSerial(self, grp): - - if isinstance(grp, h5py.Dataset): - return grp.value - - elif isinstance(grp, h5py.Group) and '__type' in grp.keys(): - typ = grp['__type'].value - if typ == 'datetime': - return construct_timestamp(str(grp['ts'].value)) - elif typ == '_null': - return None - elif typ == 'tuple': - return tuple(grp['tuple']) - elif typ == 'empty_list': - return [] - try: cls = Lookup.builtin_objects_simple[typ] - except KeyError: cls = Lookup.dynamicClasses[typ] - args = [] - for key in grp.keys(): - fromSer = self.__fromSerial(grp[key]) - args.append((key, fromSer)) - kwargs = dict(args) - kwargs.pop('__type') - return cls(**kwargs) - #else: - # return grp.value - - - def fromSerial(self, string): - HDF5Serializer.tmp_num += 1 - fn = 'tmp%d.hdf5'%(HDF5Serializer.tmp_num-1,) - fp = open(fn, 'wb') - fp.write(string) - fp.flush(), fp.close() - - root = h5py.File(fn, driver='core') - try: - fromSer = self.__fromSerial(root['dataset']) - except: - root.flush(), root.close() - os.remove(fn) - raise - - root.flush(), root.close() - os.remove(fn) - - return fromSer - - def __toSerial(self, obj, grp, name): - - if isinstance(obj, datetime.datetime): - sub_grp = grp.create_group(name) - sub_grp['__type'] = 'datetime' - sub_grp['ts'] = obj.isoformat(' ') - - elif isinstance(obj, tuple(Lookup.builtin_objects_simple.values())): - sub_grp = grp.create_group(name) - sub_grp['__type'] = Lookup.obj_dtypes[obj.__class__] - for key, value in obj.__dict__.items(): - if value != None and key not in ['totalNS', 'totalPS']: - sub_grp[key] = value - - elif obj == None: - sub_grp = grp.create_group(name) - sub_grp['__type'] = '_null' - - elif isinstance(obj, DynamicObject.Object): - # Create the new group and assign unique identifier for this type of DynamicObject - sub_grp = grp.create_group(name) - tag = '%s.%s'%(obj.getObjectName(), obj.getRevisionNumber()) - sub_grp['__type'] = tag - # Put all of the DynamicObject's attributes into the new h5py group - for key, value in obj.__dict__.items(): - self.__toSerial(value, sub_grp, key) - - elif isinstance(obj, tuple): - sub_grp = grp.create_group(name) - sub_grp['__type'] = 'tuple' - sub_grp['tuple'] = obj - - elif isinstance(obj, list) and len(obj) == 0: - sub_grp = grp.create_group(name) - sub_grp['__type'] = 'empty_list' - - else: - grp[name] = obj - - def toSerial(self, obj): - HDF5Serializer.tmp_num += 1 - fn = 'tmp%d.hdf5'%(HDF5Serializer.tmp_num,) - root = h5py.File(fn, driver='core') - try: - self.__toSerial(obj, root, 'dataset') - except: - root.flush(), root.close() - os.remove(fn) - raise - root.flush(), root.close() - - fp = open(fn, 'rb') - msg = fp.read() - fp.close() - os.remove(fn) - - return msg - -# Alias for the standard json serializer: -class jsonSerializer(Serializer): - def fromSerial(self, string): - #return json.loads(string) - return jsonpickle.decode(string) - def toSerial(self, string): - #return json.dumps(string) - return jsonpickle.encode(string, max_depth=500) - -# Dict mapping from serializer type to corresponding class object: -serializers = {'yaml': YAMLSerializer, - 'msgpack': MessagePackSerializer, - 'hdf5': HDF5Serializer, - 'json': jsonSerializer} - -instances = {'yaml': YAMLSerializer(), - 'msgpack': MessagePackSerializer(), - 'hdf5': HDF5Serializer(), - 'json': jsonSerializer()} - -serial_types = dict([(v,u) for u,v in serializers.items()]) - -compression_types = ['gzip', ''] - diff --git a/schainpy/model/serializer/test/__init__.py b/schainpy/model/serializer/test/__init__.py deleted file mode 100644 index e69de29..0000000 --- a/schainpy/model/serializer/test/__init__.py +++ /dev/null diff --git a/schainpy/model/serializer/test/serialtest.py b/schainpy/model/serializer/test/serialtest.py deleted file mode 100644 index 0b8ded6..0000000 --- a/schainpy/model/serializer/test/serialtest.py +++ /dev/null @@ -1,187 +0,0 @@ - -# -# when Serializer is imported alone, fault indicates this package is -# dependent on lookup, but don't import Lookup, instead: -# -import DynamicObject # dependent on pysvn -import Serializer # dependent on Dynamic Object - -import msgpack -import redis - -import numpy as np - -class NamedObject(): - - #--------------------------------------------- - @staticmethod - def indexListMatch(list_of_lists, name, value, name2=None, value2=None, unique_f=False): - # - # for each list's attribute compare with value - # if match, return True plus list - # else return False plus empty list - # - # search needs to be named part of class for object else . is unrecognized - # - # unique_f finds non-uniqueness - - - index = [] # return empty indices - list_data = [] # return empty list - ii = 0 - for theList in list_of_lists: - - cmd0 = "theList.%s == value" % (name) - cmd1 = "isInlist(theList.%s,value)" % name - # if name is valid then - # match name against value - # match name (as list) against value - if (eval(cmd0) or eval(cmd1)): - if (name2 != None): - cmd2 = "theList.%s == value2" % name2 - cmd3 = "isInlist(theList.%s,value2)" % name2 - if (eval(cmd2) or eval(cmd3)): - if (unique_f): - index = index + [ii] - list_data = list_data + [theList] # save list of lists if non-unique - # don't exit on match, may be non-unique - else: - list_data = theList # save the list - index = [ii] - break - else: - if (unique_f): - index = index + [ii] - list_data = list_data + [theList] # list of lists if non-unique - else: - list_data = theList - index = [ii] - break # exit on match - #endif - ii = ii + 1 - #end for - - return index, list_data # return indices of matches and list (or list of lists) - - #end indexListMatch - - #--------------------------------------------- - @staticmethod - def namedListMatch(list_of_lists, name, value, name2=None, value2=None, unique_f=None): - # - # for each list's attribute compare with value - # if match, return True plus list - # else return False plus empty list - # - # search needs to be named part of class for object else . is unrecognized - # - # unique_f finds non-uniqueness ('None' is same as False) - - match_f = False - list_data = [] # initialize - - for theList in list_of_lists: - - cmd0 = "theList.%s == value" % (name) - cmd1 = "isInlist(theList.%s,value)" % name - # if name is valid then - # match name against value - # match name (as list) against value - if (eval(cmd0) or eval(cmd1)): - if (name2 != None): - cmd2 = "theList.%s == value2" % name2 - cmd3 = "isInlist(theList.%s,value2)" % name2 - if (eval(cmd2) or eval(cmd3)): - match_f = True - if (unique_f): - list_data = list_data + [theList] # save list of lists if non-unique - # don't exit on match, may be non-unique - else: - list_data = theList # save the list - break - else: - match_f = True - if (unique_f): - list_data = list_data + [theList] # list of lists if non-unique - else: - list_data = theList - break # exit on match - #endif - #end for - - return match_f, list_data # return match, and list (or list of lists) - - #end namedListMatch - - #--------------------------------------------- - @staticmethod - def combineLists(object): - # - # used for dumping elements in list of lists for debugging - # - ret_list =[] - ii = 0 - while ii < len(object): - ret_list = ret_list + [object[ii].list] # not a real list, so can't use built-in list iterator - ii = ii + 1 - return ret_list - - # end combineLists - - - -class StateListObject(NamedObject): - def __init__(self, concurrent=None, hierarchical=None, history=None, state=None): - self.concurrent = concurrent - self.hierarchical = hierarchical - self.history = history - self.state = state - self.list = [self.concurrent, self.hierarchical, self.history, self.state ] -#end class StateListObject - -source_object = "my test string" - -serializer = "yaml" -# -# python versioning issue (ver 2.7 -> else path) -# -if isinstance(serializer,Serializer.Serializer): - serial_type = Serializer.serial_types[serializer] -else: - serial_type = serializer - serializer = Serializer.serializers[serializer]() - - - -datastr = serializer.toSerial(source_object) - -dest_object = serializer.fromSerial(datastr) - -print "dest_object=",dest_object - -myObject = StateListObject(hierarchical="yes",state=np.array([1,2,3.0])) - -datastr = serializer.toSerial(myObject) - -packed = msgpack.packb(datastr) - -try: - r= redis.StrictRedis(host='localhost',port=6379,db=0) -except Exception as eobj: - print "is the redis server running?",eobj -else: - - r.set('baz',packed) # converts to string - x = r.get('baz') - -unpacked = msgpack.unpackb(x) - -dest_object = serializer.fromSerial(unpacked) - -print "val1=",dest_object.hierarchical -val2 = dest_object.state -print "val2=",val2 -# can numpy array be used as array? -print val2.shape - - diff --git a/schainpy/model/serializer/test/test _serializer_speed.py b/schainpy/model/serializer/test/test _serializer_speed.py deleted file mode 100644 index 5b891d3..0000000 --- a/schainpy/model/serializer/test/test _serializer_speed.py +++ /dev/null @@ -1,175 +0,0 @@ -''' -Created on Jul 16, 2014 - -@author: roj-idl71 -''' -""" -Dependencies: - - pip install tabulate simplejson python-cjson ujson yajl msgpack-python - -""" - -from timeit import timeit -from tabulate import tabulate - -setup = '''d = { - 'words': """ - Lorem ipsum dolor sit amet, consectetur adipiscing - elit. Mauris adipiscing adipiscing placerat. - Vestibulum augue augue, - pellentesque quis sollicitudin id, adipiscing. - """, - 'boolean' : False, - 'list': range(10), - 'dict': dict((str(i),'a') for i in xrange(10)), - 'int': 100, - 'float': 100.123456, -}''' - -setup = '''import numpy; -import datetime; -d = { - 'words': """ - Lorem ipsum dolor sit amet, consectetur adipiscing - elit. Mauris adipiscing adipiscing placerat. - Vestibulum augue augue, - pellentesque quis sollicitudin id, adipiscing. - """, - 'boolean' : False, - 'list': range(10), - 'dict': dict((str(i),'a') for i in xrange(10)), - 'int': 100, - 'float': 100.123456, - 'datetime' : datetime.datetime(2001,1,1,10,10,10) -}''' - - -setup_pickle = '%s ; import pickle ; src = pickle.dumps(d)' % setup -setup_pickle2 = '%s ; import pickle ; src = pickle.dumps(d, 2)' % setup -setup_cpickle = '%s ; import cPickle ; src = cPickle.dumps(d)' % setup -setup_cpickle2 = '%s ; import cPickle ; src = cPickle.dumps(d, 2)' % setup -setup_json = '%s ; import json; src = json.dumps(d)' % setup -setup_ujson = '%s ; import ujson; src = ujson.encode(d)' % setup -setup_cjson = '%s ; import cjson; src = cjson.encode(d)' % setup -setup_simplejson = '%s ; import simplejson; src = simplejson.dump(d)' % setup -setup_jsonpickle = '%s ; import jsonpickle; src = jsonpickle.encode(d)' % setup -setup_yaml = '%s ; import yaml; src = yaml.dump(d)' % setup -setup_msgpack = '%s ; import msgpack; src = msgpack.dumps(d)' % setup -setup_msgpack_np = '%s; import msgpack_numpy as msgpack; src = msgpack.dumps(d)' % setup - -tests = [ - # (title, setup, enc_test, dec_test, result) - ('cPickle (binary)', 'import cPickle; %s' % setup_cpickle2, 'cPickle.dumps(d, 2)', 'r = cPickle.loads(src)', 'print r'), - ('cPickle (ascii)', 'import cPickle; %s' % setup_cpickle, 'cPickle.dumps(d, 0)', 'r = cPickle.loads(src)', 'print r'), - ('pickle (binary)', 'import pickle; %s' % setup_pickle2, 'pickle.dumps(d, 2)', 'r = pickle.loads(src)', 'print r'), - ('pickle (ascii)', 'import pickle; %s' % setup_pickle, 'pickle.dumps(d, 0)', 'r = pickle.loads(src)', 'print r'), - ('jsonpickle', 'import jsonpickle; %s' % setup_jsonpickle, 'jsonpickle.encode(d)', 'r = jsonpickle.decode(src)', 'print r'), -# ('msgpack-numpy-python', '%s' % setup_msgpack_np, 'msgpack.dumps(d)', 'r = msgpack.loads(src)', 'print r'), - ('ujson', 'import ujson; %s' % setup_ujson, 'ujson.encode(d)', 'r = ujson.decode(src)', 'print r'), -# ('msgpack-python', 'import msgpack; %s' % setup_msgpack, 'msgpack.dumps(d)', 'r = msgpack.loads(src)', 'print r'), -# ('json', 'import json; %s' % setup_json, 'json.dumps(d)', 'r = json.loads(src)', 'print r'), -# ('python-cjson-1.0.5', 'import cjson; %s' % setup_cjson, 'cjson.encode(d)', 'r = cjson.decode(src)', 'print r'), -# ('simplejson-3.3.1', 'import simplejson; %s' % setup_json, 'simplejson.dumps(d)', 'r = simplejson.loads(src)', 'print r'), - ('yaml', 'import yaml; %s' % setup_yaml, 'yaml.dump(d)', 'r = yaml.load(src)', 'print r'), -] - -loops = 1 -enc_table = [] -dec_table = [] - -print "Running tests (%d loops each)" % loops - -for title, mod, enc, dec, msg in tests: - print title - - ### Getting the package size - exec mod - size = len("".join(src)) - - print " [Encode]", enc - result = timeit(enc, mod, number=loops) - enc_table.append([title, result, size]) - - print " [Decode]", dec - result = timeit(dec, mod, number=loops) - dec_table.append([title, result]) - - print " Result" - result = timeit(msg, mod+';'+dec, number=1) - -enc_table.sort(key=lambda x: x[1]) -enc_table.insert(0, ['Package', 'Seconds', 'Size']) - -dec_table.sort(key=lambda x: x[1]) -dec_table.insert(0, ['Package', 'Seconds']) - -print "\nEncoding Test (%d loops)" % loops -print tabulate(enc_table, headers="firstrow") - -print "\nDecoding Test (%d loops)" % loops -print tabulate(dec_table, headers="firstrow") - -""" -OUTPUT: - -Running tests (15000 loops each) -pickle (ascii) - [Encode] pickle.dumps(d, 0) - [Decode] pickle.loads(src) -pickle (binary) - [Encode] pickle.dumps(d, 2) - [Decode] pickle.loads(src) -cPickle (ascii) - [Encode] cPickle.dumps(d, 0) - [Decode] cPickle.loads(src) -cPickle (binary) - [Encode] cPickle.dumps(d, 2) - [Decode] cPickle.loads(src) -json - [Encode] json.dumps(d) - [Decode] json.loads(src) -simplejson-3.3.1 - [Encode] simplejson.dumps(d) - [Decode] simplejson.loads(src) -python-cjson-1.0.5 - [Encode] cjson.encode(d) - [Decode] cjson.decode(src) -ujson-1.33 - [Encode] ujson.dumps(d) - [Decode] ujson.loads(src) -yajl 0.3.5 - [Encode] yajl.dumps(d) - [Decode] yajl.loads(src) -msgpack-python-0.3.0 - [Encode] msgpack.dumps(d) - [Decode] msgpack.loads(src) - -Encoding Test (15000 loops) -Package Seconds --------------------- --------- -ujson-1.33 0.232215 -msgpack-python-0.3.0 0.241945 -cPickle (binary) 0.305273 -yajl 0.3.5 0.634148 -python-cjson-1.0.5 0.680604 -json 0.780438 -simplejson-3.3.1 1.04763 -cPickle (ascii) 1.62062 -pickle (ascii) 14.0497 -pickle (binary) 15.4712 - -Decoding Test (15000 loops) -Package Seconds --------------------- --------- -msgpack-python-0.3.0 0.240885 -cPickle (binary) 0.393152 -ujson-1.33 0.396875 -python-cjson-1.0.5 0.694321 -yajl 0.3.5 0.748369 -simplejson-3.3.1 0.780531 -cPickle (ascii) 1.38561 -json 1.65921 -pickle (binary) 5.20554 -pickle (ascii) 17.8767 -""" \ No newline at end of file diff --git a/schainpy/model/utils/__init__.py b/schainpy/model/utils/__init__.py index 0359aa1..40ee51f 100644 --- a/schainpy/model/utils/__init__.py +++ b/schainpy/model/utils/__init__.py @@ -4,5 +4,5 @@ $Author: murco $ $Id: Processor.py 1 2012-11-12 18:56:07Z murco $ ''' -from jroutils_ftp import * -from jroutils_publish import * +from .jroutils_ftp import * +from .jroutils_publish import * diff --git a/schainpy/model/utils/jroutils_ftp.py b/schainpy/model/utils/jroutils_ftp.py index a507f2f..8be1671 100644 --- a/schainpy/model/utils/jroutils_ftp.py +++ b/schainpy/model/utils/jroutils_ftp.py @@ -9,7 +9,7 @@ try: import paramiko import scp except: - print "You should install paramiko and scp libraries \nif you want to use SSH protocol to upload files to the server" + pass import time @@ -64,9 +64,9 @@ class Remote(Thread): self.stopFlag = False - print "[Remote Server] Opening server: %s" %self.__server + print("[Remote Server] Opening server: %s" %self.__server) if self.open(self.__server, self.__username, self.__password, self.__remotefolder): - print "[Remote Server] %s server was opened successfully" %self.__server + print("[Remote Server] %s server was opened successfully" %self.__server) self.close() @@ -81,31 +81,31 @@ class Remote(Thread): """ Connect to server and create a connection class (FTP or SSH) to remote server. """ - raise NotImplementedError, "Implement this method in child class" + raise NotImplementedError("Implement this method in child class") def close(self): """ Close connection to server """ - raise NotImplementedError, "Implement this method in child class" + raise NotImplementedError("Implement this method in child class") def mkdir(self, remotefolder): """ Create a folder remotely """ - raise NotImplementedError, "Implement this method in child class" + raise NotImplementedError("Implement this method in child class") def cd(self, remotefolder): """ Change working directory in remote server """ - raise NotImplementedError, "Implement this method in child class" + raise NotImplementedError("Implement this method in child class") def download(self, filename, localfolder=None): """ Download a file from server to local host """ - raise NotImplementedError, "Implement this method in child class" + raise NotImplementedError("Implement this method in child class") def sendFile(self, fullfilename): """ @@ -117,7 +117,7 @@ class Remote(Thread): Returns: 0 in error case else 1 """ - raise NotImplementedError, "Implement this method in child class" + raise NotImplementedError("Implement this method in child class") def upload(self, fullfilename, remotefolder=None): """ @@ -132,7 +132,7 @@ class Remote(Thread): Returns: 0 in error case else 1 """ - print "[Remote Server] Uploading %s to %s:%s" %(fullfilename, self.server, self.remotefolder) + print("[Remote Server] Uploading %s to %s:%s" %(fullfilename, self.server, self.remotefolder)) if not self.status: return 0 @@ -144,10 +144,10 @@ class Remote(Thread): return 0 if not self.sendFile(fullfilename): - print "[Remote Server] Error uploading file %s" %fullfilename + print("[Remote Server] Error uploading file %s" %fullfilename) return 0 - print "[Remote Server] upload finished successfully" + print("[Remote Server] upload finished successfully") return 1 @@ -180,11 +180,11 @@ class Remote(Thread): def run(self): if not self.status: - print "Finishing FTP service" + print("Finishing FTP service") return if not self.cd(self.remotefolder): - raise ValueError, "Could not access to the new remote directory: %s" %self.remotefolder + raise ValueError("Could not access to the new remote directory: %s" %self.remotefolder) while True: @@ -199,7 +199,7 @@ class Remote(Thread): # self.bussy = True self.mutex.acquire() - print "[Remote Server] Opening %s" %self.__server + print("[Remote Server] Opening %s" %self.__server) if not self.open(self.__server, self.__username, self.__password, self.__remotefolder): self.mutex.release() continue @@ -207,13 +207,13 @@ class Remote(Thread): for thisFile in self.fileList: self.upload(thisFile, self.remotefolder) - print "[Remote Server] Closing %s" %self.__server + print("[Remote Server] Closing %s" %self.__server) self.close() self.mutex.release() # self.bussy = False - print "[Remote Server] Thread stopped successfully" + print("[Remote Server] Thread stopped successfully") class FTPClient(Remote): @@ -247,29 +247,29 @@ class FTPClient(Remote): """ if server == None: - raise ValueError, "FTP server should be defined" + raise ValueError("FTP server should be defined") if username == None: - raise ValueError, "FTP username should be defined" + raise ValueError("FTP username should be defined") if password == None: - raise ValueError, "FTP password should be defined" + raise ValueError("FTP password should be defined") if remotefolder == None: - raise ValueError, "FTP remote folder should be defined" + raise ValueError("FTP remote folder should be defined") try: ftpClientObj = ftplib.FTP(server) - except ftplib.all_errors, e: - print "[FTP Server]: FTP server connection fail: %s" %server - print "[FTP Server]:", e + except ftplib.all_errors as e: + print("[FTP Server]: FTP server connection fail: %s" %server) + print("[FTP Server]:", e) self.status = 0 return 0 try: ftpClientObj.login(username, password) except ftplib.all_errors: - print "[FTP Server]: FTP username or password are incorrect" + print("[FTP Server]: FTP username or password are incorrect") self.status = 0 return 0 @@ -279,7 +279,7 @@ class FTPClient(Remote): try: ftpClientObj.cwd(remotefolder) except ftplib.all_errors: - print "[FTP Server]: FTP remote folder is invalid: %s" %remotefolder + print("[FTP Server]: FTP remote folder is invalid: %s" %remotefolder) remotefolder = ftpClientObj.pwd() self.server = server @@ -316,7 +316,7 @@ class FTPClient(Remote): try: self.__ftpClientObj.mkd(dirname) except ftplib.all_errors: - print "[FTP Server]: Error creating remote folder: %s" %remotefolder + print("[FTP Server]: Error creating remote folder: %s" %remotefolder) return 0 return 1 @@ -343,11 +343,11 @@ class FTPClient(Remote): try: self.__ftpClientObj.cwd(remotefolder) except ftplib.all_errors: - print '[FTP Server]: Error changing to %s' %remotefolder - print '[FTP Server]: Trying to create remote folder' + print('[FTP Server]: Error changing to %s' %remotefolder) + print('[FTP Server]: Trying to create remote folder') if not self.mkdir(remotefolder): - print '[FTP Server]: Remote folder could not be created' + print('[FTP Server]: Remote folder could not be created') return 0 try: @@ -372,14 +372,14 @@ class FTPClient(Remote): try: self.__ftpClientObj.storbinary(command, fp) - except ftplib.all_errors, e: - print "[FTP Server]:", e + except ftplib.all_errors as e: + print("[FTP Server]:", e) return 0 try: self.__ftpClientObj.sendcmd('SITE CHMOD 755 ' + filename) - except ftplib.all_errors, e: - print "[FTP Server]:", e + except ftplib.all_errors as e: + print("[FTP Server]:", e) fp.close() @@ -418,16 +418,16 @@ class SSHClient(Remote): import socket if server == None: - raise ValueError, "SSH server should be defined" + raise ValueError("SSH server should be defined") if username == None: - raise ValueError, "SSH username should be defined" + raise ValueError("SSH username should be defined") if password == None: - raise ValueError, "SSH password should be defined" + raise ValueError("SSH password should be defined") if remotefolder == None: - raise ValueError, "SSH remote folder should be defined" + raise ValueError("SSH remote folder should be defined") sshClientObj = paramiko.SSHClient() @@ -437,16 +437,16 @@ class SSHClient(Remote): self.status = 0 try: sshClientObj.connect(server, username=username, password=password, port=port) - except paramiko.AuthenticationException, e: + except paramiko.AuthenticationException as e: # print "SSH username or password are incorrect: %s" - print "[SSH Server]:", e + print("[SSH Server]:", e) return 0 - except SSHException, e: - print "[SSH Server]:", e + except SSHException as e: + print("[SSH Server]:", e) return 0 except socket.error: self.status = 0 - print "[SSH Server]:", e + print("[SSH Server]:", e) return 0 self.status = 1 @@ -463,7 +463,7 @@ class SSHClient(Remote): self.status = 1 if not self.cd(remotefolder): - raise ValueError, "[SSH Server]: Could not access to remote folder: %s" %remotefolder + raise ValueError("[SSH Server]: Could not access to remote folder: %s" %remotefolder) return 0 self.remotefolder = remotefolder @@ -564,8 +564,8 @@ class SSHClient(Remote): try: self.__scpClientObj.put(fullfilename, remote_path=self.remotefolder) - except scp.ScpError, e: - print "[SSH Server]", str(e) + except scp.ScpError as e: + print("[SSH Server]", str(e)) return 0 remotefile = os.path.join(self.remotefolder, os.path.split(fullfilename)[-1]) @@ -596,7 +596,7 @@ class SendToServer(ProcessingUnit): self.clientObj = SSHClient(server, username, password, remotefolder, period) if not self.clientObj: - raise ValueError, "%s has been chosen as remote access protocol but it is not valid" %protocol + raise ValueError("%s has been chosen as remote access protocol but it is not valid" %protocol) self.clientObj.start() @@ -614,7 +614,7 @@ class SendToServer(ProcessingUnit): for thisFolder in folderList: - print "[Remote Server]: Searching files on %s" %thisFolder + print("[Remote Server]: Searching files on %s" %thisFolder) filenameList = glob.glob1(thisFolder, '*%s' %self.ext) @@ -643,18 +643,18 @@ class SendToServer(ProcessingUnit): self.isConfig = True if not self.clientObj.is_alive(): - print "[Remote Server]: Restarting connection " + print("[Remote Server]: Restarting connection ") self.setup(**kwargs) if time.time() - self.init >= self.period: fullfilenameList = self.findFiles() if self.clientObj.updateFileList(fullfilenameList): - print "[Remote Server]: Sending the next files ", str(fullfilenameList) + print("[Remote Server]: Sending the next files ", str(fullfilenameList)) self.init = time.time() def close(self): - print "[Remote Server] Stopping thread" + print("[Remote Server] Stopping thread") self.clientObj.stop() @@ -710,7 +710,7 @@ class FTP(object): # print 'Connect to FTP Server: Successfully' except ftplib.all_errors: - print 'Error FTP Service' + print('Error FTP Service') self.status = 1 return @@ -721,14 +721,14 @@ class FTP(object): try: self.dirList = self.ftp.nlst() - except ftplib.error_perm, resp: + except ftplib.error_perm as resp: if str(resp) == "550 No files found": - print "no files in this directory" + print("no files in this directory") self.status = 1 return except ftplib.all_errors: - print 'Error Displaying Dir-Files' + print('Error Displaying Dir-Files') self.status = 1 return @@ -763,7 +763,7 @@ class FTP(object): try: self.ftp.mkd(dirname) except: - print 'Error creating remote folder:%s'%dirname + print('Error creating remote folder:%s'%dirname) return 1 return 0 @@ -783,7 +783,7 @@ class FTP(object): try: self.ftp.delete(filename) except: - print 'Error deleting remote file:%s'%filename + print('Error deleting remote file:%s'%filename) return 1 return 0 @@ -805,7 +805,7 @@ class FTP(object): if not(filename in self.fileList): - print 'filename:%s not exists'%filename + print('filename:%s not exists'%filename) self.status = 1 return self.status @@ -814,11 +814,11 @@ class FTP(object): self.file = open(newfilename, 'wb') try: - print 'Download: ' + filename + print('Download: ' + filename) self.ftp.retrbinary('RETR ' + filename, self.__handleDownload) - print 'Download Complete' + print('Download Complete') except ftplib.all_errors: - print 'Error Downloading ' + filename + print('Error Downloading ' + filename) self.status = 1 return self.status @@ -861,12 +861,12 @@ class FTP(object): command = "STOR " + tail - print 'Uploading: ' + tail + print('Uploading: ' + tail) self.ftp.storbinary(command, self.file) - print 'Upload Completed' + print('Upload Completed') except ftplib.all_errors: - print 'Error Uploading ' + tail + print('Error Uploading ' + tail) self.status = 1 return self.status @@ -895,11 +895,11 @@ class FTP(object): """ self.remotefolder = remotefolder - print 'Change to ' + self.remotefolder + print('Change to ' + self.remotefolder) try: self.ftp.cwd(remotefolder) except ftplib.all_errors: - print 'Error Change to ' + self.remotefolder + print('Error Change to ' + self.remotefolder) infoList = None self.folderList = None return infoList,self.folderList @@ -909,14 +909,14 @@ class FTP(object): try: self.dirList = self.ftp.nlst() - except ftplib.error_perm, resp: + except ftplib.error_perm as resp: if str(resp) == "550 No files found": - print "no files in this directory" + print("no files in this directory") infoList = None self.folderList = None return infoList,self.folderList except ftplib.all_errors: - print 'Error Displaying Dir-Files' + print('Error Displaying Dir-Files') infoList = None self.folderList = None return infoList,self.folderList @@ -957,8 +957,8 @@ class SendByFTP(Operation): def error_print(self, ValueError): - print ValueError, 'Error FTP' - print "don't worry the program is running..." + print(ValueError, 'Error FTP') + print("don't worry the program is running...") def worker_ftp(self, server, username, password, remotefolder, filenameList): @@ -981,7 +981,7 @@ class SendByFTP(Operation): if p.is_alive(): p.terminate() p.join() - print 'killing ftp process...' + print('killing ftp process...') self.status = 0 return @@ -1005,4 +1005,4 @@ class SendByFTP(Operation): self.counter = 0 - self.status = 1 + self.status = 1 \ No newline at end of file diff --git a/schainpy/model/utils/jroutils_publish.py b/schainpy/model/utils/jroutils_publish.py index 7f2b188..d631130 100644 --- a/schainpy/model/utils/jroutils_publish.py +++ b/schainpy/model/utils/jroutils_publish.py @@ -7,16 +7,14 @@ import glob import time import json import numpy -import paho.mqtt.client as mqtt import zmq import datetime import ftplib -from zmq.utils.monitor import recv_monitor_message from functools import wraps from threading import Thread from multiprocessing import Process -from schainpy.model.proc.jroproc_base import Operation, ProcessingUnit +from schainpy.model.proc.jroproc_base import Operation, ProcessingUnit, MPDecorator from schainpy.model.data.jrodata import JROData from schainpy.utils import log @@ -54,428 +52,46 @@ def get_plot_code(s): else: return 24 -def roundFloats(obj): - if isinstance(obj, list): - return map(roundFloats, obj) - elif isinstance(obj, float): - return round(obj, 2) - def decimate(z, MAXNUMY): dy = int(len(z[0])/MAXNUMY) + 1 return z[::, ::dy] -class throttle(object): - ''' - Decorator that prevents a function from being called more than once every - time period. - To create a function that cannot be called more than once a minute, but - will sleep until it can be called: - @throttle(minutes=1) - def foo(): - pass - - for i in range(10): - foo() - print "This function has run %s times." % i - ''' - - def __init__(self, seconds=0, minutes=0, hours=0): - self.throttle_period = datetime.timedelta( - seconds=seconds, minutes=minutes, hours=hours - ) - - self.time_of_last_call = datetime.datetime.min - - def __call__(self, fn): - @wraps(fn) - def wrapper(*args, **kwargs): - coerce = kwargs.pop('coerce', None) - if coerce: - self.time_of_last_call = datetime.datetime.now() - return fn(*args, **kwargs) - else: - now = datetime.datetime.now() - time_since_last_call = now - self.time_of_last_call - time_left = self.throttle_period - time_since_last_call - - if time_left > datetime.timedelta(seconds=0): - return - - self.time_of_last_call = datetime.datetime.now() - return fn(*args, **kwargs) - - return wrapper - -class Data(object): - ''' - Object to hold data to be plotted - ''' - - def __init__(self, plottypes, throttle_value, exp_code, buffering=True): - self.plottypes = plottypes - self.throttle = throttle_value - self.exp_code = exp_code - self.buffering = buffering - self.ended = False - self.localtime = False - self.meta = {} - self.__times = [] - self.__heights = [] - - def __str__(self): - dum = ['{}{}'.format(key, self.shape(key)) for key in self.data] - return 'Data[{}][{}]'.format(';'.join(dum), len(self.__times)) - - def __len__(self): - return len(self.__times) - - def __getitem__(self, key): - if key not in self.data: - raise KeyError(log.error('Missing key: {}'.format(key))) - - if 'spc' in key or not self.buffering: - ret = self.data[key] - else: - ret = numpy.array([self.data[key][x] for x in self.times]) - if ret.ndim > 1: - ret = numpy.swapaxes(ret, 0, 1) - return ret - - def __contains__(self, key): - return key in self.data - - def setup(self): - ''' - Configure object - ''' - - self.type = '' - self.ended = False - self.data = {} - self.__times = [] - self.__heights = [] - self.__all_heights = set() - for plot in self.plottypes: - if 'snr' in plot: - plot = 'snr' - self.data[plot] = {} - - def shape(self, key): - ''' - Get the shape of the one-element data for the given key - ''' - - if len(self.data[key]): - if 'spc' in key or not self.buffering: - return self.data[key].shape - return self.data[key][self.__times[0]].shape - return (0,) - - def update(self, dataOut, tm): - ''' - Update data object with new dataOut - ''' - - if tm in self.__times: - return - - self.type = dataOut.type - self.parameters = getattr(dataOut, 'parameters', []) - if hasattr(dataOut, 'pairsList'): - self.pairs = dataOut.pairsList - if hasattr(dataOut, 'meta'): - self.meta = dataOut.meta - self.channels = dataOut.channelList - self.interval = dataOut.getTimeInterval() - self.localtime = dataOut.useLocalTime - if 'spc' in self.plottypes or 'cspc' in self.plottypes: - self.xrange = (dataOut.getFreqRange(1)/1000., dataOut.getAcfRange(1), dataOut.getVelRange(1)) - self.__heights.append(dataOut.heightList) - self.__all_heights.update(dataOut.heightList) - self.__times.append(tm) - - for plot in self.plottypes: - if plot == 'spc': - z = dataOut.data_spc/dataOut.normFactor - buffer = 10*numpy.log10(z) - if plot == 'cspc': - buffer = dataOut.data_cspc - if plot == 'noise': - buffer = 10*numpy.log10(dataOut.getNoise()/dataOut.normFactor) - if plot == 'rti': - buffer = dataOut.getPower() - if plot == 'snr_db': - buffer = dataOut.data_SNR - if plot == 'snr': - buffer = 10*numpy.log10(dataOut.data_SNR) - if plot == 'dop': - buffer = 10*numpy.log10(dataOut.data_DOP) - if plot == 'mean': - buffer = dataOut.data_MEAN - if plot == 'std': - buffer = dataOut.data_STD - if plot == 'coh': - buffer = dataOut.getCoherence() - if plot == 'phase': - buffer = dataOut.getCoherence(phase=True) - if plot == 'output': - buffer = dataOut.data_output - if plot == 'param': - buffer = dataOut.data_param - - if 'spc' in plot: - self.data[plot] = buffer - else: - if self.buffering: - self.data[plot][tm] = buffer - else: - self.data[plot] = buffer - - def normalize_heights(self): - ''' - Ensure same-dimension of the data for different heighList - ''' - - H = numpy.array(list(self.__all_heights)) - H.sort() - for key in self.data: - shape = self.shape(key)[:-1] + H.shape - for tm, obj in self.data[key].items(): - h = self.__heights[self.__times.index(tm)] - if H.size == h.size: - continue - index = numpy.where(numpy.in1d(H, h))[0] - dummy = numpy.zeros(shape) + numpy.nan - if len(shape) == 2: - dummy[:, index] = obj - else: - dummy[index] = obj - self.data[key][tm] = dummy - - self.__heights = [H for tm in self.__times] - - def jsonify(self, decimate=False): - ''' - Convert data to json - ''' - - data = {} - tm = self.times[-1] - dy = int(self.heights.size/MAXNUMY) + 1 - for key in self.data: - if key in ('spc', 'cspc') or not self.buffering: - dx = int(self.data[key].shape[1]/MAXNUMX) + 1 - data[key] = roundFloats(self.data[key][::, ::dx, ::dy].tolist()) - else: - data[key] = roundFloats(self.data[key][tm].tolist()) - - ret = {'data': data} - ret['exp_code'] = self.exp_code - ret['time'] = tm - ret['interval'] = self.interval - ret['localtime'] = self.localtime - ret['yrange'] = roundFloats(self.heights[::dy].tolist()) - if 'spc' in self.data or 'cspc' in self.data: - ret['xrange'] = roundFloats(self.xrange[2][::dx].tolist()) - else: - ret['xrange'] = [] - if hasattr(self, 'pairs'): - ret['pairs'] = self.pairs - else: - ret['pairs'] = [] - - for key, value in self.meta.items(): - ret[key] = value - - return json.dumps(ret) - - @property - def times(self): - ''' - Return the list of times of the current data - ''' - - ret = numpy.array(self.__times) - ret.sort() - return ret - - @property - def heights(self): - ''' - Return the list of heights of the current data - ''' - - return numpy.array(self.__heights[-1]) class PublishData(Operation): ''' Operation to send data over zmq. ''' - __attrs__ = ['host', 'port', 'delay', 'zeromq', 'mqtt', 'verbose'] + __attrs__ = ['host', 'port', 'delay', 'verbose'] - def __init__(self, **kwargs): - """Inicio.""" - Operation.__init__(self, **kwargs) - self.isConfig = False - self.client = None - self.zeromq = None - self.mqtt = None - - def on_disconnect(self, client, userdata, rc): - if rc != 0: - log.warning('Unexpected disconnection.') - self.connect() - - def connect(self): - log.warning('trying to connect') - try: - self.client.connect( - host=self.host, - port=self.port, - keepalive=60*10, - bind_address='') - self.client.loop_start() - # self.client.publish( - # self.topic + 'SETUP', - # json.dumps(setup), - # retain=True - # ) - except: - log.error('MQTT Conection error.') - self.client = False - - def setup(self, port=1883, username=None, password=None, clientId="user", zeromq=1, verbose=True, **kwargs): + def setup(self, server='zmq.pipe', delay=0, verbose=True, **kwargs): self.counter = 0 - self.topic = kwargs.get('topic', 'schain') self.delay = kwargs.get('delay', 0) - self.plottype = kwargs.get('plottype', 'spectra') - self.host = kwargs.get('host', "10.10.10.82") - self.port = kwargs.get('port', 3000) - self.clientId = clientId self.cnt = 0 - self.zeromq = zeromq - self.mqtt = kwargs.get('plottype', 0) - self.client = None self.verbose = verbose - setup = [] - if mqtt is 1: - self.client = mqtt.Client( - client_id=self.clientId + self.topic + 'SCHAIN', - clean_session=True) - self.client.on_disconnect = self.on_disconnect - self.connect() - for plot in self.plottype: - setup.append({ - 'plot': plot, - 'topic': self.topic + plot, - 'title': getattr(self, plot + '_' + 'title', False), - 'xlabel': getattr(self, plot + '_' + 'xlabel', False), - 'ylabel': getattr(self, plot + '_' + 'ylabel', False), - 'xrange': getattr(self, plot + '_' + 'xrange', False), - 'yrange': getattr(self, plot + '_' + 'yrange', False), - 'zrange': getattr(self, plot + '_' + 'zrange', False), - }) - if zeromq is 1: - context = zmq.Context() - self.zmq_socket = context.socket(zmq.PUSH) - server = kwargs.get('server', 'zmq.pipe') - - if 'tcp://' in server: - address = server - else: - address = 'ipc:///tmp/%s' % server - - self.zmq_socket.connect(address) - time.sleep(1) + context = zmq.Context() + self.zmq_socket = context.socket(zmq.PUSH) + server = kwargs.get('server', 'zmq.pipe') + + if 'tcp://' in server: + address = server + else: + address = 'ipc:///tmp/%s' % server + + self.zmq_socket.connect(address) + time.sleep(1) def publish_data(self): self.dataOut.finished = False - if self.mqtt is 1: - yData = self.dataOut.heightList[:2].tolist() - if self.plottype == 'spectra': - data = getattr(self.dataOut, 'data_spc') - z = data/self.dataOut.normFactor - zdB = 10*numpy.log10(z) - xlen, ylen = zdB[0].shape - dx = int(xlen/MAXNUMX) + 1 - dy = int(ylen/MAXNUMY) + 1 - Z = [0 for i in self.dataOut.channelList] - for i in self.dataOut.channelList: - Z[i] = zdB[i][::dx, ::dy].tolist() - payload = { - 'timestamp': self.dataOut.utctime, - 'data': roundFloats(Z), - 'channels': ['Ch %s' % ch for ch in self.dataOut.channelList], - 'interval': self.dataOut.getTimeInterval(), - 'type': self.plottype, - 'yData': yData - } - - elif self.plottype in ('rti', 'power'): - data = getattr(self.dataOut, 'data_spc') - z = data/self.dataOut.normFactor - avg = numpy.average(z, axis=1) - avgdB = 10*numpy.log10(avg) - xlen, ylen = z[0].shape - dy = numpy.floor(ylen/self.__MAXNUMY) + 1 - AVG = [0 for i in self.dataOut.channelList] - for i in self.dataOut.channelList: - AVG[i] = avgdB[i][::dy].tolist() - payload = { - 'timestamp': self.dataOut.utctime, - 'data': roundFloats(AVG), - 'channels': ['Ch %s' % ch for ch in self.dataOut.channelList], - 'interval': self.dataOut.getTimeInterval(), - 'type': self.plottype, - 'yData': yData - } - elif self.plottype == 'noise': - noise = self.dataOut.getNoise()/self.dataOut.normFactor - noisedB = 10*numpy.log10(noise) - payload = { - 'timestamp': self.dataOut.utctime, - 'data': roundFloats(noisedB.reshape(-1, 1).tolist()), - 'channels': ['Ch %s' % ch for ch in self.dataOut.channelList], - 'interval': self.dataOut.getTimeInterval(), - 'type': self.plottype, - 'yData': yData - } - elif self.plottype == 'snr': - data = getattr(self.dataOut, 'data_SNR') - avgdB = 10*numpy.log10(data) - - ylen = data[0].size - dy = numpy.floor(ylen/self.__MAXNUMY) + 1 - AVG = [0 for i in self.dataOut.channelList] - for i in self.dataOut.channelList: - AVG[i] = avgdB[i][::dy].tolist() - payload = { - 'timestamp': self.dataOut.utctime, - 'data': roundFloats(AVG), - 'channels': ['Ch %s' % ch for ch in self.dataOut.channelList], - 'type': self.plottype, - 'yData': yData - } - else: - print "Tipo de grafico invalido" - payload = { - 'data': 'None', - 'timestamp': 'None', - 'type': None - } - - self.client.publish(self.topic + self.plottype, json.dumps(payload), qos=0) - - if self.zeromq is 1: - if self.verbose: - log.log( - 'Sending {} - {}'.format(self.dataOut.type, self.dataOut.datatime), - self.name - ) - self.zmq_socket.send_pyobj(self.dataOut) + + if self.verbose: + log.log( + 'Sending {} - {}'.format(self.dataOut.type, self.dataOut.datatime), + self.name + ) + self.zmq_socket.send_pyobj(self.dataOut) def run(self, dataOut, **kwargs): self.dataOut = dataOut @@ -487,15 +103,12 @@ class PublishData(Operation): time.sleep(self.delay) def close(self): - if self.zeromq is 1: - self.dataOut.finished = True - self.zmq_socket.send_pyobj(self.dataOut) - time.sleep(0.1) - self.zmq_socket.close() - if self.client: - self.client.loop_stop() - self.client.disconnect() - + + self.dataOut.finished = True + self.zmq_socket.send_pyobj(self.dataOut) + time.sleep(0.1) + self.zmq_socket.close() + class ReceiverData(ProcessingUnit): @@ -535,211 +148,43 @@ class ReceiverData(ProcessingUnit): self.dataOut.datatime.ctime(),), 'Receiving') - -class PlotterReceiver(ProcessingUnit, Process): - - throttle_value = 5 - __attrs__ = ['server', 'plottypes', 'realtime', 'localtime', 'throttle', - 'exp_code', 'web_server', 'buffering'] - - def __init__(self, **kwargs): - - ProcessingUnit.__init__(self, **kwargs) - Process.__init__(self) - self.mp = False - self.isConfig = False - self.isWebConfig = False - self.connections = 0 - server = kwargs.get('server', 'zmq.pipe') - web_server = kwargs.get('web_server', None) - if 'tcp://' in server: - address = server - else: - address = 'ipc:///tmp/%s' % server - self.address = address - self.web_address = web_server - self.plottypes = [s.strip() for s in kwargs.get('plottypes', 'rti').split(',')] - self.realtime = kwargs.get('realtime', False) - self.localtime = kwargs.get('localtime', True) - self.buffering = kwargs.get('buffering', True) - self.throttle_value = kwargs.get('throttle', 5) - self.exp_code = kwargs.get('exp_code', None) - self.sendData = self.initThrottle(self.throttle_value) - self.dates = [] - self.setup() - - def setup(self): - - self.data = Data(self.plottypes, self.throttle_value, self.exp_code, self.buffering) - self.isConfig = True - - def event_monitor(self, monitor): - - events = {} - - for name in dir(zmq): - if name.startswith('EVENT_'): - value = getattr(zmq, name) - events[value] = name - - while monitor.poll(): - evt = recv_monitor_message(monitor) - if evt['event'] == 32: - self.connections += 1 - if evt['event'] == 512: - pass - - evt.update({'description': events[evt['event']]}) - - if evt['event'] == zmq.EVENT_MONITOR_STOPPED: - break - monitor.close() - print('event monitor thread done!') - - def initThrottle(self, throttle_value): - - @throttle(seconds=throttle_value) - def sendDataThrottled(fn_sender, data): - fn_sender(data) - - return sendDataThrottled - - def send(self, data): - log.log('Sending {}'.format(data), self.name) - self.sender.send_pyobj(data) - - def run(self): - - log.log( - 'Starting from {}'.format(self.address), - self.name - ) - - self.context = zmq.Context() - self.receiver = self.context.socket(zmq.PULL) - self.receiver.bind(self.address) - monitor = self.receiver.get_monitor_socket() - self.sender = self.context.socket(zmq.PUB) - if self.web_address: - log.success( - 'Sending to web: {}'.format(self.web_address), - self.name - ) - self.sender_web = self.context.socket(zmq.REQ) - self.sender_web.connect(self.web_address) - self.poll = zmq.Poller() - self.poll.register(self.sender_web, zmq.POLLIN) - time.sleep(1) - - if 'server' in self.kwargs: - self.sender.bind("ipc:///tmp/{}.plots".format(self.kwargs['server'])) - else: - self.sender.bind("ipc:///tmp/zmq.plots") - - time.sleep(2) - - t = Thread(target=self.event_monitor, args=(monitor,)) - t.start() - - while True: - dataOut = self.receiver.recv_pyobj() - if not dataOut.flagNoData: - if dataOut.type == 'Parameters': - tm = dataOut.utctimeInit - else: - tm = dataOut.utctime - if dataOut.useLocalTime: - if not self.localtime: - tm += time.timezone - dt = datetime.datetime.fromtimestamp(tm).date() - else: - if self.localtime: - tm -= time.timezone - dt = datetime.datetime.utcfromtimestamp(tm).date() - coerce = False - if dt not in self.dates: - if self.data: - self.data.ended = True - self.send(self.data) - coerce = True - self.data.setup() - self.dates.append(dt) - - self.data.update(dataOut, tm) - - if dataOut.finished is True: - self.connections -= 1 - if self.connections == 0 and dt in self.dates: - self.data.ended = True - self.send(self.data) - # self.data.setup() - time.sleep(1) - break - else: - if self.realtime: - self.send(self.data) - if self.web_address: - retries = 5 - while True: - self.sender_web.send(self.data.jsonify()) - socks = dict(self.poll.poll(5000)) - if socks.get(self.sender_web) == zmq.POLLIN: - reply = self.sender_web.recv_string() - if reply == 'ok': - log.log("Response from server ok", self.name) - break - else: - log.warning("Malformed reply from server: {}".format(reply), self.name) - - else: - log.warning("No response from server, retrying...", self.name) - self.sender_web.setsockopt(zmq.LINGER, 0) - self.sender_web.close() - self.poll.unregister(self.sender_web) - retries -= 1 - if retries == 0: - log.error("Server seems to be offline, abandoning", self.name) - self.sender_web = self.context.socket(zmq.REQ) - self.sender_web.connect(self.web_address) - self.poll.register(self.sender_web, zmq.POLLIN) - time.sleep(1) - break - self.sender_web = self.context.socket(zmq.REQ) - self.sender_web.connect(self.web_address) - self.poll.register(self.sender_web, zmq.POLLIN) - time.sleep(1) - else: - self.sendData(self.send, self.data, coerce=coerce) - coerce = False - - return - - -class SendToFTP(Operation, Process): +@MPDecorator +class SendToFTP(Operation): ''' Operation to send data over FTP. + patternX = 'local, remote, ext, period, exp_code, sub_exp_code' ''' - __attrs__ = ['server', 'username', 'password', 'patterns', 'timeout'] + __attrs__ = ['server', 'username', 'password', 'timeout', 'patternX'] - def __init__(self, **kwargs): + def __init__(self): ''' - patterns = [(local1, remote1, ext, delay, exp_code, sub_exp_code), ...] ''' - Operation.__init__(self, **kwargs) - Process.__init__(self) - self.server = kwargs.get('server') - self.username = kwargs.get('username') - self.password = kwargs.get('password') - self.patterns = kwargs.get('patterns') - self.timeout = kwargs.get('timeout', 30) - self.times = [time.time() for p in self.patterns] - self.latest = ['' for p in self.patterns] - self.mp = False + Operation.__init__(self) self.ftp = None + self.ready = False - def setup(self): + def setup(self, server, username, password, timeout, **kwargs): + ''' + ''' + + self.server = server + self.username = username + self.password = password + self.timeout = timeout + self.patterns = [] + self.times = [] + self.latest = [] + for arg, value in kwargs.items(): + if 'pattern' in arg: + self.patterns.append(value) + self.times.append(time.time()) + self.latest.append('') + + def connect(self): + ''' + ''' log.log('Connecting to ftp://{}'.format(self.server), self.name) try: @@ -749,7 +194,7 @@ class SendToFTP(Operation, Process): if self.ftp is not None: self.ftp.close() self.ftp = None - self.isConfig = False + self.ready = False return try: @@ -759,11 +204,11 @@ class SendToFTP(Operation, Process): if self.ftp is not None: self.ftp.close() self.ftp = None - self.isConfig = False + self.ready = False return log.success('Connection success', self.name) - self.isConfig = True + self.ready = True return def check(self): @@ -775,7 +220,7 @@ class SendToFTP(Operation, Process): if self.ftp is not None: self.ftp.close() self.ftp = None - self.setup() + self.connect() def find_files(self, path, ext): @@ -788,24 +233,28 @@ class SendToFTP(Operation, Process): def getftpname(self, filename, exp_code, sub_exp_code): thisDatetime = datetime.datetime.strptime(filename.split('_')[1], '%Y%m%d') - YEAR_STR = '%4.4d'%thisDatetime.timetuple().tm_year - DOY_STR = '%3.3d'%thisDatetime.timetuple().tm_yday - exp_code = '%3.3d'%exp_code - sub_exp_code = '%2.2d'%sub_exp_code - plot_code = '%2.2d'% get_plot_code(filename) + YEAR_STR = '%4.4d' % thisDatetime.timetuple().tm_year + DOY_STR = '%3.3d' % thisDatetime.timetuple().tm_yday + exp_code = '%3.3d' % exp_code + sub_exp_code = '%2.2d' % sub_exp_code + plot_code = '%2.2d' % get_plot_code(filename) name = YEAR_STR + DOY_STR + '00' + exp_code + sub_exp_code + plot_code + '00.png' return name def upload(self, src, dst): - log.log('Uploading {} '.format(src), self.name, nl=False) + log.log('Uploading {} -> {} '.format( + src.split('/')[-1], dst.split('/')[-1]), + self.name, + nl=False + ) fp = open(src, 'rb') command = 'STOR {}'.format(dst) try: self.ftp.storbinary(command, fp, blocksize=1024) - except Exception, e: + except Exception as e: log.error('{}'.format(e), self.name) if self.ftp is not None: self.ftp.close() @@ -814,7 +263,7 @@ class SendToFTP(Operation, Process): try: self.ftp.sendcmd('SITE CHMOD 755 {}'.format(dst)) - except Exception, e: + except Exception as e: log.error('{}'.format(e), self.name) if self.ftp is not None: self.ftp.close() @@ -828,18 +277,20 @@ class SendToFTP(Operation, Process): def send_files(self): for x, pattern in enumerate(self.patterns): - local, remote, ext, delay, exp_code, sub_exp_code = pattern - if time.time()-self.times[x] >= delay: - srcname = self.find_files(local, ext) - src = os.path.join(local, srcname) - if os.path.getmtime(src) < time.time() - 30*60: + local, remote, ext, period, exp_code, sub_exp_code = pattern + if time.time()-self.times[x] >= int(period): + srcname = self.find_files(local, ext) + src = os.path.join(local, srcname) + if os.path.getmtime(src) < time.time() - 30*60: + log.warning('Skipping old file {}'.format(srcname)) continue if srcname is None or srcname == self.latest[x]: + log.warning('File alreday uploaded {}'.format(srcname)) continue if 'png' in ext: - dstname = self.getftpname(srcname, exp_code, sub_exp_code) + dstname = self.getftpname(srcname, int(exp_code), int(sub_exp_code)) else: dstname = srcname @@ -849,21 +300,27 @@ class SendToFTP(Operation, Process): self.times[x] = time.time() self.latest[x] = srcname else: - self.isConfig = False + self.ready = False break - def run(self): + def run(self, dataOut, server, username, password, timeout=10, **kwargs): - while True: - if not self.isConfig: - self.setup() - if self.ftp is not None: - self.check() - self.send_files() - time.sleep(10) + if not self.isConfig: + self.setup( + server=server, + username=username, + password=password, + timeout=timeout, + **kwargs + ) + self.isConfig = True + if not self.ready: + self.connect() + if self.ftp is not None: + self.check() + self.send_files() - def close(): + def close(self): if self.ftp is not None: self.ftp.close() - self.terminate() diff --git a/schainpy/scripts/USRP_PLOT_THOR.py b/schainpy/scripts/USRP_PLOT_THOR.py new file mode 100644 index 0000000..d2e2eb0 --- /dev/null +++ b/schainpy/scripts/USRP_PLOT_THOR.py @@ -0,0 +1,183 @@ +#!python +''' +''' + +import os, sys +import datetime +import time + +#path = os.path.dirname(os.getcwd()) +#path = os.path.dirname(path) +#sys.path.insert(0, path) + +from schainpy.controller import Project + +desc = "USRP_test" +filename = "USRP_processing.xml" +controllerObj = Project() +controllerObj.setup(id = '191', name='Test_USRP', description=desc) + +############## USED TO PLOT IQ VOLTAGE, POWER AND SPECTRA ############# + +####################################################################### +######PATH DE LECTURA, ESCRITURA, GRAFICOS Y ENVIO WEB################# +####################################################################### +#path = '/media/data/data/vientos/57.2063km/echoes/NCO_Woodman' + + +path = '/home/soporte/data_hdf5' #### with clock 35.16 db noise + +figpath = '/home/soporte/data_hdf5_imag' +#remotefolder = "/home/wmaster/graficos" +####################################################################### +################# RANGO DE PLOTEO###################################### +####################################################################### +dBmin = '30' +dBmax = '60' +xmin = '0' +xmax ='24' +ymin = '0' +ymax = '600' +####################################################################### +########################FECHA########################################## +####################################################################### +str = datetime.date.today() +today = str.strftime("%Y/%m/%d") +str2 = str - datetime.timedelta(days=1) +yesterday = str2.strftime("%Y/%m/%d") +####################################################################### +######################## UNIDAD DE LECTURA############################# +####################################################################### +readUnitConfObj = controllerObj.addReadUnit(datatype='DigitalRFReader', + path=path, + startDate="2019/01/01",#today, + endDate="2109/12/30",#today, + startTime='00:00:00', + endTime='23:59:59', + delay=0, + #set=0, + online=0, + walk=1, + ippKm = 1000) + +opObj11 = readUnitConfObj.addOperation(name='printInfo') +opObj11 = readUnitConfObj.addOperation(name='printNumberOfBlock') +####################################################################### +################ OPERACIONES DOMINIO DEL TIEMPO######################## +####################################################################### + +procUnitConfObjA = controllerObj.addProcUnit(datatype='VoltageProc', inputId=readUnitConfObj.getId()) +# +# codigo64='1,1,1,0,1,1,0,1,1,1,1,0,0,0,1,0,1,1,1,0,1,1,0,1,0,0,0,1,1,1,0,1,1,1,1,0,1,1,0,1,1,1,1,0,0,0,1,0,0,0,0,1,0,0,1,0,1,1,1,0,0,0,1,0,'+\ +# '1,1,1,0,1,1,0,1,1,1,1,0,0,0,1,0,1,1,1,0,1,1,0,1,0,0,0,1,1,1,0,1,0,0,0,1,0,0,1,0,0,0,0,1,1,1,0,1,1,1,1,0,1,1,0,1,0,0,0,1,1,1,0,1' + +#opObj11 = procUnitConfObjA.addOperation(name='setRadarFrequency') +#opObj11.addParameter(name='frequency', value='30e6', format='float') + +#opObj10 = procUnitConfObjA.addOperation(name='Scope', optype='external') +#opObj10.addParameter(name='id', value='10', format='int') +##opObj10.addParameter(name='xmin', value='0', format='int') +##opObj10.addParameter(name='xmax', value='50', format='int') +#opObj10.addParameter(name='type', value='iq') +#opObj10.addParameter(name='ymin', value='-5000', format='int') +##opObj10.addParameter(name='ymax', value='8500', format='int') + +#opObj10 = procUnitConfObjA.addOperation(name='setH0') +#opObj10.addParameter(name='h0', value='-5000', format='float') + +#opObj11 = procUnitConfObjA.addOperation(name='filterByHeights') +#opObj11.addParameter(name='window', value='1', format='int') + +#codigo='1,1,-1,1,1,-1,1,-1,-1,1,-1,-1,-1,1,-1,-1,-1,1,-1,-1,-1,1,1,1,1,-1,-1,-1' +#opObj11 = procUnitConfObjSousy.addOperation(name='Decoder', optype='other') +#opObj11.addParameter(name='code', value=codigo, format='floatlist') +#opObj11.addParameter(name='nCode', value='1', format='int') +#opObj11.addParameter(name='nBaud', value='28', format='int') + +#opObj11 = procUnitConfObjA.addOperation(name='CohInt', optype='other') +#opObj11.addParameter(name='n', value='100', format='int') + +####################################################################### +########## OPERACIONES DOMINIO DE LA FRECUENCIA######################## +####################################################################### +procUnitConfObjSousySpectra = controllerObj.addProcUnit(datatype='SpectraProc', inputId=procUnitConfObjA.getId()) +procUnitConfObjSousySpectra.addParameter(name='nFFTPoints', value='100', format='int') +procUnitConfObjSousySpectra.addParameter(name='nProfiles', value='100', format='int') +#procUnitConfObjSousySpectra.addParameter(name='pairsList', value='(0,0),(1,1),(0,1)', format='pairsList') + +#opObj13 = procUnitConfObjSousySpectra.addOperation(name='removeDC') +#opObj13.addParameter(name='mode', value='2', format='int') + +#opObj11 = procUnitConfObjSousySpectra.addOperation(name='IncohInt', optype='other') +#opObj11.addParameter(name='n', value='60', format='float') +####################################################################### +########## PLOTEO DOMINIO DE LA FRECUENCIA############################# +####################################################################### +#SpectraPlot + +opObj11 = procUnitConfObjSousySpectra.addOperation(name='SpectraPlot', optype='external') +opObj11.addParameter(name='id', value='1', format='int') +opObj11.addParameter(name='wintitle', value='Spectra', format='str') +#opObj11.addParameter(name='xmin', value=-0.01, format='float') +#opObj11.addParameter(name='xmax', value=0.01, format='float') +#opObj11.addParameter(name='zmin', value=dBmin, format='int') +#opObj11.addParameter(name='zmax', value=dBmax, format='int') +#opObj11.addParameter(name='ymin', value=ymin, format='int') +#opObj11.addParameter(name='ymax', value=ymax, format='int') +opObj11.addParameter(name='showprofile', value='1', format='int') +opObj11.addParameter(name='save', value=figpath, format='str') +opObj11.addParameter(name='save_period', value=10, format='int') + + +#RTIPLOT + +opObj11 = procUnitConfObjSousySpectra.addOperation(name='RTIPlot', optype='external') +opObj11.addParameter(name='id', value='2', format='int') +opObj11.addParameter(name='wintitle', value='RTIPlot', format='str') +#opObj11.addParameter(name='zmin', value=dBmin, format='int') +#opObj11.addParameter(name='zmax', value=dBmax, format='int') +#opObj11.addParameter(name='ymin', value=ymin, format='int') +#opObj11.addParameter(name='ymax', value=ymax, format='int') +opObj11.addParameter(name='xmin', value=0, format='int') +opObj11.addParameter(name='xmax', value=23, format='int') + +opObj11.addParameter(name='showprofile', value='1', format='int') +opObj11.addParameter(name='save', value=figpath, format='str') +opObj11.addParameter(name='save_period', value=10, format='int') + + +# opObj11 = procUnitConfObjSousySpectra.addOperation(name='CrossSpectraPlot', optype='other') +# opObj11.addParameter(name='id', value='3', format='int') +# opObj11.addParameter(name='wintitle', value='CrossSpectraPlot', format='str') +# opObj11.addParameter(name='ymin', value=ymin, format='int') +# opObj11.addParameter(name='ymax', value=ymax, format='int') +# opObj11.addParameter(name='phase_cmap', value='jet', format='str') +# opObj11.addParameter(name='zmin', value=dBmin, format='int') +# opObj11.addParameter(name='zmax', value=dBmax, format='int') +# opObj11.addParameter(name='figpath', value=figures_path, format='str') +# opObj11.addParameter(name='save', value=0, format='bool') +# opObj11.addParameter(name='pairsList', value='(0,1)', format='pairsList') +# # +# opObj11 = procUnitConfObjSousySpectra.addOperation(name='CoherenceMap', optype='other') +# opObj11.addParameter(name='id', value='4', format='int') +# opObj11.addParameter(name='wintitle', value='Coherence', format='str') +# opObj11.addParameter(name='phase_cmap', value='jet', format='str') +# opObj11.addParameter(name='xmin', value=xmin, format='float') +# opObj11.addParameter(name='xmax', value=xmax, format='float') +# opObj11.addParameter(name='figpath', value=figures_path, format='str') +# opObj11.addParameter(name='save', value=0, format='bool') +# opObj11.addParameter(name='pairsList', value='(0,1)', format='pairsList') +# +####################################################################### +############### UNIDAD DE ESCRITURA ################################### +####################################################################### +#opObj11 = procUnitConfObjSousySpectra.addOperation(name='SpectraWriter', optype='other') +#opObj11.addParameter(name='path', value=wr_path) +#opObj11.addParameter(name='blocksPerFile', value='50', format='int') +print ("Escribiendo el archivo XML") +print ("Leyendo el archivo XML") + + + +controllerObj.start() + diff --git a/schainpy/scripts/pedestal_client.py b/schainpy/scripts/pedestal_client.py new file mode 100644 index 0000000..ec88d9e --- /dev/null +++ b/schainpy/scripts/pedestal_client.py @@ -0,0 +1,87 @@ +import numpy +import sys +import zmq +import time +import h5py +import os + +path="/home/alex/Downloads/pedestal" +ext=".hdf5" + +port ="5556" +if len(sys.argv)>1: + port = sys.argv[1] + int(port) + +if len(sys.argv)>2: + port1 = sys.argv[2] + int(port1) + +#Socket to talk to server +context = zmq.Context() +socket = context.socket(zmq.SUB) + +print("Collecting updates from weather server...") +socket.connect("tcp://localhost:%s"%port) + +if len(sys.argv)>2: + socket.connect("tcp://localhost:%s"%port1) + +#Subscribe to zipcode, default is NYC,10001 +topicfilter = "10001" +socket.setsockopt_string(zmq.SUBSCRIBE,topicfilter) +#Process 5 updates +total_value=0 +count= -1 +azi= [] +elev=[] +time0=[] +#for update_nbr in range(250): +while(True): + string= socket.recv() + topic,ang_elev,ang_elev_dec,ang_azi,ang_azi_dec,seconds,seconds_dec= string.split() + ang_azi =float(ang_azi)+1e-3*float(ang_azi_dec) + ang_elev =float(ang_elev)+1e-3*float(ang_elev_dec) + seconds =float(seconds) +1e-6*float(seconds_dec) + azi.append(ang_azi) + elev.append(ang_elev) + time0.append(seconds) + count +=1 + if count == 100: + timetuple=time.localtime() + epoc = time.mktime(timetuple) + #print(epoc) + fullpath = path + ("/" if path[-1]!="/" else "") + + if not os.path.exists(fullpath): + os.mkdir(fullpath) + + azi_array = numpy.array(azi) + elev_array = numpy.array(elev) + time0_array= numpy.array(time0) + pedestal_array=numpy.array([azi,elev,time0]) + count=0 + azi= [] + elev=[] + time0=[] + #print(pedestal_array[0]) + #print(pedestal_array[1]) + + meta='PE' + filex="%s%4.4d%3.3d%10.4d%s"%(meta,timetuple.tm_year,timetuple.tm_yday,epoc,ext) + filename = os.path.join(fullpath,filex) + fp = h5py.File(filename,'w') + #print("Escribiendo HDF5...",epoc) + #·················· Data·....······································ + grp = fp.create_group("Data") + dset = grp.create_dataset("azimuth" , data=pedestal_array[0]) + dset = grp.create_dataset("elevacion", data=pedestal_array[1]) + dset = grp.create_dataset("utc" , data=pedestal_array[2]) + #·················· Metadata······································· + grp = fp.create_group("Metadata") + dset = grp.create_dataset("utctimeInit", data=pedestal_array[2][0]) + timeInterval = pedestal_array[2][1]-pedestal_array[2][0] + dset = grp.create_dataset("timeInterval", data=timeInterval) + fp.close() + +#print ("Average messagedata value for topic '%s' was %dF" % ( topicfilter,total_value / update_nbr)) diff --git a/schainpy/scripts/pedestal_server.py b/schainpy/scripts/pedestal_server.py new file mode 100644 index 0000000..446a76a --- /dev/null +++ b/schainpy/scripts/pedestal_server.py @@ -0,0 +1,48 @@ +########################################################################### +############################### SERVIDOR################################### +######################### SIMULADOR DE PEDESTAL############################ +########################################################################### +import time +import math +import numpy +import struct +from time import sleep +import zmq +import pickle +port="5556" +context = zmq.Context() +socket = context.socket(zmq.PUB) +socket.bind("tcp://*:%s"%port) +###### PARAMETROS DE ENTRADA################################ +print("PEDESTAL RESOLUCION 0.01") +print("MAXIMA VELOCIDAD DEL PEDESTAL") +ang_elev = 4.12 +ang_azi = 30 +velocidad= input ("Ingresa velocidad:") +velocidad= float(velocidad) +print (velocidad) +############################################################ +sleep(3) +print("Start program") +t1 = time.time() +count=0 +while(True): + tmp_vuelta = int(360/velocidad) + t1=t1+tmp_vuelta*count + count= count+1 + muestras_seg = 100 + t2 = time.time() + for i in range(tmp_vuelta): + for j in range(muestras_seg): + tmp_variable = (i+j/100.0) + ang_azi = (tmp_variable)*float(velocidad) + seconds = t1+ tmp_variable + topic=10001 + print ("Azim°: ","%.4f"%ang_azi,"Time:" ,"%.5f"%seconds) + seconds_dec=(seconds-int(seconds))*1e6 + ang_azi_dec= (ang_azi-int(ang_azi))*1e3 + ang_elev_dec=(ang_elev-int(ang_elev))*1e3 + sleep(0.0088) + socket.send_string("%d %d %d %d %d %d %d"%(topic,ang_elev,ang_elev_dec,ang_azi,ang_azi_dec,seconds,seconds_dec)) + t3 = time.time() + print ("Total time for 1 vuelta in Seconds",t3-t2) diff --git a/schainpy/scripts/schain.xml b/schainpy/scripts/schain.xml deleted file mode 100644 index 80c9912..0000000 --- a/schainpy/scripts/schain.xml +++ /dev/null @@ -1 +0,0 @@ - diff --git a/schainpy/scripts/test_001.py b/schainpy/scripts/test_001.py new file mode 100644 index 0000000..a77c4dd --- /dev/null +++ b/schainpy/scripts/test_001.py @@ -0,0 +1,189 @@ +#!python +''' +''' + +import os, sys +import datetime +import time + +#path = os.path.dirname(os.getcwd()) +#path = os.path.dirname(path) +#sys.path.insert(0, path) + +from schainpy.controller import Project + +desc = "USRP_test" +filename = "USRP_processing.xml" +controllerObj = Project() +controllerObj.setup(id = '191', name='Test_USRP', description=desc) + +############## USED TO PLOT IQ VOLTAGE, POWER AND SPECTRA ############# + +####################################################################### +######PATH DE LECTURA, ESCRITURA, GRAFICOS Y ENVIO WEB################# +####################################################################### +#path = '/media/data/data/vientos/57.2063km/echoes/NCO_Woodman' + + +#path = '/home/soporte/data_hdf5' #### with clock 35.16 db noise +path = '/home/alex/Downloads' +figpath = '/home/alex/Downloads' +#figpath = '/home/soporte/data_hdf5_imag' +#remotefolder = "/home/wmaster/graficos" +####################################################################### +################# RANGO DE PLOTEO###################################### +####################################################################### +dBmin = '30' +dBmax = '60' +xmin = '0' +xmax ='24' +ymin = '0' +ymax = '600' +####################################################################### +########################FECHA########################################## +####################################################################### +str = datetime.date.today() +today = str.strftime("%Y/%m/%d") +str2 = str - datetime.timedelta(days=1) +yesterday = str2.strftime("%Y/%m/%d") +####################################################################### +######################## UNIDAD DE LECTURA############################# +####################################################################### +readUnitConfObj = controllerObj.addReadUnit(datatype='VoltageReader', + path=path, + startDate="2020/01/01", #"2020/01/01",#today, + endDate= "2020/12/01", #"2020/12/30",#today, + startTime='00:00:00', + endTime='23:59:59', + delay=0, + #set=0, + online=0, + walk=1) + +opObj11 = readUnitConfObj.addOperation(name='printInfo') +#opObj11 = readUnitConfObj.addOperation(name='printNumberOfBlock') +####################################################################### +################ OPERACIONES DOMINIO DEL TIEMPO######################## +####################################################################### + +procUnitConfObjA = controllerObj.addProcUnit(datatype='VoltageProc', inputId=readUnitConfObj.getId()) +# +# codigo64='1,1,1,0,1,1,0,1,1,1,1,0,0,0,1,0,1,1,1,0,1,1,0,1,0,0,0,1,1,1,0,1,1,1,1,0,1,1,0,1,1,1,1,0,0,0,1,0,0,0,0,1,0,0,1,0,1,1,1,0,0,0,1,0,'+\ +# '1,1,1,0,1,1,0,1,1,1,1,0,0,0,1,0,1,1,1,0,1,1,0,1,0,0,0,1,1,1,0,1,0,0,0,1,0,0,1,0,0,0,0,1,1,1,0,1,1,1,1,0,1,1,0,1,0,0,0,1,1,1,0,1' + +#opObj11 = procUnitConfObjA.addOperation(name='setRadarFrequency') +#opObj11.addParameter(name='frequency', value='30e6', format='float') + +#opObj10 = procUnitConfObjA.addOperation(name='Scope', optype='external') +#opObj10.addParameter(name='id', value='10', format='int') +##opObj10.addParameter(name='xmin', value='0', format='int') +##opObj10.addParameter(name='xmax', value='50', format='int') +#opObj10.addParameter(name='type', value='iq') +#opObj10.addParameter(name='ymin', value='-5000', format='int') +##opObj10.addParameter(name='ymax', value='8500', format='int') + +#opObj10 = procUnitConfObjA.addOperation(name='setH0') +#opObj10.addParameter(name='h0', value='-5000', format='float') + +#opObj11 = procUnitConfObjA.addOperation(name='filterByHeights') +#opObj11.addParameter(name='window', value='1', format='int') + +#codigo='1,1,-1,1,1,-1,1,-1,-1,1,-1,-1,-1,1,-1,-1,-1,1,-1,-1,-1,1,1,1,1,-1,-1,-1' +#opObj11 = procUnitConfObjSousy.addOperation(name='Decoder', optype='other') +#opObj11.addParameter(name='code', value=codigo, format='floatlist') +#opObj11.addParameter(name='nCode', value='1', format='int') +#opObj11.addParameter(name='nBaud', value='28', format='int') + +#opObj11 = procUnitConfObjA.addOperation(name='CohInt', optype='other') +#opObj11.addParameter(name='n', value='100', format='int') + +####################################################################### +########## OPERACIONES DOMINIO DE LA FRECUENCIA######################## +####################################################################### +procUnitConfObjB = controllerObj.addProcUnit(datatype='SpectraProc', inputId=procUnitConfObjA.getId()) +procUnitConfObjB.addParameter(name='nFFTPoints', value='100', format='int') +procUnitConfObjB.addParameter(name='nProfiles', value='100', format='int') +#procUnitConfObjSousySpectra.addParameter(name='pairsList', value='(0,0),(1,1),(0,1)', format='pairsList') + +#opObj13 = procUnitConfObjSousySpectra.addOperation(name='removeDC') +#opObj13.addParameter(name='mode', value='2', format='int') + +#opObj11 = procUnitConfObjSousySpectra.addOperation(name='IncohInt', optype='other') +#opObj11.addParameter(name='n', value='60', format='float') +####################################################################### +########## PLOTEO DOMINIO DE LA FRECUENCIA############################# +####################################################################### +#SpectraPlot + +##opObj11 = procUnitConfObjB.addOperation(name='SpectraPlot', optype='external') +##opObj11.addParameter(name='id', value='1', format='int') +##opObj11.addParameter(name='wintitle', value='Spectra', format='str') +#opObj11.addParameter(name='xmin', value=-0.01, format='float') +#opObj11.addParameter(name='xmax', value=0.01, format='float') +#opObj11.addParameter(name='zmin', value=dBmin, format='int') +#opObj11.addParameter(name='zmax', value=dBmax, format='int') +#opObj11.addParameter(name='ymin', value=ymin, format='int') +#opObj11.addParameter(name='ymax', value=ymax, format='int') +##opObj11.addParameter(name='showprofile', value='1', format='int') +##opObj11.addParameter(name='save', value=figpath, format='str') +##opObj11.addParameter(name='save_period', value=10, format='int') + + +#RTIPLOT + +##opObj11 = procUnitConfObjB.addOperation(name='RTIPlot', optype='external') +##opObj11.addParameter(name='id', value='2', format='int') +##opObj11.addParameter(name='wintitle', value='RTIPlot', format='str') +#opObj11.addParameter(name='zmin', value=dBmin, format='int') +#opObj11.addParameter(name='zmax', value=dBmax, format='int') +#opObj11.addParameter(name='ymin', value=ymin, format='int') +#opObj11.addParameter(name='ymax', value=ymax, format='int') +##opObj11.addParameter(name='xmin', value=0, format='int') +##opObj11.addParameter(name='xmax', value=23, format='int') + +##opObj11.addParameter(name='showprofile', value='1', format='int') +##opObj11.addParameter(name='save', value=figpath, format='str') +##opObj11.addParameter(name='save_period', value=10, format='int') + + +# opObj11 = procUnitConfObjSousySpectra.addOperation(name='CrossSpectraPlot', optype='other') +# opObj11.addParameter(name='id', value='3', format='int') +# opObj11.addParameter(name='wintitle', value='CrossSpectraPlot', format='str') +# opObj11.addParameter(name='ymin', value=ymin, format='int') +# opObj11.addParameter(name='ymax', value=ymax, format='int') +# opObj11.addParameter(name='phase_cmap', value='jet', format='str') +# opObj11.addParameter(name='zmin', value=dBmin, format='int') +# opObj11.addParameter(name='zmax', value=dBmax, format='int') +# opObj11.addParameter(name='figpath', value=figures_path, format='str') +# opObj11.addParameter(name='save', value=0, format='bool') +# opObj11.addParameter(name='pairsList', value='(0,1)', format='pairsList') +# # +# opObj11 = procUnitConfObjSousySpectra.addOperation(name='CoherenceMap', optype='other') +# opObj11.addParameter(name='id', value='4', format='int') +# opObj11.addParameter(name='wintitle', value='Coherence', format='str') +# opObj11.addParameter(name='phase_cmap', value='jet', format='str') +# opObj11.addParameter(name='xmin', value=xmin, format='float') +# opObj11.addParameter(name='xmax', value=xmax, format='float') +# opObj11.addParameter(name='figpath', value=figures_path, format='str') +# opObj11.addParameter(name='save', value=0, format='bool') +# opObj11.addParameter(name='pairsList', value='(0,1)', format='pairsList') +# +####################################################################### +############### UNIDAD DE ESCRITURA ################################### +####################################################################### +#opObj11 = procUnitConfObjSousySpectra.addOperation(name='SpectraWriter', optype='other') +#opObj11.addParameter(name='path', value=wr_path) +#opObj11.addParameter(name='blocksPerFile', value='50', format='int') + +procUnitConfObjC = controllerObj.addProcUnit(datatype='ParametersProc', inputId=procUnitConfObjB.getId()) +procUnitConfObjC.addOperation(name='SpectralMoments') + +procUnitConfObjC.addOperation(name='SpectralMomentsPlot') + + +print ("Escribiendo el archivo XML") +print ("Leyendo el archivo XML") + + + +controllerObj.start() diff --git a/schainpy/scripts/test_002.py b/schainpy/scripts/test_002.py new file mode 100644 index 0000000..f3370d8 --- /dev/null +++ b/schainpy/scripts/test_002.py @@ -0,0 +1,87 @@ +#!python +''' +''' + +import os, sys +import datetime +import time + +#path = os.path.dirname(os.getcwd()) +#path = os.path.dirname(path) +#sys.path.insert(0, path) + +from schainpy.controller import Project + +desc = "USRP_test" +filename = "USRP_processing.xml" +controllerObj = Project() +controllerObj.setup(id = '191', name='Test_USRP', description=desc) + +############## USED TO PLOT IQ VOLTAGE, POWER AND SPECTRA ############# + +####################################################################### +######PATH DE LECTURA, ESCRITURA, GRAFICOS Y ENVIO WEB################# +####################################################################### +#path = '/media/data/data/vientos/57.2063km/echoes/NCO_Woodman' + + +#path = '/home/soporte/data_hdf5' #### with clock 35.16 db noise +path = '/home/alex/Downloads' +figpath = '/home/alex/Downloads' +pathfile = '/home/alex/Downloads/test_rawdata2' + +#figpath = '/home/soporte/data_hdf5_imag' +#remotefolder = "/home/wmaster/graficos" +####################################################################### +################# RANGO DE PLOTEO###################################### +####################################################################### +dBmin = '30' +dBmax = '60' +xmin = '0' +xmax ='24' +ymin = '0' +ymax = '600' +####################################################################### +########################FECHA########################################## +####################################################################### +str = datetime.date.today() +today = str.strftime("%Y/%m/%d") +str2 = str - datetime.timedelta(days=1) +yesterday = str2.strftime("%Y/%m/%d") +####################################################################### +######################## UNIDAD DE LECTURA############################# +####################################################################### +readUnitConfObj = controllerObj.addReadUnit(datatype='VoltageReader', + path=path, + startDate="2020/01/01", #"2020/01/01",#today, + endDate= "2020/12/01", #"2020/12/30",#today, + startTime='00:00:00', + endTime='23:59:59', + delay=0, + #set=0, + online=0, + walk=1) + +opObj11 = readUnitConfObj.addOperation(name='printInfo') +#opObj11 = readUnitConfObj.addOperation(name='printNumberOfBlock') +####################################################################### +################ OPERACIONES DOMINIO DEL TIEMPO######################## +####################################################################### + +procUnitConfObjA = controllerObj.addProcUnit(datatype='VoltageProc', inputId=readUnitConfObj.getId()) + + +opObj12 = procUnitConfObjA.addOperation(name='VoltageWriter', optype='other') +opObj12.addParameter(name='path', value=pathfile) +opObj12.addParameter(name='blocksPerFile', value='120', format='int') +opObj12.addParameter(name='profilesPerBlock', value='300', format='int') + + + + +print ("Escribiendo el archivo XML") +print ("Leyendo el archivo XML") + + + +controllerObj.start() diff --git a/schainpy/scripts/test_sim0001.py b/schainpy/scripts/test_sim0001.py new file mode 100644 index 0000000..fe8e982 --- /dev/null +++ b/schainpy/scripts/test_sim0001.py @@ -0,0 +1,61 @@ +import os,sys +import datetime +import time +from schainpy.controller import Project +path = '/home/alex/Downloads/NEW_WR2' +figpath = path +desc = "Simulator Test" + +controllerObj = Project() + +controllerObj.setup(id='10',name='Test Simulator',description=desc) + +readUnitConfObj = controllerObj.addReadUnit(datatype='SimulatorReader', + frequency=9.345e9, + FixRCP_IPP= 60, + Tau_0 = 30, + AcqH0_0=0, + samples=330, + AcqDH_0=0.15, + FixRCP_TXA=0.15, + FixRCP_TXB=0.15, + Fdoppler=200.0, + Hdoppler=36, + Adoppler=300, + delay=0, + online=0, + walk=0) + +#opObj11 = readUnitConfObj.addOperation(name='printInfo') + +procUnitConfObjA = controllerObj.addProcUnit(datatype='VoltageProc', inputId=readUnitConfObj.getId()) + +#opObj10 = procUnitConfObjA.addOperation(name='selectChannels') +#opObj10.addParameter(name='channelList', value=[0,1]) +#opObj10.addParameter(name='channelList', value='0',format='intlist') + +opObj11 = procUnitConfObjA.addOperation(name='PulsePairVoltage', optype='other') +opObj11.addParameter(name='n', value='32', format='int') +opObj11.addParameter(name='removeDC', value=1, format='int') +''' +type="power" +opObj10 = procUnitConfObjA.addOperation(name='ScopePlot', optype='external') +#opObj10.addParameter(name='id', value='12') +opObj10.addParameter(name='wintitle', value=type ) +opObj10.addParameter(name='type', value=type) +''' + +type="WeatherPower" +opObj10 = procUnitConfObjA.addOperation(name='PulsepairPowerPlot', optype='external') +#opObj10.addParameter(name='id', value='12') +opObj10.addParameter(name='wintitle', value=type ) + +''' +type="WeatherVeloity" + +opObj10 = procUnitConfObjA.addOperation(name='PulsepairVelocityPlot', optype='external') +#opObj10.addParameter(name='id', value='12') +opObj10.addParameter(name='wintitle', value=type ) +''' + +controllerObj.start() diff --git a/schainpy/scripts/test_sim00010.py b/schainpy/scripts/test_sim00010.py new file mode 100644 index 0000000..8710965 --- /dev/null +++ b/schainpy/scripts/test_sim00010.py @@ -0,0 +1,82 @@ +import os, sys +import datetime +import time +from schainpy.controller import Project + +desc = "USRP_test" +filename = "USRP_processing.xml" +controllerObj = Project() +controllerObj.setup(id = '191', name='Test_USRP', description=desc) + +############## USED TO PLOT IQ VOLTAGE, POWER AND SPECTRA ############# +######PATH DE LECTURA, ESCRITURA, GRAFICOS Y ENVIO WEB################# +path = '/home/alex/Downloads/test_rawdata' +figpath = '/home/alex/Downloads/hdf5_test' +######################## UNIDAD DE LECTURA############################# +''' +readUnitConfObj = controllerObj.addReadUnit(datatype='VoltageReader', + path=path, + startDate="2020/01/01", #"2020/01/01",#today, + endDate= "2020/12/01", #"2020/12/30",#today, + startTime='00:00:00', + endTime='23:59:59', + delay=0, + #set=0, + online=0, + walk=1) + +''' +readUnitConfObj = controllerObj.addReadUnit(datatype='SimulatorReader', + frequency=9.345e9, + FixRCP_IPP= 60, + Tau_0 = 30, + AcqH0_0=0, + samples=330, + AcqDH_0=0.15, + FixRCP_TXA=0.15, + FixRCP_TXB=0.15, + Fdoppler=600.0, + Hdoppler=36, + Adoppler=300,#300 + delay=0, + online=0, + walk=0, + profilesPerBlock=625, + dataBlocksPerFile=100) + #nTotalReadFiles=2) + + +#opObj11 = readUnitConfObj.addOperation(name='printInfo') + +procUnitConfObjA = controllerObj.addProcUnit(datatype='VoltageProc', inputId=readUnitConfObj.getId()) + +procUnitConfObjB = controllerObj.addProcUnit(datatype='SpectraProc', inputId=procUnitConfObjA.getId()) +procUnitConfObjB.addParameter(name='nFFTPoints', value=625, format='int') +procUnitConfObjB.addParameter(name='nProfiles', value=625, format='int') + +opObj11 = procUnitConfObjB.addOperation(name='removeDC') +opObj11.addParameter(name='mode', value=2) +#opObj11 = procUnitConfObjB.addOperation(name='SpectraPlot') +#opObj11 = procUnitConfObjB.addOperation(name='PowerProfilePlot') + +procUnitConfObjC= controllerObj.addProcUnit(datatype='ParametersProc',inputId=procUnitConfObjB.getId()) +procUnitConfObjC.addOperation(name='SpectralMoments') +#opObj11 = procUnitConfObjC.addOperation(name='PowerPlot') + +''' +opObj11 = procUnitConfObjC.addOperation(name='SpectralMomentsPlot') +#opObj11.addParameter(name='xmin', value=14) +#opObj11.addParameter(name='xmax', value=15) +#opObj11.addParameter(name='save', value=figpath) +opObj11.addParameter(name='showprofile', value=1) +#opObj11.addParameter(name='save_period', value=10) +''' + +opObj10 = procUnitConfObjC.addOperation(name='ParameterWriter') +opObj10.addParameter(name='path',value=figpath) +#opObj10.addParameter(name='mode',value=0) +opObj10.addParameter(name='blocksPerFile',value='100',format='int') +opObj10.addParameter(name='metadataList',value='utctimeInit,timeInterval',format='list') +opObj10.addParameter(name='dataList',value='data_POW,data_DOP,data_WIDTH,data_SNR')#,format='list' + +controllerObj.start() diff --git a/schainpy/scripts/test_sim0002.py b/schainpy/scripts/test_sim0002.py new file mode 100644 index 0000000..7feeec7 --- /dev/null +++ b/schainpy/scripts/test_sim0002.py @@ -0,0 +1,56 @@ +import os,sys +import datetime +import time +from schainpy.controller import Project +path = '/home/alex/Downloads/NEW_WR2/spc16removeDC' +figpath = path +desc = "Simulator Test" + +controllerObj = Project() + +controllerObj.setup(id='10',name='Test Simulator',description=desc) + +readUnitConfObj = controllerObj.addReadUnit(datatype='SimulatorReader', + frequency=9.345e9, + FixRCP_IPP= 60, + Tau_0 = 30, + AcqH0_0=0, + samples=330, + AcqDH_0=0.15, + FixRCP_TXA=0.15, + FixRCP_TXB=0.15, + Fdoppler=200.0, + Hdoppler=36, + Adoppler=300, + delay=0, + online=0, + walk=0, + nTotalReadFiles=4) + +opObj11 = readUnitConfObj.addOperation(name='printInfo') + +procUnitConfObjA = controllerObj.addProcUnit(datatype='VoltageProc', inputId=readUnitConfObj.getId()) + +opObj10 = procUnitConfObjA.addOperation(name='selectChannels') +opObj10.addParameter(name='channelList', value=[0,1]) + +procUnitConfObjB = controllerObj.addProcUnit(datatype='SpectraProc', inputId=procUnitConfObjA.getId()) +procUnitConfObjB.addParameter(name='nFFTPoints', value=200, format='int') +procUnitConfObjB.addParameter(name='nProfiles', value=200, format='int') + +opObj11 = procUnitConfObjB.addOperation(name='removeDC') +opObj11.addParameter(name='mode', value=2) + +#opObj11 = procUnitConfObjB.addOperation(name='IncohInt', optype='other') +#opObj11.addParameter(name='n', value='20', format='int') + +procUnitConfObjC = controllerObj.addProcUnit(datatype='ParametersProc', inputId=procUnitConfObjB.getId()) +procUnitConfObjC.addOperation(name='SpectralMoments') + +opObj11 = procUnitConfObjC.addOperation(name='SpectralMomentsPlot') +opObj11.addParameter(name='xmax', value=6) +opObj11.addParameter(name='save', value=figpath) +opObj11.addParameter(name='showprofile', value=1) +opObj11.addParameter(name='save_period', value=10) + +controllerObj.start() diff --git a/schainpy/scripts/test_sim0003.py b/schainpy/scripts/test_sim0003.py new file mode 100644 index 0000000..8a43732 --- /dev/null +++ b/schainpy/scripts/test_sim0003.py @@ -0,0 +1,40 @@ +import os,sys +import datetime +import time +from schainpy.controller import Project +path = '/home/alex/Downloads/NEW_WR2/spc16removeDC' +figpath = path +desc = "Simulator Test" + +controllerObj = Project() + +controllerObj.setup(id='10',name='Test Simulator',description=desc) + +readUnitConfObj = controllerObj.addReadUnit(datatype='SimulatorReader', + frequency=9.345e9, + FixRCP_IPP= 60, + Tau_0 = 30, + AcqH0_0=0, + samples=330, + AcqDH_0=0.15, + FixRCP_TXA=0.15, + FixRCP_TXB=0.15, + Fdoppler=200.0, + Hdoppler=36, + Adoppler=300, + delay=0, + online=0, + walk=0, + nTotalReadFiles=4) + +opObj11 = readUnitConfObj.addOperation(name='printInfo') +procUnitConfObjA = controllerObj.addProcUnit(datatype='VoltageProc', inputId=readUnitConfObj.getId()) + +opObj10 = procUnitConfObjA.addOperation(name='selectChannels') +opObj10.addParameter(name='channelList', value=[0]) + +opObj11 = procUnitConfObjA.addOperation(name='PulsePairVoltage', optype='other') +opObj11.addParameter(name='n', value='32', format='int')#10 +#opObj11.addParameter(name='removeDC', value=1, format='int') + +controllerObj.start() diff --git a/schainpy/scripts/test_sim0004.py b/schainpy/scripts/test_sim0004.py new file mode 100644 index 0000000..539feb9 --- /dev/null +++ b/schainpy/scripts/test_sim0004.py @@ -0,0 +1,39 @@ +import os,sys +import datetime +import time +from schainpy.controller import Project +path = '/home/alex/Downloads/NEW_WR2' +pathfile = '/home/alex/Downloads/test_rawdata' +figpath = path +desc = "Simulator Test" + +controllerObj = Project() + +controllerObj.setup(id='10',name='Test Simulator',description=desc) + +readUnitConfObj = controllerObj.addReadUnit(datatype='SimulatorReader', + frequency=9.345e9, + FixRCP_IPP= 60, + Tau_0 = 30.0, + AcqH0_0=0, + samples=330, + AcqDH_0=0.15, + FixRCP_TXA=0.15, + FixRCP_TXB=0.15, + Fdoppler=200.0, + Hdoppler=36, + Adoppler=300, + delay=0, + online=0, + walk=0, + nTotalReadFiles=3) +#opObj11 = readUnitConfObj.addOperation(name='printInfo') +procUnitConfObjA = controllerObj.addProcUnit(datatype='VoltageProc', inputId=readUnitConfObj.getId()) +#opObj10 = procUnitConfObjA.addOperation(name='selectChannels') +#opObj10.addParameter(name='channelList', value=[0,1]) +#opObj10.addParameter(name='channelList', value='0',format='intlist') +opObj12 = procUnitConfObjA.addOperation(name='VoltageWriter', optype='other') +opObj12.addParameter(name='path', value=pathfile) +opObj12.addParameter(name='blocksPerFile', value='120', format='int') +opObj12.addParameter(name='profilesPerBlock', value='300', format='int') +controllerObj.start() diff --git a/schainpy/scripts/test_sim0005.py b/schainpy/scripts/test_sim0005.py new file mode 100644 index 0000000..fd917af --- /dev/null +++ b/schainpy/scripts/test_sim0005.py @@ -0,0 +1,72 @@ +import os, sys +import datetime +import time +from schainpy.controller import Project + +desc = "USRP_test" +filename = "USRP_processing.xml" +controllerObj = Project() +controllerObj.setup(id = '191', name='Test_USRP', description=desc) + +############## USED TO PLOT IQ VOLTAGE, POWER AND SPECTRA ############# +######PATH DE LECTURA, ESCRITURA, GRAFICOS Y ENVIO WEB################# +path = '/home/alex/Downloads/test_rawdata' +figpath = '/home/alex/Downloads' +################# RANGO DE PLOTEO###################################### +dBmin = '30' +dBmax = '60' +xmin = '0' +xmax ='24' +ymin = '0' +ymax = '600' +########################FECHA########################################## +str = datetime.date.today() +today = str.strftime("%Y/%m/%d") +str2 = str - datetime.timedelta(days=1) +yesterday = str2.strftime("%Y/%m/%d") +######################## UNIDAD DE LECTURA############################# +readUnitConfObj = controllerObj.addReadUnit(datatype='VoltageReader', + path=path, + startDate="2020/01/01", #"2020/01/01",#today, + endDate= "2020/12/01", #"2020/12/30",#today, + startTime='00:00:00', + endTime='23:59:59', + delay=0, + #set=0, + online=0, + walk=1) + +opObj11 = readUnitConfObj.addOperation(name='printInfo') +#opObj11 = readUnitConfObj.addOperation(name='printNumberOfBlock') +####################################################################### +################ OPERACIONES DOMINIO DEL TIEMPO######################## +####################################################################### + +procUnitConfObjA = controllerObj.addProcUnit(datatype='VoltageProc', inputId=readUnitConfObj.getId()) +''' +opObj11 = procUnitConfObjA.addOperation(name='PulsePairVoltage', optype='other') +opObj11.addParameter(name='n', value='256', format='int') +opObj11.addParameter(name='removeDC', value=1, format='int') +''' +''' +type="power" +opObj10 = procUnitConfObjA.addOperation(name='ScopePlot', optype='external') +#opObj10.addParameter(name='id', value='12') +opObj10.addParameter(name='wintitle', value=type ) +opObj10.addParameter(name='type', value=type) +106 32 +102 64 +99 128 +99 256s +''' +''' +type="WeatherPower" +opObj10 = procUnitConfObjA.addOperation(name='PulsepairPowerPlot', optype='external') +#opObj10.addParameter(name='id', value='12') +opObj10.addParameter(name='wintitle', value=type ) + +opObj11 = procUnitConfObjA.addOperation(name='PulsepairVelocityPlot', optype='other') +opObj11.addParameter(name='xmax', value=8) +''' + +controllerObj.start() diff --git a/schainpy/scripts/test_sim0006.py b/schainpy/scripts/test_sim0006.py new file mode 100644 index 0000000..1646f20 --- /dev/null +++ b/schainpy/scripts/test_sim0006.py @@ -0,0 +1,39 @@ +import os, sys +import datetime +import time +from schainpy.controller import Project + +desc = "USRP_test" +filename = "USRP_processing.xml" +controllerObj = Project() +controllerObj.setup(id = '191', name='Test_USRP', description=desc) + +############## USED TO PLOT IQ VOLTAGE, POWER AND SPECTRA ############# +######PATH DE LECTURA, ESCRITURA, GRAFICOS Y ENVIO WEB################# +path = '/home/alex/Downloads/test_rawdata' +figpath = '/home/alex/Downloads' +######################## UNIDAD DE LECTURA############################# +readUnitConfObj = controllerObj.addReadUnit(datatype='VoltageReader', + path=path, + startDate="2020/01/01", #"2020/01/01",#today, + endDate= "2020/12/01", #"2020/12/30",#today, + startTime='00:00:00', + endTime='23:59:59', + delay=0, + #set=0, + online=0, + walk=1) + +opObj11 = readUnitConfObj.addOperation(name='printInfo') + +procUnitConfObjA = controllerObj.addProcUnit(datatype='VoltageProc', inputId=readUnitConfObj.getId()) + +#opObj10 = procUnitConfObjA.addOperation(name='selectChannels') +#opObj10.addParameter(name='channelList', value=[0,1]) +#opObj10.addParameter(name='channelList', value='0',format='intlist') + +opObj11 = procUnitConfObjA.addOperation(name='PulsePairVoltage', optype='other') +opObj11.addParameter(name='n', value='16', format='int') +opObj11.addParameter(name='removeDC', value=1, format='int') + +controllerObj.start() diff --git a/schainpy/scripts/test_sim0007.py b/schainpy/scripts/test_sim0007.py new file mode 100644 index 0000000..92ee891 --- /dev/null +++ b/schainpy/scripts/test_sim0007.py @@ -0,0 +1,53 @@ +import os, sys +import datetime +import time +from schainpy.controller import Project + +desc = "USRP_test" +filename = "USRP_processing.xml" +controllerObj = Project() +controllerObj.setup(id = '191', name='Test_USRP', description=desc) + +############## USED TO PLOT IQ VOLTAGE, POWER AND SPECTRA ############# +######PATH DE LECTURA, ESCRITURA, GRAFICOS Y ENVIO WEB################# +path = '/home/alex/Downloads/test_rawdata' +figpath = '/home/alex/Downloads' +######################## UNIDAD DE LECTURA############################# +readUnitConfObj = controllerObj.addReadUnit(datatype='VoltageReader', + path=path, + startDate="2020/01/01", #"2020/01/01",#today, + endDate= "2020/12/01", #"2020/12/30",#today, + startTime='00:00:00', + endTime='23:59:59', + delay=0, + #set=0, + online=0, + walk=1) + +opObj11 = readUnitConfObj.addOperation(name='printInfo') + +procUnitConfObjA = controllerObj.addProcUnit(datatype='VoltageProc', inputId=readUnitConfObj.getId()) + +procUnitConfObjB = controllerObj.addProcUnit(datatype='SpectraProc', inputId=procUnitConfObjA.getId()) +procUnitConfObjB.addParameter(name='nFFTPoints', value=64, format='int') +procUnitConfObjB.addParameter(name='nProfiles', value=64, format='int') +''' +32 99.96 113.11 529.94s +64 97.3 122.96 326.26 +128 96.29 100.23 230 894 +256 98.65 102.83 182 640 +''' +opObj11 = procUnitConfObjB.addOperation(name='removeDC') +opObj11.addParameter(name='mode', value=2) + +#opObj11 = procUnitConfObjB.addOperation(name='IncohInt', optype='other') +#opObj11.addParameter(name='n', value='20', format='int') + +procUnitConfObjC = controllerObj.addProcUnit(datatype='ParametersProc', inputId=procUnitConfObjB.getId()) +procUnitConfObjC.addOperation(name='SpectralMoments') +opObj11 = procUnitConfObjC.addOperation(name='SpectralMomentsPlot') +#opObj11.addParameter(name='xmin', value=14) +opObj11.addParameter(name='xmax', value=6) +#opObj11.addParameter(name='save', value=figpath) +opObj11.addParameter(name='showprofile', value=1) +controllerObj.start() diff --git a/schainpy/scripts/test_sim0008.py b/schainpy/scripts/test_sim0008.py new file mode 100644 index 0000000..5d53ccd --- /dev/null +++ b/schainpy/scripts/test_sim0008.py @@ -0,0 +1,80 @@ +import os,sys +import datetime +import time +from schainpy.controller import Project +path = '/home/alex/Downloads/NEW_WR2/spc16removeDC' +figpath = path +desc = "Simulator Test" + +controllerObj = Project() + +controllerObj.setup(id='10',name='Test Simulator',description=desc) + +readUnitConfObj = controllerObj.addReadUnit(datatype='SimulatorReader', + frequency=9.345e9, + FixRCP_IPP= 60, + Tau_0 = 30, + AcqH0_0=0, + samples=330, + AcqDH_0=0.15, + FixRCP_TXA=0.15, + FixRCP_TXB=0.15, + Fdoppler=600.0, + Hdoppler=36, + Adoppler=300, + delay=0, + online=0, + walk=0, + nTotalReadFiles=3) + +opObj11 = readUnitConfObj.addOperation(name='printInfo') + +procUnitConfObjA = controllerObj.addProcUnit(datatype='VoltageProc', inputId=readUnitConfObj.getId()) + +opObj10 = procUnitConfObjA.addOperation(name='selectChannels') +opObj10.addParameter(name='channelList', value=[0]) + +procUnitConfObjB = controllerObj.addProcUnit(datatype='SpectraProc', inputId=procUnitConfObjA.getId()) +procUnitConfObjB.addParameter(name='nFFTPoints', value=300, format='int') +procUnitConfObjB.addParameter(name='nProfiles', value=300, format='int') + +opObj11 = procUnitConfObjB.addOperation(name='removeDC') +opObj11.addParameter(name='mode', value=2) + +#opObj11 = procUnitConfObjB.addOperation(name='IncohInt', optype='other') +#opObj11.addParameter(name='n', value='10', format='int') + +#opObj11 = procUnitConfObjB.addOperation(name='SpectraPlot') +#opObj11 = procUnitConfObjB.addOperation(name='PowerProfilePlot') +#opObj11.addParameter(name='xmin', value=13) +#opObj11.addParameter(name='xmax', value=.4) +#opObj11 = procUnitConfObjB.addOperation(name='NoisePlot') +#opObj11.addParameter(name='xmin', value=13) +#opObj11.addParameter(name='xmax', value=14) + + +procUnitConfObjC = controllerObj.addProcUnit(datatype='ParametersProc', inputId=procUnitConfObjB.getId()) +procUnitConfObjC.addOperation(name='SpectralMoments') + +opObj11 = procUnitConfObjC.addOperation(name='SpectralMomentsPlot') +#opObj11.addParameter(name='xmin', value=14) +#opObj11.addParameter(name='xmax', value=15) +#opObj11.addParameter(name='save', value=figpath) +opObj11.addParameter(name='showprofile', value=1) +#opObj11.addParameter(name='save_period', value=10) +''' +opObj11 = procUnitConfObjC.addOperation(name='SnrPlot') +opObj11.addParameter(name='zmin', value=-10) +opObj11.addParameter(name='zmax', value=40) +#opObj11.addParameter(name='save', value=figpath) +#opObj11.addParameter(name='showprofile', value=1) +#opObj11.addParameter(name='save_period', value=10) +''' +opObj11 = procUnitConfObjC.addOperation(name='SpectralWidthPlot') +opObj11.addParameter(name='xmin', value=5) +opObj11.addParameter(name='xmax', value=6) +#opObj11.addParameter(name='save', value=figpath) +#opObj11.addParameter(name='showprofile', value=1) +#opObj11.addParameter(name='save_period', value=10) + +controllerObj.start() diff --git a/schainpy/scripts/test_sim0009.py b/schainpy/scripts/test_sim0009.py new file mode 100644 index 0000000..43d6640 --- /dev/null +++ b/schainpy/scripts/test_sim0009.py @@ -0,0 +1,73 @@ +import os,sys +import datetime +import time +from schainpy.controller import Project +path = '/home/alex/Downloads/NEW_WR2/spc16removeDC' +figpath = path +desc = "Simulator Test" + +controllerObj = Project() + +controllerObj.setup(id='10',name='Test Simulator',description=desc) + +readUnitConfObj = controllerObj.addReadUnit(datatype='SimulatorReader', + frequency=9.345e9, + FixRCP_IPP= 60, + Tau_0 = 30, + AcqH0_0=0, + samples=330, + AcqDH_0=0.15, + FixRCP_TXA=0.15, + FixRCP_TXB=0.15, + Fdoppler=600.0, + Hdoppler=36, + Adoppler=300,#300 + delay=0, + online=0, + walk=0, + profilesPerBlock=625, + dataBlocksPerFile=100)#,#nTotalReadFiles=2) +''' +readUnitConfObj = controllerObj.addReadUnit(datatype='VoltageReader', + path=path, + startDate="2020/01/01", #"2020/01/01",#today, + endDate= "2020/12/01", #"2020/12/30",#today, + startTime='00:00:00', + endTime='23:59:59', + delay=0, + #set=0, + online=0, + walk=1) +''' +opObj11 = readUnitConfObj.addOperation(name='printInfo') + +procUnitConfObjA = controllerObj.addProcUnit(datatype='VoltageProc', inputId=readUnitConfObj.getId()) +#opObj11 = procUnitConfObjA.addOperation(name='CohInt', optype='other') +#opObj11.addParameter(name='n', value='10', format='int') + +#opObj10 = procUnitConfObjA.addOperation(name='selectChannels') +#opObj10.addParameter(name='channelList', value=[0]) +opObj11 = procUnitConfObjA.addOperation(name='PulsePairVoltage', optype='other') +opObj11.addParameter(name='n', value='625', format='int')#10 +opObj11.addParameter(name='removeDC', value=1, format='int') + +#opObj11 = procUnitConfObjA.addOperation(name='PulsepairPowerPlot', optype='other') +#opObj11 = procUnitConfObjA.addOperation(name='PulsepairSignalPlot', optype='other') + + +#opObj11 = procUnitConfObjA.addOperation(name='PulsepairVelocityPlot', optype='other') +#opObj11.addParameter(name='xmax', value=8) + +#opObj11 = procUnitConfObjA.addOperation(name='PulsepairSpecwidthPlot', optype='other') + +procUnitConfObjB= controllerObj.addProcUnit(datatype='ParametersProc',inputId=procUnitConfObjA.getId()) + + +opObj10 = procUnitConfObjB.addOperation(name='ParameterWriter') +opObj10.addParameter(name='path',value=figpath) +#opObj10.addParameter(name='mode',value=0) +opObj10.addParameter(name='blocksPerFile',value='100',format='int') +opObj10.addParameter(name='metadataList',value='utctimeInit,timeInterval',format='list') +opObj10.addParameter(name='dataList',value='dataPP_POW,dataPP_DOP,dataPP_SNR,dataPP_WIDTH')#,format='list' + +controllerObj.start() diff --git a/schainpy/scripts/wr_integrador.py b/schainpy/scripts/wr_integrador.py new file mode 100644 index 0000000..d719cb3 --- /dev/null +++ b/schainpy/scripts/wr_integrador.py @@ -0,0 +1,162 @@ +import os,numpy,h5py +from shutil import copyfile + +def isNumber(str): + try: + float(str) + return True + except: + return False + +def getfirstFilefromPath(path,meta,ext): + validFilelist = [] + fileList = os.listdir(path) + if len(fileList)<1: + return None + # meta 1234 567 8-18 BCDE + # H,D,PE YYYY DDD EPOC .ext + + for thisFile in fileList: + if meta =="PE": + try: + number= int(thisFile[len(meta)+7:len(meta)+17]) + except: + print("There is a file or folder with different format") + if meta == "D": + try: + number= int(thisFile[8:11]) + except: + print("There is a file or folder with different format") + + if not isNumber(str=number): + continue + if (os.path.splitext(thisFile)[-1].lower() != ext.lower()): + continue + validFilelist.sort() + validFilelist.append(thisFile) + if len(validFilelist)>0: + validFilelist = sorted(validFilelist,key=str.lower) + return validFilelist + return None + +def gettimeutcfromDirFilename(path,file): + dir_file= path+"/"+file + fp = h5py.File(dir_file,'r') + epoc = fp['Metadata'].get('utctimeInit')[()] + fp.close() + return epoc + +def getDatavaluefromDirFilename(path,file,value): + dir_file= path+"/"+file + fp = h5py.File(dir_file,'r') + array = fp['Data'].get(value)[()] + fp.close() + return array + + +#·········· Velocidad de Pedestal················· +w = input ("Ingresa velocidad de Pedestal: ") +w = 4 +w = float(w) +#·········· Resolucion minimo en grados··········· +alfa = input ("Ingresa resolucion minima en grados: ") +alfa = 1 +alfa = float(alfa) +#·········· IPP del Experimento ·················· +IPP = input ("Ingresa el IPP del experimento: ") +IPP = 0.0004 +IPP = float(IPP) +#·········· MODE ·················· +mode = input ("Ingresa el MODO del experimento T or F: ") +mode = "T" +mode = str(mode) + +#·········· Tiempo en generar la resolucion min··· +#············ MCU ·· var_ang = w * (var_tiempo)··· +var_tiempo = alfa/w +#·········· Tiempo Equivalente en perfiles········ +#·········· var_tiempo = IPP * ( num_perfiles )· +num_perfiles = int(var_tiempo/IPP) + +#··········DATA PEDESTAL·························· +dir_pedestal = "/home/alex/Downloads/pedestal" +#·········· DATA ADQ······························ +if mode=="T": + dir_adq = "/home/alex/Downloads/hdf5_testPP/d2020194" # Time domain +else: + dir_adq = "/home/alex/Downloads/hdf5_test/d2020194" # Frequency domain + +print( "Velocidad angular :", w) +print( "Resolucion minima en grados :", alfa) +print( "Numero de perfiles equivalente:", num_perfiles) +print( "Mode :", mode) + +#············ First File············· +list_pedestal = getfirstFilefromPath(path=dir_pedestal,meta="PE",ext=".hdf5") +list_adq = getfirstFilefromPath(path=dir_adq ,meta="D",ext=".hdf5") + +#············ utc time ·············· +utc_pedestal= gettimeutcfromDirFilename(path=dir_pedestal,file=list_pedestal[0]) +utc_adq = gettimeutcfromDirFilename(path=dir_adq ,file=list_adq[0]) + +print("utc_pedestal :",utc_pedestal) +print("utc_adq :",utc_adq) +#·············Relacion: utc_adq (+/-) var_tiempo*nro_file= utc_pedestal +time_Interval_p = 0.01 +n_perfiles_p = 100 +if utc_adq>utc_pedestal: + nro_file = int((int(utc_adq) - int(utc_pedestal))/(time_Interval_p*n_perfiles_p)) + ff_pedestal = list_pedestal[nro_file] + utc_pedestal = gettimeutcfromDirFilename(path=dir_pedestal,file=ff_pedestal) + nro_key_p = int((utc_adq-utc_pedestal)/time_Interval_p) + if utc_adq >utc_pedestal: + ff_pedestal = ff_pedestal + else: + nro_file = nro_file-1 + ff_pedestal = list_pedestal[nro_file] + angulo = getDatavaluefromDirFilename(path=dir_pedestal,file=ff_pedestal,value="azimuth") + nro_key_p = int((utc_adq-utc_pedestal)/time_Interval_p) + print("nro_file :",nro_file) + print("name_file :",ff_pedestal) + print("utc_pedestal_file :",utc_pedestal) + print("nro_key_p :",nro_key_p) + print("utc_pedestal_init :",utc_pedestal+nro_key_p*time_Interval_p) + print("angulo_array :",angulo[nro_key_p]) +#4+25+25+25+21 +#while True: +list_pedestal = getfirstFilefromPath(path=dir_pedestal,meta="PE",ext=".hdf5") +list_adq = getfirstFilefromPath(path=dir_adq ,meta="D",ext=".hdf5") + +nro_file = nro_file #10 +nro_key_perfil = nro_key_p +blocksPerFile = 100 +wr_path = "/home/alex/Downloads/hdf5_wr/" +# Lectura de archivos de adquisicion para adicion de azimuth +for thisFile in range(len(list_adq)): + print("thisFileAdq",thisFile) + angulo_adq = numpy.zeros(blocksPerFile) + tmp = 0 + for j in range(blocksPerFile): + iterador = nro_key_perfil + 25*(j-tmp) + if iterador < n_perfiles_p: + nro_file = nro_file + else: + nro_file = nro_file+1 + tmp = j + iterador = nro_key_perfil + ff_pedestal = list_pedestal[nro_file] + angulo = getDatavaluefromDirFilename(path=dir_pedestal,file=ff_pedestal,value="azimuth") + angulo_adq[j]= angulo[iterador] + copyfile(dir_adq+"/"+list_adq[thisFile],wr_path+list_adq[thisFile]) + fp = h5py.File(wr_path+list_adq[thisFile],'a') + grp = fp.create_group("Pedestal") + dset = grp.create_dataset("azimuth" , data=angulo_adq) + fp.close() + print("Angulo",angulo_adq) + print("Angulo",len(angulo_adq)) + nro_key_perfil=iterador + 25 + if nro_key_perfil< n_perfiles_p: + nro_file = nro_file + else: + nro_file = nro_file+1 + nro_key_perfil= nro_key_p diff --git a/schainpy/speedTools.bash b/schainpy/speedTools.bash deleted file mode 100644 index 70024c9..0000000 --- a/schainpy/speedTools.bash +++ /dev/null @@ -1,11 +0,0 @@ -#Use this script in order to find the slowest module or function in your python code. -#Reference: http://lukauskas.co.uk/articles/2014/02/12/how-to-make-python-faster-without-trying-that-much/ - -#!/bin/sh - -script = "testRawData.py" -output = "profile.pdf" - -python -m cProfile -o profile.pstats $script - -gprof2dot -f pstats profile.pstats | dot -Tpdf -o $output \ No newline at end of file diff --git a/schainpy/utils/paramsFinder.py b/schainpy/utils/paramsFinder.py deleted file mode 100644 index 0efaf1c..0000000 --- a/schainpy/utils/paramsFinder.py +++ /dev/null @@ -1,80 +0,0 @@ -import schainpy -from schainpy.model import Operation, ProcessingUnit -from pydoc import locate - -def clean_modules(module): - noEndsUnder = [x for x in module if not x.endswith('__')] - noStartUnder = [x for x in noEndsUnder if not x.startswith('__')] - noFullUpper = [x for x in noStartUnder if not x.isupper()] - return noFullUpper - -def check_module(possible, instance): - def check(x): - try: - instancia = locate('schainpy.model.{}'.format(x)) - return isinstance(instancia(), instance) - except Exception as e: - return False - clean = clean_modules(possible) - return [x for x in clean if check(x)] - - -def getProcs(): - module = dir(schainpy.model) - procs = check_module(module, ProcessingUnit) - try: - procs.remove('ProcessingUnit') - except Exception as e: - pass - return procs - -def getOperations(): - module = dir(schainpy.model) - noProcs = [x for x in module if not x.endswith('Proc')] - operations = check_module(noProcs, Operation) - try: - operations.remove('Operation') - except Exception as e: - pass - return operations - -def getArgs(op): - module = locate('schainpy.model.{}'.format(op)) - args = module().getAllowedArgs() - try: - args.remove('self') - except Exception as e: - pass - try: - args.remove('dataOut') - except Exception as e: - pass - return args - -def getAll(): - allModules = dir(schainpy.model) - modules = check_module(allModules, Operation) - modules.extend(check_module(allModules, ProcessingUnit)) - return modules - -def formatArgs(op): - args = getArgs(op) - - argsAsKey = ["\t'{}'".format(x) for x in args] - argsFormatted = ": 'string',\n".join(argsAsKey) - - print op - print "parameters = { \n" + argsFormatted + ": 'string',\n }" - print '\n' - - -if __name__ == "__main__": - getAll() - [formatArgs(x) for x in getAll()] - - ''' - parameters = { - 'id': , - 'wintitle': , - } - ''' \ No newline at end of file diff --git a/schainpy/utils/trash b/schainpy/utils/trash deleted file mode 100644 index 384299d..0000000 --- a/schainpy/utils/trash +++ /dev/null @@ -1 +0,0 @@ -You should install "digital_rf_hdf5" module if you want to read USRP data diff --git a/schainpy/zerorpc/MyClient01.py b/schainpy/zerorpc/MyClient01.py deleted file mode 100644 index 0babed8..0000000 --- a/schainpy/zerorpc/MyClient01.py +++ /dev/null @@ -1,147 +0,0 @@ -''' -Created on Jul 11, 2014 - -@author: roj-idl71 -''' -import time -from gevent import sleep - -import zerorpc -from schainpy.model import * -from schainpy.serializer.DataTranslate import serial2Obj, serial2Dict -# import schainpy.model.io.jroIO_usrp - -def createObjVolt(): - ''' - This function creates a processing object "VoltProc" with some operations. - such as: "CohInt", "Scope", etc - These class are found inside schainpy.model.proc and schainpy.model.graphics - ''' - procObj = VoltageProc() - - opObj = CohInt() - procObj.addOperation(opObj, 1) - - opObj = Scope() - procObj.addOperation(opObj, 2) - - return procObj - -def createObjSpec(): - ''' - This function creates a processing object "SpecProc" with some operation objects - such as: "IncohInt", "SpectraPlot", "RTIPlot", etc - These class are found inside schainpy.model.proc and schainpy.model.graphics - ''' - - procObj = SpectraProc() - - opObj = IncohInt() - procObj.addOperation(opObj, objId = 1) - - opObj = SpectraPlot() - procObj.addOperation(opObj, objId = 2) - - opObj = RTIPlot() - procObj.addOperation(opObj, objId = 3) - - opObj = SpectraPlot() - procObj.addOperation(opObj, objId = 4) - - opObj = RTIPlot() - procObj.addOperation(opObj, objId = 5) - - return procObj - -def processingSpec(procObj, dataInObj): - - procObj.setInput(dataInObj) - procObj.run(nFFTPoints = 16) - - procObj.call(opType = "external", - opId = 1, - n=1) - - procObj.call(opType = "external", - opId = 2, - id=191, - zmin=-100, - zmax=-40) - - procObj.call(opType = "external", - opId = 3, - id=192, - zmin=-100, - zmax=-40, - timerange=10*60) - -# procObj.call(opType = "self", -# opName = "selectChannels", -# channelList = [0,1]) -# -# procObj.call(opType = "self", -# opName = "selectHeights", -# minHei = 300, -# maxHei = 400) -# -# procObj.call(opType = "external", -# opId = 4, -# id=193, -# zmin=-100, -# zmax=-40) -# -# procObj.call(opType = "external", -# opId = 5, -# id=194, -# zmin=-100, -# zmax=-40, -# timerange=10*60) - -def printSpeed(deltaTime, mySerial): - - #################### - size = len(mySerial)/1024. - vel = 1.0*size / deltaTime - - print "Index [", replayerObj.getProfileIndex(), "]: ", - print "Total time %5.2f ms, Data size %5.2f KB, Speed %5.2f MB/s" %(deltaTime, size, vel) - #################### - -if __name__ == '__main__': - - procObj = createObjSpec() - - replayerObj = zerorpc.Client() - replayerObj.connect("tcp://127.0.0.1:4242") - - serializer = replayerObj.getSerializer() - - ini = time.time() - mySerialMetadata = replayerObj.getSerialMetaData() - deltaTime = (time.time() - ini)*1024 - - printSpeed(deltaTime, mySerialMetadata) - - myMetaDict = serial2Dict(mySerialMetadata, - serializer = serializer) -# print myMetaDict - while True: - ini = time.time() - mySerialData = replayerObj.getSerialData() - deltaTime = (time.time() - ini)*1024 - - if not mySerialData: - print "No more data" - break - -# myDataDict = SERIALIZER.loads(mySerialData) -# print myDataDict -# continue - - printSpeed(deltaTime, mySerialData) - - dataInObj = serial2Obj(mySerialData, - metadataDict=myMetaDict, - serializer = serializer) - processingSpec(procObj, dataInObj) - sleep(1e-1) \ No newline at end of file diff --git a/schainpy/zerorpc/MyClient02.py b/schainpy/zerorpc/MyClient02.py deleted file mode 100644 index dffb82f..0000000 --- a/schainpy/zerorpc/MyClient02.py +++ /dev/null @@ -1,13 +0,0 @@ -''' -Created on Jul 11, 2014 - -@author: roj-idl71 -''' - -import zerorpc - -if __name__ == '__main__': - c = zerorpc.Client() - c.connect("tcp://127.0.0.1:4242") - c.load("file2") # AAAHH! The previously loaded model gets overwritten here! - print c.getModelName() \ No newline at end of file diff --git a/schainpy/zerorpc/MyServer.py b/schainpy/zerorpc/MyServer.py deleted file mode 100644 index 5d8ad2e..0000000 --- a/schainpy/zerorpc/MyServer.py +++ /dev/null @@ -1,33 +0,0 @@ -''' -Created on Jul 11, 2014 - -@author: roj-idl71 -''' -# import sys -import datetime -import zerorpc - -from schainpy.model.io.jrodataIO import USRPReaderAPI -# from schainpy.serializer.DataTranslate import serial2Obj - -if __name__ == '__main__': - - replayerObj = USRPReaderAPI(serializer='msgpack') - - replayerObj.setup(path='/Volumes/DATA/haystack/passive_radar/', - startDate=datetime.date(2000,1,1), - endDate=datetime.date(2015,1,1), - startTime=datetime.time(0,0,0), - endTime=datetime.time(23,59,59), - online=1, - nSamples=500, - channelList = [0,1,2,3,4,5,6,7]) - - replayerObj.start() - - print "Initializing 'zerorpc' server" - s = zerorpc.Server(replayerObj) - s.bind("tcp://0.0.0.0:4242") - s.run() - - print "End" \ No newline at end of file diff --git a/schainpy/zerorpc/__init__.py b/schainpy/zerorpc/__init__.py deleted file mode 100644 index e69de29..0000000 --- a/schainpy/zerorpc/__init__.py +++ /dev/null diff --git a/schainpy/zerorpc/test/testServer.py b/schainpy/zerorpc/test/testServer.py deleted file mode 100644 index 0f81f86..0000000 --- a/schainpy/zerorpc/test/testServer.py +++ /dev/null @@ -1,177 +0,0 @@ -''' -Created on Jul 15, 2014 - -@author: roj-idl71 -''' - -import sys -import yaml -import numpy -import jsonpickle - -# import schainpy.serializer.DynamicSerializer as DynamicSerializer - - - -def isNotClassVar(myObj): - - return not hasattr(myObj,'__dict__') - -def isDictFormat(thisValue): - - if type(thisValue) != type({}): - return False - - if '__name__' not in thisValue.keys(): - return False - - return True - -def obj2Dict(myObj): - - myDict = {} - - myDict['__name__'] = myObj.__class__.__name__ - - for thisKey, thisValue in myObj.__dict__.items(): - - if isNotClassVar(thisValue): - myDict[thisKey] = thisValue - continue - - ## If this value is another class instance - myNewDict = obj2Dict(thisValue) - myDict[thisKey] = myNewDict - - return myDict - -def dict2Obj(myDict): - ''' - ''' - - if '__name__' not in myDict.keys(): - return None - - className = eval(myDict['__name__']) - - myObj = className() - - for thisKey, thisValue in myDict.items(): - - if thisKey == '__name__': - continue - - if not isDictFormat(thisValue): - setattr(myObj, thisKey, thisValue) - continue - - myNewObj = dict2Obj(thisValue) - setattr(myObj, thisKey, myNewObj) - - return myObj - -class myTestClass3(object): - - def __init__(self): - ''' - ''' - self.y1 = 'y1' - self.y2 = 'y2' - -class myTestClass2(object): - - def __init__(self): - ''' - ''' - self.x1 = 'x1' - self.x2 = 'x2' - self.otherObj = myTestClass3() - - -class myTestClass(object): - - flagNoData = True - value1 = 1 - value2 = 2 - myObj = None - - def __init__(self): - - ''' - ''' - self.flagNoData = True - self.value1 = 1 - self.value2 = 2 - self.myObj = myTestClass2() - - def get_dtype(self): - - ''' - ''' - return self.value1 - - def set_dtype(self, value): - - ''' - ''' - - self.value1 = value - - dtype = property(get_dtype, set_dtype) - -def myMsgPackTest(): - - import msgpack - import msgpack_numpy as m - import numpy as np - - x = np.random.rand(5) - x_enc = m.encode(x) - x_rec = m.decode(x_enc) - - print x_rec -# -# x_enc = msgpack.packb(x, default=m.encoder) -# x_rec = msgpack.unpackb(x_enc, object_hook=m.decoder) - -if __name__ == '__main__': - - myMsgPackTest() - - sys.exit() - - serializerObj = DynamicSerializer.DynamicSerializer('json') - serializerObj = jsonpickle - - myTestObj = myTestClass() - - myTestObj.flagNoData = False - myTestObj.value1 = [1+3.4j,4,'5',] - myTestObj.value2 = {'x2': numpy.complex(1,2),'x1': 'x1'} -# myTestObj.myObj.x2 = numpy.arange(15, dtype=numpy.complex) - - myDict = obj2Dict(myTestObj) - - myNewObj = dict2Obj(myDict) - -# print myDict -# print myTestObj.__dict__ -# print myNewObj.__dict__ - -# sys.exit() - print myDict - - newSerial = serializerObj.encode(myDict) -# print newSerial - - newDict = serializerObj.decode(newSerial) - print newDict - - myNewObj = dict2Obj(newDict) - - print - print - print 50*'###' - print myTestObj.__dict__ - print myNewObj.__dict__ - \ No newline at end of file diff --git a/schainpy/zerorpc/test/testServer_Voltage.py b/schainpy/zerorpc/test/testServer_Voltage.py deleted file mode 100644 index 4ce04bd..0000000 --- a/schainpy/zerorpc/test/testServer_Voltage.py +++ /dev/null @@ -1,46 +0,0 @@ -''' -Created on Jul 15, 2014 - -@author: roj-idl71 -''' - -import sys -import cPickle - -from schainpy.model.data.jrodata import Voltage -# from schainpy.model.io.jrodataIO import USRPReaderMP -from schainpy.serializer.DynamicSerializer import DynamicSerializer -from schainpy.serializer.DataTranslate import obj2Dict, dict2Obj - - -if __name__ == "__main__": - - serializerObj = DynamicSerializer('yaml') - - myTestObj = Voltage() - - myDict = obj2Dict(myTestObj) - - myNewObj = dict2Obj(myDict) - -# print myDict -# print myTestObj.__dict__ -# print myNewObj.__dict__ -# -# print -# print '#############################' -# print -# newValue = serializerObj.dumps(myDict) -# print newValue -# -# newValue = serializerObj.loads(newValue) -# print newValue - - - print '###########CPICKLE##################' - print myDict - newSerialized = cPickle.dumps(myDict, 2) -# print newValue - - newDict = cPickle.loads(newSerialized) - print newDict \ No newline at end of file diff --git a/setup.py b/setup.py index 3701f0b..0518d4d 100644 --- a/setup.py +++ b/setup.py @@ -2,6 +2,7 @@ Created on Jul 16, 2014 @author: Miguel Urco +@author: Juan C. Espinoza ''' import os @@ -17,54 +18,46 @@ class build_ext(_build_ext): import numpy self.include_dirs.append(numpy.get_include()) -setup(name = "schainpy", - version = __version__, - description = "Python tools to read, write and process Jicamarca data", - author = "Miguel Urco", - author_email = "miguel.urco@jro.igp.gob.pe", - url = "http://jro.igp.gob.pe", - packages = {'schainpy', - 'schainpy.model', - 'schainpy.model.data', - 'schainpy.model.graphics', - 'schainpy.model.io', - 'schainpy.model.proc', - 'schainpy.model.serializer', - 'schainpy.model.utils', - 'schainpy.utils', - 'schainpy.gui', - 'schainpy.gui.figures', - 'schainpy.gui.viewcontroller', - 'schainpy.gui.viewer', - 'schainpy.gui.viewer.windows', - 'schainpy.cli'}, - ext_package = 'schainpy', - package_data = {'': ['schain.conf.template'], - 'schainpy.gui.figures': ['*.png', '*.jpg'], - 'schainpy.files': ['*.oga'] - }, - include_package_data = False, - scripts = ['schainpy/gui/schainGUI'], - ext_modules = [ - Extension("cSchain", ["schainpy/model/proc/extensions.c"]) - ], - entry_points = { - 'console_scripts': [ - 'schain = schainpy.cli.cli:main', - ], - }, - cmdclass = {'build_ext': build_ext}, - setup_requires = ["numpy >= 1.11.2"], - install_requires = [ - "scipy >= 0.14.0", - "h5py >= 2.2.1", - "matplotlib >= 2.0.0", - "pyfits >= 3.4", - "paramiko >= 2.1.2", - "paho-mqtt >= 1.2", - "zmq", - "fuzzywuzzy", - "click", - "python-Levenshtein" - ], +setup( + name = "schainpy", + version = __version__, + description = "Python tools to read, write and process Jicamarca data", + author = "Miguel Urco, Juan C. Espinoza", + author_email = "juan.espinoza@jro.igp.gob.pe", + url = "http://jro-dev.igp.gob.pe/rhodecode/schain", + packages = { + 'schainpy', + 'schainpy.model', + 'schainpy.model.data', + 'schainpy.model.graphics', + 'schainpy.model.io', + 'schainpy.model.proc', + 'schainpy.model.utils', + 'schainpy.utils', + 'schainpy.gui', + 'schainpy.cli', + }, + package_data = {'': ['schain.conf.template'], + 'schainpy.files': ['*.oga'] + }, + include_package_data = False, + scripts = ['schainpy/gui/schainGUI'], + entry_points = { + 'console_scripts': [ + 'schain = schainpy.cli.cli:main', + ], + }, + cmdclass = {'build_ext': build_ext}, + ext_modules=[ + Extension("schainpy.model.data._noise", ["schainc/_noise.c"]), + ], + setup_requires = ["numpy"], + install_requires = [ + "scipy", + "h5py", + "matplotlib", + "pyzmq", + "fuzzywuzzy", + "click", + ], )