##// END OF EJS Templates
schain
RSS
merge with v2.3
jespinoza -
r1063:ecbe1f26c35b merge
parent child
Show More
Side by Side Unified
Context file:
r1063:ecbe1f26c35b Loading diff...:

The requested changes are too big and content was truncated. Show full diff

Show file before | Show file after | Hide comments
@@ -0,0 +1,109
1 ## CHANGELOG:
2
3 ### 2.3
4 * Added high order function `multiSchain` for multiprocessing scripts.
5 * Added two new Processing Units `PublishData` and `ReceiverData` for receiving and sending dataOut through multiple ways (tcp, ipc, inproc).
6 * Added a new graphics Processing Unit `PlotterReceiver`. It is decoupled from normal processing sequence with support for data generated by multiprocessing scripts.
7 * Added support for sending realtime graphic to web server.
8 * GUI command `schain` is now `schainGUI`.
9 * Added a CLI tool named `schain`.
10 * Scripts templates can be now generated with `schain generate`.
11 * Now it is possible to search Processing Units and Operations with `schain search [module]` to get the right name and its allowed parameters.
12 * `schain xml` to run xml scripts.
13 * Added suggestions when parameters are poorly written.
14 * `Controller.start()` now runs in a different process than the process calling it.
15 * Added `schainpy.utils.log` for log standarization.
16 * Running script on online mode no longer ignores date and hour. Issue #1109.
17 * Added support for receving voltage data directly from JARS (tcp, ipc).
18 * Updated README for MAC OS GUI installation.
19 * Setup now installs numpy.
20
21 ### 2.2.6
22 * Graphics generated by the GUI are now the same as generated by scripts. Issue #1074.
23 * Added support for C extensions.
24 * function `hildebrand_sehkon` optimized with a C wrapper.
25 * Numpy version updated.
26 * Migration to GIT.
27
28 ### 2.2.5:
29 * splitProfiles and combineProfiles modules were added to VoltageProc and Signal Chain GUI.
30 * nProfiles of USRP data (hdf5) is the number of profiles thera are in one second.
31 * jroPlotter works directly with data objects instead of dictionaries
32 * script "schain" was added to Signal Chain installer
33
34 ### 2.2.4.1:
35 * jroIO_usrp.py is update to read Sandra's data
36 * decimation in Spectra and RTI plots is always enabled.
37 * time* window option added to GUI
38
39 ### 2.2.4:
40 * jroproc_spectra_lags.py added to schainpy
41 * Bug fixed in schainGUI: ProcUnit was created with the same id in some cases.
42 * Bug fixed in jroHeaderIO: Header size validation.
43
44 ### 2.2.3.1:
45 * Filtering block by time has been added.
46 * Bug fixed plotting RTI, CoherenceMap and others using xmin and xmax parameters. The first day worked
47 properly but the next days did not.
48
49 ### 2.2.3:
50 * Bug fixed in GUI: Error getting(reading) Code value
51 * Bug fixed in GUI: Flip option always needs channelList field
52 * Bug fixed in jrodata: when one branch modified a value in "dataOut" (example: dataOut.code) this value
53 was modified for every branch (because this was a reference). It was modified in data.copy()
54 * Bug fixed in jroproc_voltage.profileSelector(): rangeList replaces to profileRangeList.
55
56 ### 2.2.2:
57 * VoltageProc: ProfileSelector, Reshape, Decoder with nTxs!=1 and getblock=True was tested
58 * Rawdata and testRawdata.py added to Signal Chain project
59
60 ### 2.2.1:
61 * Bugs fixed in GUI
62 * Views were improved in GUI
63 * Support to MST* ISR experiments
64 * Bug fixed getting noise using hyldebrant. (minimum number of points > 20%)
65 * handleError added to jroplotter.py
66
67 ### 2.2.0:
68 * GUI: use of external plotter
69 * Compatible with matplotlib 1.5.0
70
71 ### 2.1.5:
72 * serializer module added to Signal Chain
73 * jroplotter.py added to Signal Chain
74
75 ### 2.1.4.2:
76 * A new Plotter Class was added
77 * Project.start() does not accept filename as a parameter anymore
78
79 ### 2.1.4.1:
80 * Send notifications when an error different to ValueError is detected
81
82 ### 2.1.4:
83 * Sending error notifications to signal chain administrator
84 * Login to email server added
85
86 ### 2.1.3.3:
87 * Colored Button Icons were added to GUI
88
89 ### 2.1.3.2:
90 * GUI: user interaction enhanced
91 * controller_api.py: Safe access to ControllerThead
92
93 ### 2.1.3.1:
94 * GUI: every icon were resized
95 * jroproc_voltage.py: Print a message when "Read from code" option is selected and the code is not defined inside data file
96
97 ### 2.1.3:
98 * jroplot_heispectra.py: SpectraHeisScope was not showing the right channels
99 * jroproc_voltage.py: Bug fixed selecting profiles (self.nProfiles took a wrong value),
100 Bug fixed selecting heights by block (selecting profiles instead heights)
101 * jroproc_voltage.py: New feature added: decoding data by block using FFT.
102 * jroIO_heispectra.py: Bug fixed in FitsReader. Using local Fits instance instead schainpy.mode.data.jrodata.Fits.
103 * jroIO_heispectra.py: Channel index list does not exist.
104
105 ### 2.1.2:
106 * jroutils_ftp.py: Bug fixed, Any error sending file stopped the Server Thread
107 Server thread opens and closes remote server each time file list is sent
108 * jroplot_spectra.py: Noise path was not being created when noise data is saved.
109 * jroIO_base.py: startTime can be greater than endTime. Example: SpreadF [18:00 * 07:00] No newline at end of file
Show file before | Show file after | Hide comments
1 NO CONTENT: new file 100644
The requested commit or file is too big and content was truncated. Show full diff
Show file before | Show file after | Hide comments
1 NO CONTENT: new file 100644
The requested commit or file is too big and content was truncated. Show full diff
Show file before | Show file after | Hide comments
1 NO CONTENT: new file 100644
The requested commit or file is too big and content was truncated. Show full diff
Show file before | Show file after | Hide comments
1 NO CONTENT: new file 100644
The requested commit or file is too big and content was truncated. Show full diff
Show file before | Show file after | Hide comments
1 NO CONTENT: new file 100644
The requested commit or file is too big and content was truncated. Show full diff
Show file before | Show file after | Hide comments
1 NO CONTENT: new file 100644
The requested commit or file is too big and content was truncated. Show full diff
Show file before | Show file after | Hide comments
1 NO CONTENT: new file 100644
The requested commit or file is too big and content was truncated. Show full diff
Show file before | Show file after | Hide comments
@@ -1,113 +1,115
1 1 # Byte-compiled / optimized / DLL files
2 2 __pycache__/
3 3 *.py[cod]
4 4 *$py.class
5 5
6 6 # C extensions
7 7 *.so
8 8
9 9 # Distribution / packaging
10 10 .Python
11 11 env/
12 12 build/
13 13 develop-eggs/
14 14 dist/
15 15 downloads/
16 16 eggs/
17 17 .eggs/
18 18 lib/
19 19 lib64/
20 20 parts/
21 21 sdist/
22 22 var/
23 23 wheels/
24 24 *.egg-info/
25 25 .installed.cfg
26 26 *.egg
27 27
28 28 # PyInstaller
29 29 # Usually these files are written by a python script from a template
30 30 # before PyInstaller builds the exe, so as to inject date/other infos into it.
31 31 *.manifest
32 32 *.spec
33 33
34 34 # Installer logs
35 35 pip-log.txt
36 36 pip-delete-this-directory.txt
37 37
38 38 # Unit test / coverage reports
39 39 htmlcov/
40 40 .tox/
41 41 .coverage
42 42 .coverage.*
43 43 .cache
44 44 nosetests.xml
45 45 coverage.xml
46 46 *,cover
47 47 .hypothesis/
48 48
49 49 # Translations
50 50 *.mo
51 51 *.pot
52 52
53 53 # Django stuff:
54 54 *.log
55 55 local_settings.py
56 56
57 57 # Flask stuff:
58 58 instance/
59 59 .webassets-cache
60 60
61 61 # Scrapy stuff:
62 62 .scrapy
63 63
64 64 # Sphinx documentation
65 65 docs/_build/
66 66
67 67 # PyBuilder
68 68 target/
69 69
70 70 # Jupyter Notebook
71 71 .ipynb_checkpoints
72 72
73 73 # pyenv
74 74 .python-version
75 75
76 76 # celery beat schedule file
77 77 celerybeat-schedule
78 78
79 79 # SageMath parsed files
80 80 *.sage.py
81 81
82 82 # dotenv
83 83 .env
84 84
85 85 # virtualenv
86 86 .venv
87 87 venv/
88 88 ENV/
89 89
90 90 # Spyder project settings
91 91 .spyderproject
92 92 .spyproject
93 93
94 94 # Rope project settings
95 95 .ropeproject
96 96
97 97 # mkdocs documentation
98 98 /site
99 99
100 100 # eclipse
101 101 .project
102 102 .pydevproject
103 103
104 104 # vscode
105 105
106 106 .vscode
107 107
108 schainpy/scripts/
109 schaingui/node_modules/
108 110 .svn/
109 111 *.png
110 112 *.pyc
111 schainpy/scripts
112
113 schaingui/node_modules
113 *.xml
114 *.log
115 trash No newline at end of file
Show file before | Show file after | Hide comments
@@ -1,132 +1,147
1 1 # Signal Chain
2 2
3 3 ## Introduction
4 4
5 5 Signal Chain (SCh) is a radar data processing library developed using [Python](www.python.org) at JRO. SCh provides modules to read, write, process and plot data.
6 6
7 7 ## Installation
8 8
9 9 Install system dependencies, clone the latest version from [git](http://jro-dev.igp.gob.pe/rhodecode/schain/) and install it as a normal python package.
10 10
11 ### Linux based system
11 12 ```
12 13 $ sudo apt-get install python-pip python-dev gfortran libpng-dev freetype* libblas-dev liblapack-dev libatlas-base-dev python-qt4 python-tk libssl-dev libhdf5-dev
13 14 $ sudo pip install numpy
14 15 $ git clone http://jro-dev.igp.gob.pe/rhodecode/schain/
15 16 $ cd schain
16 17 $ sudo pip install ./
17 ```
18
19 **Its recommended to install schain in a virtual environment**
20 18
21 19 ```
20 **It is recommended to install schain in a virtual environment**
21 ```
22 22 $ sudo pip install virtualenv
23 23 $ virtualenv /path/to/virtual --system-site-packages
24 24 $ source /path/to/virtual/bin/activate
25 25 (virtual) $ cd schain
26 26 (virtual) $ pip install ./
27
27 28 ```
28 29
30 ### MAC Os
31 ```
32 $ brew install cartr/qt4/pyqt
33 $ git clone http://jro-dev.igp.gob.pe/rhodecode/schain/
34 $ cd schain
35 $ pip install ./
36 ```
37
38 if ```pip install ./``` does not work, install a proper python enviroment, and repeat the steps.
39 ```
40 $ brew install python
41 ```
42
43
29 44 ## First Script
30 45
31 46 Read Spectra data (.pdata) - remove dc - plot spectra & RTI
32 47
33 48 Import SCh and creating a project
34 49
35 50 ```python
36 51 #!/usr/bin/python
37 52
38 53 from schainpy.controller import Project
39 54
40 55 controller = Project()
41 56 controller.setup(id = '100',
42 57 name='test',
43 58 description='Basic experiment')
44 59
45 60
46 61 ```
47 62
48 63 Adding read unit and operations
49 64
50 65 ```python
51 66 read_unit = controller.addReadUnit(datatype='Spectra',
52 67 path='/path/to/pdata/',
53 68 startDate='2014/01/31',
54 69 endDate='2014/03/31',
55 70 startTime='00:00:00',
56 71 endTime='23:59:59',
57 72 online=0,
58 73 walk=0)
59 74
60 75 proc_unit = controller.addProcUnit(datatype='Spectra',
61 76 inputId=read_unit.getId())
62 77
63 78 op = proc_unit.addOperation(name='selectChannels')
64 79 op.addParameter(name='channelList', value='0,1', format='intlist')
65 80
66 81 op = proc_unit.addOperation(name='selectHeights')
67 82 op.addParameter(name='minHei', value='80', format='float')
68 83 op.addParameter(name='maxHei', value='200', format='float')
69 84
70 85 op = proc_unit.addOperation(name='removeDC')
71 86
72 87 ```
73 88
74 89 Plotting data & start project
75 90
76 91 ```python
77 92 op = proc_unit.addOperation(name='SpectraPlot', optype='other')
78 93 op.addParameter(name='id', value='1', format='int')
79 94 op.addParameter(name='wintitle', value='Spectra', format='str')
80 95
81 96 op = procUnitConfObj1.addOperation(name='RTIPlot', optype='other')
82 97 op.addParameter(name='id', value='2', format='int')
83 98 op.addParameter(name='wintitle', value='RTI', format='str')
84 99
85 100 controller.start()
86 101
87 102 ```
88 103
89 104 Full script
90 105
91 106
92 107 ```python
93 108 #!/usr/bin/python
94 109
95 110 from schainpy.controller import Project
96 111
97 112 controller = Project()
98 113 controller.setup(id = '100',
99 114 name='test',
100 115 description='Basic experiment')
101 116 read_unit = controller.addReadUnit(datatype='Spectra',
102 117 path='/path/to/pdata/',
103 118 startDate='2014/01/31',
104 119 endDate='2014/03/31',
105 120 startTime='00:00:00',
106 121 endTime='23:59:59',
107 122 online=0,
108 123 walk=0)
109 124
110 125 proc_unit = controller.addProcUnit(datatype='Spectra',
111 126 inputId=read_unit.getId())
112 127
113 128 op = proc_unit.addOperation(name='selectChannels')
114 129 op.addParameter(name='channelList', value='0,1', format='intlist')
115 130
116 131 op = proc_unit.addOperation(name='selectHeights')
117 132 op.addParameter(name='minHei', value='80', format='float')
118 133 op.addParameter(name='maxHei', value='200', format='float')
119 134
120 135 op = proc_unit.addOperation(name='removeDC')
121 136
122 137 op = proc_unit.addOperation(name='SpectraPlot', optype='other')
123 138 op.addParameter(name='id', value='6', format='int')
124 139 op.addParameter(name='wintitle', value='Spectra', format='str')
125 140
126 141 op = procUnitConfObj1.addOperation(name='RTIPlot', optype='other')
127 142 op.addParameter(name='id', value='2', format='int')
128 143 op.addParameter(name='wintitle', value='RTI', format='str')
129 144
130 145 controller.start()
131 146
132 147 ``` No newline at end of file
Show file before | Show file after | Hide comments
@@ -1,188 +1,157
1 1 import click
2 2 import schainpy
3 3 import subprocess
4 4 import os
5 5 import sys
6 6 import glob
7 7 save_stdout = sys.stdout
8 8 sys.stdout = open('trash', 'w')
9 9 from multiprocessing import cpu_count
10 10 from schaincli import templates
11 from schainpy import controller_api
11 from schainpy.controller import Project
12 12 from schainpy.model import Operation, ProcessingUnit
13 13 from schainpy.utils import log
14 14 from importlib import import_module
15 15 from pydoc import locate
16 16 from fuzzywuzzy import process
17 from schainpy.utils import paramsFinder
17 18 sys.stdout = save_stdout
18 19
19 20
20 21 def print_version(ctx, param, value):
21 22 if not value or ctx.resilient_parsing:
22 23 return
23 24 click.echo(schainpy.__version__)
24 25 ctx.exit()
25 26
26 27
27 28 cliLogger = log.makelogger('schain cli')
28 29 PREFIX = 'experiment'
29 30
30 31
31 32 @click.command()
32 33 @click.option('--version', '-v', is_flag=True, callback=print_version, help='SChain version', type=str)
33 34 @click.option('--xml', '-x', default=None, help='run an XML file', type=click.Path(exists=True, resolve_path=True))
34 35 @click.argument('command', default='run', required=True)
35 36 @click.argument('nextcommand', default=None, required=False, type=str)
36 37 def main(command, nextcommand, version, xml):
37 38 """COMMAND LINE INTERFACE FOR SIGNAL CHAIN - JICAMARCA RADIO OBSERVATORY \n
38 39 Available commands.\n
39 40 --xml: runs a schain XML generated file\n
40 41 run: runs any python script starting 'experiment_'\n
41 42 generate: generates a template schain script\n
42 43 search: return avilable operations, procs or arguments of the give operation/proc\n"""
43 44 if xml is not None:
44 45 runFromXML(xml)
45 46 elif command == 'generate':
46 47 generate()
47 48 elif command == 'test':
48 49 test()
49 50 elif command == 'run':
50 51 runschain(nextcommand)
51 52 elif command == 'search':
52 53 search(nextcommand)
53 54 else:
54 55 log.error('Command {} is not defined'.format(command))
55 56
56 57 def check_module(possible, instance):
57 58 def check(x):
58 59 try:
59 60 instancia = locate('schainpy.model.{}'.format(x))
60 61 return isinstance(instancia(), instance)
61 62 except Exception as e:
62 63 return False
63 64 clean = clean_modules(possible)
64 65 return [x for x in clean if check(x)]
65 66
66 67
67 68 def clean_modules(module):
68 69 noEndsUnder = [x for x in module if not x.endswith('__')]
69 70 noStartUnder = [x for x in noEndsUnder if not x.startswith('__')]
70 71 noFullUpper = [x for x in noStartUnder if not x.isupper()]
71 72 return noFullUpper
72 73
73 74
74 75 def search(nextcommand):
75 76 if nextcommand is None:
76 77 log.error('There is no Operation/ProcessingUnit to search')
77 78 elif nextcommand == 'procs':
78 module = dir(import_module('schainpy.model'))
79 procs = check_module(module, ProcessingUnit)
80 try:
81 procs.remove('ProcessingUnit')
82 except Exception as e:
83 pass
79 procs = paramsFinder.getProcs()
84 80 log.success('Current ProcessingUnits are:\n\033[1m{}\033[0m'.format('\n'.join(procs)))
85 81
86 82 elif nextcommand == 'operations':
87 module = dir(import_module('schainpy.model'))
88 noProcs = [x for x in module if not x.endswith('Proc')]
89 operations = check_module(noProcs, Operation)
90 try:
91 operations.remove('Operation')
92 except Exception as e:
93 pass
83 operations = paramsFinder.getOperations()
94 84 log.success('Current Operations are:\n\033[1m{}\033[0m'.format('\n'.join(operations)))
95 85 else:
96 86 try:
97 module = locate('schainpy.model.{}'.format(nextcommand))
98 args = module().getAllowedArgs()
87 args = paramsFinder.getArgs(nextcommand)
99 88 log.warning('Use this feature with caution. It may not return all the allowed arguments')
100 try:
101 args.remove('self')
102 except Exception as e:
103 pass
104 try:
105 args.remove('dataOut')
106 except Exception as e:
107 pass
108 89 if len(args) == 0:
109 90 log.success('{} has no arguments'.format(nextcommand))
110 91 else:
111 92 log.success('Showing arguments of {} are:\n\033[1m{}\033[0m'.format(nextcommand, '\n'.join(args)))
112 93 except Exception as e:
113 94 log.error('Module {} does not exists'.format(nextcommand))
114 allModules = dir(import_module('schainpy.model'))
115 module = check_module(allModules, Operation)
116 module.extend(check_module(allModules, ProcessingUnit))
117 similar = process.extractOne(nextcommand, module)[0]
118 log.success('Searching {} instead'.format(similar))
95 allModules = paramsFinder.getAll()
96 similar = process.extractOne(nextcommand, allModules)[0]
97 log.success('Showing {} instead'.format(similar))
119 98 search(similar)
120 99
121 100
122 101 def runschain(nextcommand):
123 102 if nextcommand is None:
124 103 currentfiles = glob.glob('./{}_*.py'.format(PREFIX))
125 104 numberfiles = len(currentfiles)
126 105 if numberfiles > 1:
127 106 log.error('There is more than one file to run')
128 107 elif numberfiles == 1:
129 108 subprocess.call(['python ' + currentfiles[0]], shell=True)
130 109 else:
131 110 log.error('There is no file to run')
132 111 else:
133 112 try:
134 113 subprocess.call(['python ' + nextcommand], shell=True)
135 114 except Exception as e:
136 115 log.error("I cannot run the file. Does it exists?")
137 116
138 117
139 118 def basicInputs():
140 119 inputs = {}
141 120 inputs['desc'] = click.prompt('Enter a description', default="A schain project", type=str)
142 121 inputs['name'] = click.prompt('Name of the project', default="project", type=str)
143 122 inputs['path'] = click.prompt('Data path', default=os.getcwd(), type=click.Path(exists=True, resolve_path=True))
144 123 inputs['startDate'] = click.prompt('Start date', default='1970/01/01', type=str)
145 124 inputs['endDate'] = click.prompt('End date', default='2017/12/31', type=str)
146 125 inputs['startHour'] = click.prompt('Start hour', default='00:00:00', type=str)
147 126 inputs['endHour'] = click.prompt('End hour', default='23:59:59', type=str)
148 127 inputs['figpath'] = inputs['path'] + '/figs'
149 128 return inputs
150 129
151 130
152 131 def generate():
153 132 inputs = basicInputs()
154 133 inputs['multiprocess'] = click.confirm('Is this a multiprocess script?')
155 134 if inputs['multiprocess']:
156 135 inputs['nProcess'] = click.prompt('How many process?', default=cpu_count(), type=int)
157 136 current = templates.multiprocess.format(**inputs)
158 137 else:
159 138 current = templates.basic.format(**inputs)
160 139 scriptname = '{}_{}.py'.format(PREFIX, inputs['name'])
161 140 script = open(scriptname, 'w')
162 141 try:
163 142 script.write(current)
164 143 log.success('Script {} generated'.format(scriptname))
165 144 except Exception as e:
166 145 log.error('I cannot create the file. Do you have writing permissions?')
167 146
168 147
169 148 def test():
170 149 log.warning('testing')
171 150
172 151
173 152 def runFromXML(filename):
174 controller = controller_api.ControllerThread()
153 controller = Project()
175 154 if not controller.readXml(filename):
176 155 return
177
178 plotterObj = controller.useExternalPlotter()
179
180 156 controller.start()
181 plotterObj.start()
182
183 cliLogger("Finishing all processes")
184
185 controller.join(5)
186
187 cliLogger("End of script")
188 157 return
Show file before | Show file after | Hide comments
@@ -1,7 +1,7
1 1 '''
2 2 Created on Feb 7, 2012
3 3
4 4 @author $Author$
5 5 @version $Id$
6 6 '''
7 __version__ = "2.3" No newline at end of file
7 __version__ = "2.3"
Show file before | Show file after | Hide comments
@@ -1,1319 +1,1297
1 1 '''
2 2 Created on September , 2012
3 3 @author:
4 4 '''
5 5
6 6 import sys
7 7 import ast
8 8 import datetime
9 9 import traceback
10 10 import math
11 11 import time
12 from multiprocessing import Process, Queue, cpu_count
13
14 import schainpy
15 import schainpy.admin
12 from multiprocessing import Process, cpu_count
16 13
17 14 from xml.etree.ElementTree import ElementTree, Element, SubElement, tostring
18 15 from xml.dom import minidom
19 16
17 import schainpy
18 import schainpy.admin
20 19 from schainpy.model import *
21 from time import sleep
22
23 def prettify(elem):
24 """Return a pretty-printed XML string for the Element.
25 """
26 rough_string = tostring(elem, 'utf-8')
27 reparsed = minidom.parseString(rough_string)
28 return reparsed.toprettyxml(indent=" ")
29
30 def multiSchain(child, nProcess=cpu_count(), startDate=None, endDate=None, by_day=False):
31 skip = 0
32 cursor = 0
33 nFiles = None
34 processes = []
35 dt1 = datetime.datetime.strptime(startDate, '%Y/%m/%d')
36 dt2 = datetime.datetime.strptime(endDate, '%Y/%m/%d')
20 from schainpy.utils import log
21
22 DTYPES = {
23 'Voltage': '.r',
24 'Spectra': '.pdata'
25 }
26
27 def MPProject(project, n=cpu_count()):
28 '''
29 Project wrapper to run schain in n processes
30 '''
31
32 rconf = project.getReadUnitObj()
33 op = rconf.getOperationObj('run')
34 dt1 = op.getParameterValue('startDate')
35 dt2 = op.getParameterValue('endDate')
37 36 days = (dt2 - dt1).days
38
37
39 38 for day in range(days+1):
40 39 skip = 0
41 40 cursor = 0
42 q = Queue()
43 41 processes = []
44 dt = (dt1 + datetime.timedelta(day)).strftime('%Y/%m/%d')
45 firstProcess = Process(target=child, args=(cursor, skip, q, dt))
46 firstProcess.start()
47 if by_day:
48 continue
49 nFiles = q.get()
50 if nFiles==0:
42 dt = dt1 + datetime.timedelta(day)
43 dt_str = dt.strftime('%Y/%m/%d')
44 reader = JRODataReader()
45 paths, files = reader.searchFilesOffLine(path=rconf.path,
46 startDate=dt,
47 endDate=dt,
48 ext=DTYPES[rconf.datatype])
49 nFiles = len(files)
50 if nFiles == 0:
51 51 continue
52 firstProcess.terminate()
53 skip = int(math.ceil(nFiles/nProcess))
54 while True:
55 processes.append(Process(target=child, args=(cursor, skip, q, dt)))
56 processes[cursor].start()
57 if nFiles < cursor*skip:
58 break
52 skip = int(math.ceil(nFiles/n))
53 while nFiles > cursor*skip:
54 rconf.update(startDate=dt_str, endDate=dt_str, cursor=cursor,
55 skip=skip)
56 p = project.clone()
57 p.start()
58 processes.append(p)
59 59 cursor += 1
60 60
61 61 def beforeExit(exctype, value, trace):
62 62 for process in processes:
63 63 process.terminate()
64 64 process.join()
65 65 print traceback.print_tb(trace)
66 66
67 67 sys.excepthook = beforeExit
68 68
69 69 for process in processes:
70 70 process.join()
71 71 process.terminate()
72 72
73 73 time.sleep(3)
74 74
75
76 75 class ParameterConf():
77 76
78 77 id = None
79 78 name = None
80 79 value = None
81 80 format = None
82 81
83 82 __formated_value = None
84 83
85 84 ELEMENTNAME = 'Parameter'
86 85
87 86 def __init__(self):
88 87
89 88 self.format = 'str'
90 89
91 90 def getElementName(self):
92 91
93 92 return self.ELEMENTNAME
94 93
95 94 def getValue(self):
96 95
97 96 value = self.value
98 97 format = self.format
99 98
100 99 if self.__formated_value != None:
101 100
102 101 return self.__formated_value
103 102
104 103 if format == 'obj':
105 104 return value
106 105
107 106 if format == 'str':
108 107 self.__formated_value = str(value)
109 108 return self.__formated_value
110 109
111 110 if value == '':
112 raise ValueError, "%s: This parameter value is empty" %self.name
111 raise ValueError, '%s: This parameter value is empty' %self.name
113 112
114 113 if format == 'list':
115 114 strList = value.split(',')
116 115
117 116 self.__formated_value = strList
118 117
119 118 return self.__formated_value
120 119
121 120 if format == 'intlist':
122 """
121 '''
123 122 Example:
124 123 value = (0,1,2)
125 """
124 '''
126 125
127 126 new_value = ast.literal_eval(value)
128 127
129 128 if type(new_value) not in (tuple, list):
130 129 new_value = [int(new_value)]
131 130
132 131 self.__formated_value = new_value
133 132
134 133 return self.__formated_value
135 134
136 135 if format == 'floatlist':
137 """
136 '''
138 137 Example:
139 138 value = (0.5, 1.4, 2.7)
140 """
139 '''
141 140
142 141 new_value = ast.literal_eval(value)
143 142
144 143 if type(new_value) not in (tuple, list):
145 144 new_value = [float(new_value)]
146 145
147 146 self.__formated_value = new_value
148 147
149 148 return self.__formated_value
150 149
151 150 if format == 'date':
152 151 strList = value.split('/')
153 152 intList = [int(x) for x in strList]
154 153 date = datetime.date(intList[0], intList[1], intList[2])
155 154
156 155 self.__formated_value = date
157 156
158 157 return self.__formated_value
159 158
160 159 if format == 'time':
161 160 strList = value.split(':')
162 161 intList = [int(x) for x in strList]
163 162 time = datetime.time(intList[0], intList[1], intList[2])
164 163
165 164 self.__formated_value = time
166 165
167 166 return self.__formated_value
168 167
169 168 if format == 'pairslist':
170 """
169 '''
171 170 Example:
172 171 value = (0,1),(1,2)
173 """
172 '''
174 173
175 174 new_value = ast.literal_eval(value)
176 175
177 176 if type(new_value) not in (tuple, list):
178 raise ValueError, "%s has to be a tuple or list of pairs" %value
177 raise ValueError, '%s has to be a tuple or list of pairs' %value
179 178
180 179 if type(new_value[0]) not in (tuple, list):
181 180 if len(new_value) != 2:
182 raise ValueError, "%s has to be a tuple or list of pairs" %value
181 raise ValueError, '%s has to be a tuple or list of pairs' %value
183 182 new_value = [new_value]
184 183
185 184 for thisPair in new_value:
186 185 if len(thisPair) != 2:
187 raise ValueError, "%s has to be a tuple or list of pairs" %value
186 raise ValueError, '%s has to be a tuple or list of pairs' %value
188 187
189 188 self.__formated_value = new_value
190 189
191 190 return self.__formated_value
192 191
193 192 if format == 'multilist':
194 """
193 '''
195 194 Example:
196 195 value = (0,1,2),(3,4,5)
197 """
196 '''
198 197 multiList = ast.literal_eval(value)
199 198
200 199 if type(multiList[0]) == int:
201 multiList = ast.literal_eval("(" + value + ")")
200 multiList = ast.literal_eval('(' + value + ')')
202 201
203 202 self.__formated_value = multiList
204 203
205 204 return self.__formated_value
206 205
207 206 if format == 'bool':
208 207 value = int(value)
209 208
210 209 if format == 'int':
211 210 value = float(value)
212 211
213 212 format_func = eval(format)
214 213
215 214 self.__formated_value = format_func(value)
216 215
217 216 return self.__formated_value
218 217
219 218 def updateId(self, new_id):
220 219
221 220 self.id = str(new_id)
222 221
223 222 def setup(self, id, name, value, format='str'):
224 223 self.id = str(id)
225 224 self.name = name
226 225 if format == 'obj':
227 226 self.value = value
228 227 else:
229 228 self.value = str(value)
230 229 self.format = str.lower(format)
231 230
232 231 self.getValue()
233 232
234 233 return 1
235 234
236 235 def update(self, name, value, format='str'):
237 236
238 237 self.name = name
239 238 self.value = str(value)
240 239 self.format = format
241 240
242 241 def makeXml(self, opElement):
243 242 if self.name not in ('queue',):
244 243 parmElement = SubElement(opElement, self.ELEMENTNAME)
245 244 parmElement.set('id', str(self.id))
246 245 parmElement.set('name', self.name)
247 246 parmElement.set('value', self.value)
248 247 parmElement.set('format', self.format)
249 248
250 249 def readXml(self, parmElement):
251 250
252 251 self.id = parmElement.get('id')
253 252 self.name = parmElement.get('name')
254 253 self.value = parmElement.get('value')
255 254 self.format = str.lower(parmElement.get('format'))
256 255
257 256 #Compatible with old signal chain version
258 257 if self.format == 'int' and self.name == 'idfigure':
259 258 self.name = 'id'
260 259
261 260 def printattr(self):
262 261
263 print "Parameter[%s]: name = %s, value = %s, format = %s" %(self.id, self.name, self.value, self.format)
262 print 'Parameter[%s]: name = %s, value = %s, format = %s' %(self.id, self.name, self.value, self.format)
264 263
265 264 class OperationConf():
266 265
267 266 id = None
268 267 name = None
269 268 priority = None
270 269 type = None
271 270
272 271 parmConfObjList = []
273 272
274 273 ELEMENTNAME = 'Operation'
275 274
276 275 def __init__(self):
277 276
278 277 self.id = '0'
279 278 self.name = None
280 279 self.priority = None
281 280 self.type = 'self'
282 281
283 282
284 283 def __getNewId(self):
285 284
286 285 return int(self.id)*10 + len(self.parmConfObjList) + 1
287 286
288 287 def updateId(self, new_id):
289 288
290 289 self.id = str(new_id)
291 290
292 291 n = 1
293 292 for parmObj in self.parmConfObjList:
294 293
295 294 idParm = str(int(new_id)*10 + n)
296 295 parmObj.updateId(idParm)
297 296
298 297 n += 1
299 298
300 299 def getElementName(self):
301 300
302 301 return self.ELEMENTNAME
303 302
304 303 def getParameterObjList(self):
305 304
306 305 return self.parmConfObjList
307 306
308 307 def getParameterObj(self, parameterName):
309 308
310 309 for parmConfObj in self.parmConfObjList:
311 310
312 311 if parmConfObj.name != parameterName:
313 312 continue
314 313
315 314 return parmConfObj
316 315
317 316 return None
318 317
319 318 def getParameterObjfromValue(self, parameterValue):
320 319
321 320 for parmConfObj in self.parmConfObjList:
322 321
323 322 if parmConfObj.getValue() != parameterValue:
324 323 continue
325 324
326 325 return parmConfObj.getValue()
327 326
328 327 return None
329 328
330 329 def getParameterValue(self, parameterName):
331 330
332 331 parameterObj = self.getParameterObj(parameterName)
333 332
334 333 # if not parameterObj:
335 334 # return None
336 335
337 336 value = parameterObj.getValue()
338 337
339 338 return value
340 339
341 340
342 341 def getKwargs(self):
343 342
344 343 kwargs = {}
345 344
346 345 for parmConfObj in self.parmConfObjList:
347 346 if self.name == 'run' and parmConfObj.name == 'datatype':
348 347 continue
349 348
350 349 kwargs[parmConfObj.name] = parmConfObj.getValue()
351 350
352 351 return kwargs
353 352
354 353 def setup(self, id, name, priority, type):
355 354
356 355 self.id = str(id)
357 356 self.name = name
358 357 self.type = type
359 358 self.priority = priority
360 359
361 360 self.parmConfObjList = []
362 361
363 362 def removeParameters(self):
364 363
365 364 for obj in self.parmConfObjList:
366 365 del obj
367 366
368 367 self.parmConfObjList = []
369 368
370 369 def addParameter(self, name, value, format='str'):
371
370
371 if value is None:
372 return None
372 373 id = self.__getNewId()
373 374
374 375 parmConfObj = ParameterConf()
375 376 if not parmConfObj.setup(id, name, value, format):
376 377 return None
377 378
378 379 self.parmConfObjList.append(parmConfObj)
379 380
380 381 return parmConfObj
381 382
382 383 def changeParameter(self, name, value, format='str'):
383 384
384 385 parmConfObj = self.getParameterObj(name)
385 386 parmConfObj.update(name, value, format)
386 387
387 388 return parmConfObj
388 389
389 390 def makeXml(self, procUnitElement):
390 391
391 392 opElement = SubElement(procUnitElement, self.ELEMENTNAME)
392 393 opElement.set('id', str(self.id))
393 394 opElement.set('name', self.name)
394 395 opElement.set('type', self.type)
395 396 opElement.set('priority', str(self.priority))
396 397
397 398 for parmConfObj in self.parmConfObjList:
398 399 parmConfObj.makeXml(opElement)
399 400
400 401 def readXml(self, opElement):
401 402
402 403 self.id = opElement.get('id')
403 404 self.name = opElement.get('name')
404 405 self.type = opElement.get('type')
405 406 self.priority = opElement.get('priority')
406 407
407 408 #Compatible with old signal chain version
408 409 #Use of 'run' method instead 'init'
409 410 if self.type == 'self' and self.name == 'init':
410 411 self.name = 'run'
411 412
412 413 self.parmConfObjList = []
413 414
414 415 parmElementList = opElement.iter(ParameterConf().getElementName())
415 416
416 417 for parmElement in parmElementList:
417 418 parmConfObj = ParameterConf()
418 419 parmConfObj.readXml(parmElement)
419 420
420 421 #Compatible with old signal chain version
421 422 #If an 'plot' OPERATION is found, changes name operation by the value of its type PARAMETER
422 423 if self.type != 'self' and self.name == 'Plot':
423 424 if parmConfObj.format == 'str' and parmConfObj.name == 'type':
424 425 self.name = parmConfObj.value
425 426 continue
426 427
427 428 self.parmConfObjList.append(parmConfObj)
428 429
429 430 def printattr(self):
430 431
431 print "%s[%s]: name = %s, type = %s, priority = %s" %(self.ELEMENTNAME,
432 print '%s[%s]: name = %s, type = %s, priority = %s' %(self.ELEMENTNAME,
432 433 self.id,
433 434 self.name,
434 435 self.type,
435 436 self.priority)
436 437
437 438 for parmConfObj in self.parmConfObjList:
438 439 parmConfObj.printattr()
439 440
440 441 def createObject(self, plotter_queue=None):
441 442
442 443
443 444 if self.type == 'self':
444 raise ValueError, "This operation type cannot be created"
445 raise ValueError, 'This operation type cannot be created'
445 446
446 if self.type == 'plotter':
447 #Plotter(plotter_name)
447 if self.type == 'plotter':
448 448 if not plotter_queue:
449 raise ValueError, "plotter_queue is not defined. Use:\nmyProject = Project()\nmyProject.setPlotterQueue(plotter_queue)"
449 raise ValueError, 'plotter_queue is not defined. Use:\nmyProject = Project()\nmyProject.setPlotterQueue(plotter_queue)'
450 450
451 451 opObj = Plotter(self.name, plotter_queue)
452 452
453 453 if self.type == 'external' or self.type == 'other':
454 454
455 455 className = eval(self.name)
456 456 kwargs = self.getKwargs()
457 457
458 458 opObj = className(**kwargs)
459 459
460 460 return opObj
461 461
462 462
463 463 class ProcUnitConf():
464 464
465 465 id = None
466 466 name = None
467 467 datatype = None
468 468 inputId = None
469 469 parentId = None
470 470
471 471 opConfObjList = []
472 472
473 473 procUnitObj = None
474 474 opObjList = []
475 475
476 476 ELEMENTNAME = 'ProcUnit'
477 477
478 478 def __init__(self):
479 479
480 480 self.id = None
481 481 self.datatype = None
482 482 self.name = None
483 483 self.inputId = None
484 484
485 485 self.opConfObjList = []
486 486
487 487 self.procUnitObj = None
488 488 self.opObjDict = {}
489 489
490 490 def __getPriority(self):
491 491
492 492 return len(self.opConfObjList)+1
493 493
494 494 def __getNewId(self):
495 495
496 496 return int(self.id)*10 + len(self.opConfObjList) + 1
497 497
498 498 def getElementName(self):
499 499
500 500 return self.ELEMENTNAME
501 501
502 502 def getId(self):
503 503
504 504 return self.id
505 505
506 506 def updateId(self, new_id, parentId=parentId):
507 507
508 508
509 509 new_id = int(parentId)*10 + (int(self.id) % 10)
510 510 new_inputId = int(parentId)*10 + (int(self.inputId) % 10)
511 511
512 512 #If this proc unit has not inputs
513 513 if self.inputId == '0':
514 514 new_inputId = 0
515 515
516 516 n = 1
517 517 for opConfObj in self.opConfObjList:
518 518
519 519 idOp = str(int(new_id)*10 + n)
520 520 opConfObj.updateId(idOp)
521 521
522 522 n += 1
523 523
524 524 self.parentId = str(parentId)
525 525 self.id = str(new_id)
526 526 self.inputId = str(new_inputId)
527 527
528 528
529 529 def getInputId(self):
530 530
531 531 return self.inputId
532 532
533 533 def getOperationObjList(self):
534 534
535 535 return self.opConfObjList
536 536
537 537 def getOperationObj(self, name=None):
538 538
539 539 for opConfObj in self.opConfObjList:
540 540
541 541 if opConfObj.name != name:
542 542 continue
543 543
544 544 return opConfObj
545 545
546 546 return None
547 547
548 548 def getOpObjfromParamValue(self, value=None):
549 549
550 550 for opConfObj in self.opConfObjList:
551 551 if opConfObj.getParameterObjfromValue(parameterValue=value) != value:
552 552 continue
553 553 return opConfObj
554 554 return None
555 555
556 556 def getProcUnitObj(self):
557 557
558 558 return self.procUnitObj
559 559
560 560 def setup(self, id, name, datatype, inputId, parentId=None):
561 561
562 562 #Compatible with old signal chain version
563 563 if datatype==None and name==None:
564 raise ValueError, "datatype or name should be defined"
564 raise ValueError, 'datatype or name should be defined'
565 565
566 566 if name==None:
567 567 if 'Proc' in datatype:
568 568 name = datatype
569 569 else:
570 570 name = '%sProc' %(datatype)
571 571
572 572 if datatype==None:
573 573 datatype = name.replace('Proc','')
574 574
575 575 self.id = str(id)
576 576 self.name = name
577 577 self.datatype = datatype
578 578 self.inputId = inputId
579 579 self.parentId = parentId
580 580
581 581 self.opConfObjList = []
582 582
583 583 self.addOperation(name='run', optype='self')
584 584
585 585 def removeOperations(self):
586 586
587 587 for obj in self.opConfObjList:
588 588 del obj
589 589
590 590 self.opConfObjList = []
591 591 self.addOperation(name='run')
592 592
593 593 def addParameter(self, **kwargs):
594 594 '''
595 Add parameters to "run" operation
595 Add parameters to 'run' operation
596 596 '''
597 597 opObj = self.opConfObjList[0]
598 598
599 599 opObj.addParameter(**kwargs)
600 600
601 601 return opObj
602 602
603 603 def addOperation(self, name, optype='self'):
604 604
605 605 id = self.__getNewId()
606 606 priority = self.__getPriority()
607 607
608 608 opConfObj = OperationConf()
609 609 opConfObj.setup(id, name=name, priority=priority, type=optype)
610 610
611 611 self.opConfObjList.append(opConfObj)
612 612
613 613 return opConfObj
614 614
615 615 def makeXml(self, projectElement):
616 616
617 617 procUnitElement = SubElement(projectElement, self.ELEMENTNAME)
618 618 procUnitElement.set('id', str(self.id))
619 619 procUnitElement.set('name', self.name)
620 620 procUnitElement.set('datatype', self.datatype)
621 621 procUnitElement.set('inputId', str(self.inputId))
622 622
623 623 for opConfObj in self.opConfObjList:
624 624 opConfObj.makeXml(procUnitElement)
625 625
626 626 def readXml(self, upElement):
627 627
628 628 self.id = upElement.get('id')
629 629 self.name = upElement.get('name')
630 630 self.datatype = upElement.get('datatype')
631 631 self.inputId = upElement.get('inputId')
632 632
633 if self.ELEMENTNAME == "ReadUnit":
634 self.datatype = self.datatype.replace("Reader", "")
633 if self.ELEMENTNAME == 'ReadUnit':
634 self.datatype = self.datatype.replace('Reader', '')
635 635
636 if self.ELEMENTNAME == "ProcUnit":
637 self.datatype = self.datatype.replace("Proc", "")
636 if self.ELEMENTNAME == 'ProcUnit':
637 self.datatype = self.datatype.replace('Proc', '')
638 638
639 639 if self.inputId == 'None':
640 640 self.inputId = '0'
641 641
642 642 self.opConfObjList = []
643 643
644 644 opElementList = upElement.iter(OperationConf().getElementName())
645 645
646 646 for opElement in opElementList:
647 647 opConfObj = OperationConf()
648 648 opConfObj.readXml(opElement)
649 649 self.opConfObjList.append(opConfObj)
650 650
651 651 def printattr(self):
652 652
653 print "%s[%s]: name = %s, datatype = %s, inputId = %s" %(self.ELEMENTNAME,
653 print '%s[%s]: name = %s, datatype = %s, inputId = %s' %(self.ELEMENTNAME,
654 654 self.id,
655 655 self.name,
656 656 self.datatype,
657 657 self.inputId)
658 658
659 659 for opConfObj in self.opConfObjList:
660 660 opConfObj.printattr()
661 661
662 662
663 663 def getKwargs(self):
664 664
665 665 opObj = self.opConfObjList[0]
666 666 kwargs = opObj.getKwargs()
667 667
668 668 return kwargs
669 669
670 670 def createObjects(self, plotter_queue=None):
671 671
672 672 className = eval(self.name)
673 673 kwargs = self.getKwargs()
674 674 procUnitObj = className(**kwargs)
675 675
676 676 for opConfObj in self.opConfObjList:
677 677
678 678 if opConfObj.type=='self' and self.name=='run':
679 679 continue
680 680 elif opConfObj.type=='self':
681 681 procUnitObj.addOperationKwargs(opConfObj.id, **opConfObj.getKwargs())
682 682 continue
683 683
684 684 opObj = opConfObj.createObject(plotter_queue)
685 685
686 686 self.opObjDict[opConfObj.id] = opObj
687 687
688 688 procUnitObj.addOperation(opObj, opConfObj.id)
689 689
690 690 self.procUnitObj = procUnitObj
691 691
692 692 return procUnitObj
693 693
694 694 def run(self):
695 695
696 696 is_ok = False
697 697
698 698 for opConfObj in self.opConfObjList:
699 699
700 700 kwargs = {}
701 701 for parmConfObj in opConfObj.getParameterObjList():
702 702 if opConfObj.name == 'run' and parmConfObj.name == 'datatype':
703 703 continue
704 704
705 705 kwargs[parmConfObj.name] = parmConfObj.getValue()
706 706
707 707 sts = self.procUnitObj.call(opType = opConfObj.type,
708 708 opName = opConfObj.name,
709 709 opId = opConfObj.id)
710 710
711 711 is_ok = is_ok or sts
712 712
713 713 return is_ok
714 714
715 715 def close(self):
716 716
717 717 for opConfObj in self.opConfObjList:
718 718 if opConfObj.type == 'self':
719 719 continue
720 720
721 721 opObj = self.procUnitObj.getOperationObj(opConfObj.id)
722 722 opObj.close()
723 723
724 724 self.procUnitObj.close()
725 725
726 726 return
727 727
728 728 class ReadUnitConf(ProcUnitConf):
729 729
730 730 path = None
731 731 startDate = None
732 732 endDate = None
733 733 startTime = None
734 734 endTime = None
735 735
736 736 ELEMENTNAME = 'ReadUnit'
737 737
738 738 def __init__(self):
739 739
740 740 self.id = None
741 741 self.datatype = None
742 742 self.name = None
743 743 self.inputId = None
744 744
745 745 self.parentId = None
746 746
747 747 self.opConfObjList = []
748 748 self.opObjList = []
749 749
750 750 def getElementName(self):
751 751
752 752 return self.ELEMENTNAME
753 753
754 def setup(self, id, name, datatype, path='', startDate="", endDate="", startTime="",
755 endTime="", parentId=None, queue=None, server=None, **kwargs):
754 def setup(self, id, name, datatype, path='', startDate='', endDate='',
755 startTime='', endTime='', parentId=None, server=None, **kwargs):
756 756
757 757 #Compatible with old signal chain version
758 758 if datatype==None and name==None:
759 raise ValueError, "datatype or name should be defined"
759 raise ValueError, 'datatype or name should be defined'
760 760
761 761 if name==None:
762 762 if 'Reader' in datatype:
763 763 name = datatype
764 764 else:
765 765 name = '%sReader' %(datatype)
766 766 if datatype==None:
767 767 datatype = name.replace('Reader','')
768 768
769 769 self.id = id
770 770 self.name = name
771 771 self.datatype = datatype
772 772 if path != '':
773 773 self.path = os.path.abspath(path)
774 774 self.startDate = startDate
775 775 self.endDate = endDate
776 776 self.startTime = startTime
777 777 self.endTime = endTime
778
779 778 self.inputId = '0'
780 779 self.parentId = parentId
781 self.queue = queue
782 780 self.server = server
783 781 self.addRunOperation(**kwargs)
784 782
785 def update(self, datatype, path, startDate, endDate, startTime, endTime, parentId=None, name=None, **kwargs):
783 def update(self, **kwargs):
786 784
787 #Compatible with old signal chain version
788 if datatype==None and name==None:
789 raise ValueError, "datatype or name should be defined"
790
791 if name==None:
785 if 'datatype' in kwargs:
786 datatype = kwargs.pop('datatype')
792 787 if 'Reader' in datatype:
793 name = datatype
788 self.name = datatype
794 789 else:
795 name = '%sReader' %(datatype)
796
797 if datatype==None:
798 datatype = name.replace('Reader','')
799
800 self.datatype = datatype
801 self.name = name
802 self.path = path
803 self.startDate = startDate
804 self.endDate = endDate
805 self.startTime = startTime
806 self.endTime = endTime
790 self.name = '%sReader' %(datatype)
791 self.datatype = self.name.replace('Reader', '')
807 792
793 attrs = ('path', 'startDate', 'endDate', 'startTime', 'endTime', 'parentId')
794
795 for attr in attrs:
796 if attr in kwargs:
797 setattr(self, attr, kwargs.pop(attr))
798
808 799 self.inputId = '0'
809 self.parentId = parentId
810
811 800 self.updateRunOperation(**kwargs)
812 801
813 802 def removeOperations(self):
814 803
815 804 for obj in self.opConfObjList:
816 805 del obj
817 806
818 807 self.opConfObjList = []
819 808
820 809 def addRunOperation(self, **kwargs):
821 810
822 811 opObj = self.addOperation(name = 'run', optype = 'self')
823 812
824 813 if self.server is None:
825 opObj.addParameter(name='datatype' , value=self.datatype, format='str')
826 opObj.addParameter(name='path' , value=self.path, format='str')
827 opObj.addParameter(name='startDate' , value=self.startDate, format='date')
828 opObj.addParameter(name='endDate' , value=self.endDate, format='date')
829 opObj.addParameter(name='startTime' , value=self.startTime, format='time')
830 opObj.addParameter(name='endTime' , value=self.endTime, format='time')
831 opObj.addParameter(name='queue' , value=self.queue, format='obj')
814 opObj.addParameter(name='datatype', value=self.datatype, format='str')
815 opObj.addParameter(name='path', value=self.path, format='str')
816 opObj.addParameter(name='startDate', value=self.startDate, format='date')
817 opObj.addParameter(name='endDate', value=self.endDate, format='date')
818 opObj.addParameter(name='startTime', value=self.startTime, format='time')
819 opObj.addParameter(name='endTime', value=self.endTime, format='time')
820
832 821 for key, value in kwargs.items():
833 822 opObj.addParameter(name=key, value=value, format=type(value).__name__)
834 823 else:
835 824 opObj.addParameter(name='server' , value=self.server, format='str')
836 825
837 826
838 827 return opObj
839 828
840 829 def updateRunOperation(self, **kwargs):
841 830
842 opObj = self.getOperationObj(name = 'run')
831 opObj = self.getOperationObj(name='run')
843 832 opObj.removeParameters()
844 833
845 opObj.addParameter(name='datatype' , value=self.datatype, format='str')
846 opObj.addParameter(name='path' , value=self.path, format='str')
847 opObj.addParameter(name='startDate' , value=self.startDate, format='date')
848 opObj.addParameter(name='endDate' , value=self.endDate, format='date')
849 opObj.addParameter(name='startTime' , value=self.startTime, format='time')
850 opObj.addParameter(name='endTime' , value=self.endTime, format='time')
851
834 opObj.addParameter(name='datatype', value=self.datatype, format='str')
835 opObj.addParameter(name='path', value=self.path, format='str')
836 opObj.addParameter(name='startDate', value=self.startDate, format='date')
837 opObj.addParameter(name='endDate', value=self.endDate, format='date')
838 opObj.addParameter(name='startTime', value=self.startTime, format='time')
839 opObj.addParameter(name='endTime', value=self.endTime, format='time')
840
852 841 for key, value in kwargs.items():
853 842 opObj.addParameter(name=key, value=value, format=type(value).__name__)
854 843
855 844 return opObj
856 845
857 # def makeXml(self, projectElement):
858 #
859 # procUnitElement = SubElement(projectElement, self.ELEMENTNAME)
860 # procUnitElement.set('id', str(self.id))
861 # procUnitElement.set('name', self.name)
862 # procUnitElement.set('datatype', self.datatype)
863 # procUnitElement.set('inputId', str(self.inputId))
864 #
865 # for opConfObj in self.opConfObjList:
866 # opConfObj.makeXml(procUnitElement)
867
868 846 def readXml(self, upElement):
869 847
870 848 self.id = upElement.get('id')
871 849 self.name = upElement.get('name')
872 850 self.datatype = upElement.get('datatype')
873 851 self.inputId = upElement.get('inputId')
874 852
875 if self.ELEMENTNAME == "ReadUnit":
876 self.datatype = self.datatype.replace("Reader", "")
853 if self.ELEMENTNAME == 'ReadUnit':
854 self.datatype = self.datatype.replace('Reader', '')
877 855
878 856 if self.inputId == 'None':
879 857 self.inputId = '0'
880 858
881 859 self.opConfObjList = []
882 860
883 861 opElementList = upElement.iter(OperationConf().getElementName())
884 862
885 863 for opElement in opElementList:
886 864 opConfObj = OperationConf()
887 865 opConfObj.readXml(opElement)
888 866 self.opConfObjList.append(opConfObj)
889 867
890 868 if opConfObj.name == 'run':
891 869 self.path = opConfObj.getParameterValue('path')
892 870 self.startDate = opConfObj.getParameterValue('startDate')
893 871 self.endDate = opConfObj.getParameterValue('endDate')
894 872 self.startTime = opConfObj.getParameterValue('startTime')
895 873 self.endTime = opConfObj.getParameterValue('endTime')
896 874
897 class Project():
875 class Project(Process):
898 876
899 877 id = None
900 name = None
878 # name = None
901 879 description = None
902 880 filename = None
903 881
904 882 procUnitConfObjDict = None
905 883
906 884 ELEMENTNAME = 'Project'
907 885
908 886 plotterQueue = None
909 887
910 888 def __init__(self, plotter_queue=None):
911 889
890 Process.__init__(self)
912 891 self.id = None
913 self.name = None
892 # self.name = None
914 893 self.description = None
915 894
916 895 self.plotterQueue = plotter_queue
917 896
918 897 self.procUnitConfObjDict = {}
919 898
920 899 def __getNewId(self):
921 900
922 901 idList = self.procUnitConfObjDict.keys()
923 902
924 903 id = int(self.id)*10
925 904
926 905 while True:
927 906 id += 1
928 907
929 908 if str(id) in idList:
930 909 continue
931 910
932 911 break
933 912
934 913 return str(id)
935 914
936 915 def getElementName(self):
937 916
938 917 return self.ELEMENTNAME
939 918
940 919 def getId(self):
941 920
942 921 return self.id
943 922
944 923 def updateId(self, new_id):
945 924
946 925 self.id = str(new_id)
947 926
948 927 keyList = self.procUnitConfObjDict.keys()
949 928 keyList.sort()
950 929
951 930 n = 1
952 931 newProcUnitConfObjDict = {}
953 932
954 933 for procKey in keyList:
955 934
956 935 procUnitConfObj = self.procUnitConfObjDict[procKey]
957 936 idProcUnit = str(int(self.id)*10 + n)
958 937 procUnitConfObj.updateId(idProcUnit, parentId = self.id)
959 938
960 939 newProcUnitConfObjDict[idProcUnit] = procUnitConfObj
961 940 n += 1
962 941
963 942 self.procUnitConfObjDict = newProcUnitConfObjDict
964 943
965 def setup(self, id, name, description):
944 def setup(self, id, name='', description=''):
966 945
946 print
947 print '*'*60
948 print ' Starting SIGNAL CHAIN PROCESSING v%s ' % schainpy.__version__
949 print '*'*60
950 print
967 951 self.id = str(id)
968 self.name = name
969 952 self.description = description
970 953
971 954 def update(self, name, description):
972 955
973 self.name = name
974 956 self.description = description
975 957
958 def clone(self):
959
960 p = Project()
961 p.procUnitConfObjDict = self.procUnitConfObjDict
962 return p
963
976 964 def addReadUnit(self, id=None, datatype=None, name=None, **kwargs):
977 965
978 966 if id is None:
979 967 idReadUnit = self.__getNewId()
980 968 else:
981 969 idReadUnit = str(id)
982 970
983 971 readUnitConfObj = ReadUnitConf()
984 972 readUnitConfObj.setup(idReadUnit, name, datatype, parentId=self.id, **kwargs)
985 973
986 974 self.procUnitConfObjDict[readUnitConfObj.getId()] = readUnitConfObj
987 975
988 976 return readUnitConfObj
989 977
990 978 def addProcUnit(self, inputId='0', datatype=None, name=None):
991 979
992 980 idProcUnit = self.__getNewId()
993 981
994 982 procUnitConfObj = ProcUnitConf()
995 983 procUnitConfObj.setup(idProcUnit, name, datatype, inputId, parentId=self.id)
996 984
997 985 self.procUnitConfObjDict[procUnitConfObj.getId()] = procUnitConfObj
998 986
999 987 return procUnitConfObj
1000 988
1001 989 def removeProcUnit(self, id):
1002 990
1003 991 if id in self.procUnitConfObjDict.keys():
1004 992 self.procUnitConfObjDict.pop(id)
1005 993
1006 994 def getReadUnitId(self):
1007 995
1008 996 readUnitConfObj = self.getReadUnitObj()
1009 997
1010 998 return readUnitConfObj.id
1011 999
1012 1000 def getReadUnitObj(self):
1013 1001
1014 1002 for obj in self.procUnitConfObjDict.values():
1015 if obj.getElementName() == "ReadUnit":
1003 if obj.getElementName() == 'ReadUnit':
1016 1004 return obj
1017 1005
1018 1006 return None
1019 1007
1020 1008 def getProcUnitObj(self, id=None, name=None):
1021 1009
1022 1010 if id != None:
1023 1011 return self.procUnitConfObjDict[id]
1024 1012
1025 1013 if name != None:
1026 1014 return self.getProcUnitObjByName(name)
1027 1015
1028 1016 return None
1029 1017
1030 1018 def getProcUnitObjByName(self, name):
1031 1019
1032 1020 for obj in self.procUnitConfObjDict.values():
1033 1021 if obj.name == name:
1034 1022 return obj
1035 1023
1036 1024 return None
1037 1025
1038 1026 def procUnitItems(self):
1039 1027
1040 1028 return self.procUnitConfObjDict.items()
1041 1029
1042 1030 def makeXml(self):
1043 1031
1044 1032 projectElement = Element('Project')
1045 1033 projectElement.set('id', str(self.id))
1046 1034 projectElement.set('name', self.name)
1047 1035 projectElement.set('description', self.description)
1048 1036
1049 1037 for procUnitConfObj in self.procUnitConfObjDict.values():
1050 1038 procUnitConfObj.makeXml(projectElement)
1051 1039
1052 1040 self.projectElement = projectElement
1053 1041
1054 1042 def writeXml(self, filename=None):
1055 1043
1056 1044 if filename == None:
1057 1045 if self.filename:
1058 1046 filename = self.filename
1059 1047 else:
1060 filename = "schain.xml"
1048 filename = 'schain.xml'
1061 1049
1062 1050 if not filename:
1063 print "filename has not been defined. Use setFilename(filename) for do it."
1051 print 'filename has not been defined. Use setFilename(filename) for do it.'
1064 1052 return 0
1065 1053
1066 1054 abs_file = os.path.abspath(filename)
1067 1055
1068 1056 if not os.access(os.path.dirname(abs_file), os.W_OK):
1069 print "No write permission on %s" %os.path.dirname(abs_file)
1057 print 'No write permission on %s' %os.path.dirname(abs_file)
1070 1058 return 0
1071 1059
1072 1060 if os.path.isfile(abs_file) and not(os.access(abs_file, os.W_OK)):
1073 print "File %s already exists and it could not be overwriten" %abs_file
1061 print 'File %s already exists and it could not be overwriten' %abs_file
1074 1062 return 0
1075 1063
1076 1064 self.makeXml()
1077 1065
1078 1066 ElementTree(self.projectElement).write(abs_file, method='xml')
1079 1067
1080 1068 self.filename = abs_file
1081 1069
1082 1070 return 1
1083 1071
1084 1072 def readXml(self, filename = None):
1085 1073
1086 1074 if not filename:
1087 print "filename is not defined"
1075 print 'filename is not defined'
1088 1076 return 0
1089 1077
1090 1078 abs_file = os.path.abspath(filename)
1091 1079
1092 1080 if not os.path.isfile(abs_file):
1093 print "%s file does not exist" %abs_file
1081 print '%s file does not exist' %abs_file
1094 1082 return 0
1095 1083
1096 1084 self.projectElement = None
1097 1085 self.procUnitConfObjDict = {}
1098 1086
1099 1087 try:
1100 1088 self.projectElement = ElementTree().parse(abs_file)
1101 1089 except:
1102 print "Error reading %s, verify file format" %filename
1090 print 'Error reading %s, verify file format' %filename
1103 1091 return 0
1104 1092
1105 1093 self.project = self.projectElement.tag
1106 1094
1107 1095 self.id = self.projectElement.get('id')
1108 1096 self.name = self.projectElement.get('name')
1109 1097 self.description = self.projectElement.get('description')
1110 1098
1111 1099 readUnitElementList = self.projectElement.iter(ReadUnitConf().getElementName())
1112 1100
1113 1101 for readUnitElement in readUnitElementList:
1114 1102 readUnitConfObj = ReadUnitConf()
1115 1103 readUnitConfObj.readXml(readUnitElement)
1116 1104
1117 1105 if readUnitConfObj.parentId == None:
1118 1106 readUnitConfObj.parentId = self.id
1119 1107
1120 1108 self.procUnitConfObjDict[readUnitConfObj.getId()] = readUnitConfObj
1121 1109
1122 1110 procUnitElementList = self.projectElement.iter(ProcUnitConf().getElementName())
1123 1111
1124 1112 for procUnitElement in procUnitElementList:
1125 1113 procUnitConfObj = ProcUnitConf()
1126 1114 procUnitConfObj.readXml(procUnitElement)
1127 1115
1128 1116 if procUnitConfObj.parentId == None:
1129 1117 procUnitConfObj.parentId = self.id
1130 1118
1131 1119 self.procUnitConfObjDict[procUnitConfObj.getId()] = procUnitConfObj
1132 1120
1133 1121 self.filename = abs_file
1134 1122
1135 1123 return 1
1136 1124
1137 1125 def printattr(self):
1138 1126
1139 print "Project[%s]: name = %s, description = %s" %(self.id,
1127 print 'Project[%s]: name = %s, description = %s' %(self.id,
1140 1128 self.name,
1141 1129 self.description)
1142 1130
1143 1131 for procUnitConfObj in self.procUnitConfObjDict.values():
1144 1132 procUnitConfObj.printattr()
1145 1133
1146 1134 def createObjects(self):
1147 1135
1148 1136 for procUnitConfObj in self.procUnitConfObjDict.values():
1149 1137 procUnitConfObj.createObjects(self.plotterQueue)
1150 1138
1151 1139 def __connect(self, objIN, thisObj):
1152 1140
1153 1141 thisObj.setInput(objIN.getOutputObj())
1154 1142
1155 1143 def connectObjects(self):
1156 1144
1157 1145 for thisPUConfObj in self.procUnitConfObjDict.values():
1158 1146
1159 1147 inputId = thisPUConfObj.getInputId()
1160 1148
1161 1149 if int(inputId) == 0:
1162 1150 continue
1163 1151
1164 1152 #Get input object
1165 1153 puConfINObj = self.procUnitConfObjDict[inputId]
1166 1154 puObjIN = puConfINObj.getProcUnitObj()
1167 1155
1168 1156 #Get current object
1169 1157 thisPUObj = thisPUConfObj.getProcUnitObj()
1170 1158
1171 1159 self.__connect(puObjIN, thisPUObj)
1172 1160
1173 def __handleError(self, procUnitConfObj, send_email=True):
1161 def __handleError(self, procUnitConfObj, send_email=False):
1174 1162
1175 1163 import socket
1176 1164
1177 1165 err = traceback.format_exception(sys.exc_info()[0],
1178 1166 sys.exc_info()[1],
1179 1167 sys.exc_info()[2])
1180 1168
1181 print "***** Error occurred in %s *****" %(procUnitConfObj.name)
1182 print "***** %s" %err[-1]
1169 print '***** Error occurred in %s *****' %(procUnitConfObj.name)
1170 print '***** %s' %err[-1]
1183 1171
1184 message = "".join(err)
1172 message = ''.join(err)
1185 1173
1186 1174 sys.stderr.write(message)
1187 1175
1188 1176 if not send_email:
1189 1177 return
1190 1178
1191 subject = "SChain v%s: Error running %s\n" %(schainpy.__version__, procUnitConfObj.name)
1179 subject = 'SChain v%s: Error running %s\n' %(schainpy.__version__, procUnitConfObj.name)
1192 1180
1193 subtitle = "%s: %s\n" %(procUnitConfObj.getElementName() ,procUnitConfObj.name)
1194 subtitle += "Hostname: %s\n" %socket.gethostbyname(socket.gethostname())
1195 subtitle += "Working directory: %s\n" %os.path.abspath("./")
1196 subtitle += "Configuration file: %s\n" %self.filename
1197 subtitle += "Time: %s\n" %str(datetime.datetime.now())
1181 subtitle = '%s: %s\n' %(procUnitConfObj.getElementName() ,procUnitConfObj.name)
1182 subtitle += 'Hostname: %s\n' %socket.gethostbyname(socket.gethostname())
1183 subtitle += 'Working directory: %s\n' %os.path.abspath('./')
1184 subtitle += 'Configuration file: %s\n' %self.filename
1185 subtitle += 'Time: %s\n' %str(datetime.datetime.now())
1198 1186
1199 1187 readUnitConfObj = self.getReadUnitObj()
1200 1188 if readUnitConfObj:
1201 subtitle += "\nInput parameters:\n"
1202 subtitle += "[Data path = %s]\n" %readUnitConfObj.path
1203 subtitle += "[Data type = %s]\n" %readUnitConfObj.datatype
1204 subtitle += "[Start date = %s]\n" %readUnitConfObj.startDate
1205 subtitle += "[End date = %s]\n" %readUnitConfObj.endDate
1206 subtitle += "[Start time = %s]\n" %readUnitConfObj.startTime
1207 subtitle += "[End time = %s]\n" %readUnitConfObj.endTime
1189 subtitle += '\nInput parameters:\n'
1190 subtitle += '[Data path = %s]\n' %readUnitConfObj.path
1191 subtitle += '[Data type = %s]\n' %readUnitConfObj.datatype
1192 subtitle += '[Start date = %s]\n' %readUnitConfObj.startDate
1193 subtitle += '[End date = %s]\n' %readUnitConfObj.endDate
1194 subtitle += '[Start time = %s]\n' %readUnitConfObj.startTime
1195 subtitle += '[End time = %s]\n' %readUnitConfObj.endTime
1208 1196
1209 1197 adminObj = schainpy.admin.SchainNotify()
1210 1198 adminObj.sendAlert(message=message,
1211 1199 subject=subject,
1212 1200 subtitle=subtitle,
1213 1201 filename=self.filename)
1214 1202
1215 1203 def isPaused(self):
1216 1204 return 0
1217 1205
1218 1206 def isStopped(self):
1219 1207 return 0
1220 1208
1221 1209 def runController(self):
1222 """
1210 '''
1223 1211 returns 0 when this process has been stopped, 1 otherwise
1224 """
1212 '''
1225 1213
1226 1214 if self.isPaused():
1227 print "Process suspended"
1215 print 'Process suspended'
1228 1216
1229 1217 while True:
1230 sleep(0.1)
1218 time.sleep(0.1)
1231 1219
1232 1220 if not self.isPaused():
1233 1221 break
1234 1222
1235 1223 if self.isStopped():
1236 1224 break
1237 1225
1238 print "Process reinitialized"
1226 print 'Process reinitialized'
1239 1227
1240 1228 if self.isStopped():
1241 print "Process stopped"
1229 print 'Process stopped'
1242 1230 return 0
1243 1231
1244 1232 return 1
1245 1233
1246 1234 def setFilename(self, filename):
1247 1235
1248 1236 self.filename = filename
1249 1237
1250 1238 def setPlotterQueue(self, plotter_queue):
1251 1239
1252 raise NotImplementedError, "Use schainpy.controller_api.ControllerThread instead Project class"
1240 raise NotImplementedError, 'Use schainpy.controller_api.ControllerThread instead Project class'
1253 1241
1254 1242 def getPlotterQueue(self):
1255 1243
1256 raise NotImplementedError, "Use schainpy.controller_api.ControllerThread instead Project class"
1244 raise NotImplementedError, 'Use schainpy.controller_api.ControllerThread instead Project class'
1257 1245
1258 1246 def useExternalPlotter(self):
1259 1247
1260 raise NotImplementedError, "Use schainpy.controller_api.ControllerThread instead Project class"
1248 raise NotImplementedError, 'Use schainpy.controller_api.ControllerThread instead Project class'
1261 1249
1262 1250 def run(self):
1263 1251
1264 print
1265 print "*"*60
1266 print " Starting SIGNAL CHAIN PROCESSING v%s " %schainpy.__version__
1267 print "*"*60
1268 print
1252 log.success('Starting {}'.format(self.name))
1253
1254 self.createObjects()
1255 self.connectObjects()
1269 1256
1270 1257 keyList = self.procUnitConfObjDict.keys()
1271 1258 keyList.sort()
1272 1259
1273 1260 while(True):
1274 1261
1275 1262 is_ok = False
1276 1263
1277 1264 for procKey in keyList:
1278 # print "Running the '%s' process with %s" %(procUnitConfObj.name, procUnitConfObj.id)
1279 1265
1280 1266 procUnitConfObj = self.procUnitConfObjDict[procKey]
1281 1267
1282 1268 try:
1283 1269 sts = procUnitConfObj.run()
1284 1270 is_ok = is_ok or sts
1285 1271 except KeyboardInterrupt:
1286 1272 is_ok = False
1287 1273 break
1288 1274 except ValueError, e:
1289 sleep(0.5)
1275 time.sleep(0.5)
1290 1276 self.__handleError(procUnitConfObj, send_email=True)
1291 1277 is_ok = False
1292 1278 break
1293 1279 except:
1294 sleep(0.5)
1280 time.sleep(0.5)
1295 1281 self.__handleError(procUnitConfObj)
1296 1282 is_ok = False
1297 1283 break
1298 1284
1299 1285 #If every process unit finished so end process
1300 1286 if not(is_ok):
1301 # print "Every process unit have finished"
1302 1287 break
1303 1288
1304 1289 if not self.runController():
1305 1290 break
1306 1291
1307 1292 #Closing every process
1308 1293 for procKey in keyList:
1309 1294 procUnitConfObj = self.procUnitConfObjDict[procKey]
1310 1295 procUnitConfObj.close()
1311 1296
1312 print "Process finished"
1313
1314 def start(self, filename=None):
1315
1316 self.writeXml(filename)
1317 self.createObjects()
1318 self.connectObjects()
1319 self.run()
1297 log.success('{} finished'.format(self.name))
Show file before | Show file after | Hide comments
@@ -1,179 +1,179
1 1 import threading
2 2 from Queue import Queue
3 3
4 4 from schainpy.controller import Project
5 5 from schainpy.model.graphics.jroplotter import PlotManager
6 6
7 7 class ControllerThread(threading.Thread, Project):
8 8
9 9 def __init__(self, plotter_queue=None):
10 10
11 11 threading.Thread.__init__(self)
12 12 Project.__init__(self, plotter_queue)
13 13
14 14 self.setDaemon(True)
15 15
16 16 self.lock = threading.Lock()
17 self.control = {'stop':False, 'pause':False}
18
17 self.control = { 'stop':False, 'pause':False }
18
19 19 def __del__(self):
20 20
21 21 self.control['stop'] = True
22 22
23 23 def stop(self):
24 24
25 25 self.lock.acquire()
26 26
27 27 self.control['stop'] = True
28 28
29 29 self.lock.release()
30 30
31 31 def pause(self):
32 32
33 33 self.lock.acquire()
34 34
35 35 self.control['pause'] = not(self.control['pause'])
36 36 paused = self.control['pause']
37 37
38 38 self.lock.release()
39 39
40 40 return paused
41 41
42 42 def isPaused(self):
43 43
44 44 self.lock.acquire()
45 45 paused = self.control['pause']
46 46 self.lock.release()
47 47
48 48 return paused
49 49
50 50 def isStopped(self):
51 51
52 52 self.lock.acquire()
53 53 stopped = self.control['stop']
54 54 self.lock.release()
55 55
56 56 return stopped
57 57
58 58 def run(self):
59 59 self.control['stop'] = False
60 60 self.control['pause'] = False
61 61
62 62 self.writeXml()
63 63
64 64 self.createObjects()
65 65 self.connectObjects()
66 66 Project.run(self)
67 67
68 68 def isRunning(self):
69 69
70 70 return self.is_alive()
71 71
72 72 def isFinished(self):
73 73
74 74 return not self.is_alive()
75 75
76 76 def setPlotters(self):
77 77
78 78 plotterList = PlotManager.plotterList
79 79
80 80 for thisPUConfObj in self.procUnitConfObjDict.values():
81 81
82 82 inputId = thisPUConfObj.getInputId()
83 83
84 84 if int(inputId) == 0:
85 85 continue
86 86
87 87 for thisOpObj in thisPUConfObj.getOperationObjList():
88 88
89 89 if thisOpObj.type == "self":
90 90 continue
91 91
92 92 if thisOpObj.name in plotterList:
93 93 thisOpObj.type = "other"
94 94
95 95 def setPlotterQueue(self, plotter_queue):
96 96
97 97 self.plotterQueue = plotter_queue
98 98
99 99 def getPlotterQueue(self):
100 100
101 101 return self.plotterQueue
102 102
103 103 def useExternalPlotter(self):
104 104
105 105 self.plotterQueue = Queue(10)
106 106 self.setPlotters()
107 107
108 108 plotManagerObj = PlotManager(self.plotterQueue)
109 109 plotManagerObj.setController(self)
110 110
111 111 return plotManagerObj
112 112
113 113 # from PyQt4 import QtCore
114 114 # from PyQt4.QtCore import SIGNAL
115 115 #
116 116 # class ControllerQThread(QtCore.QThread, Project):
117 117 #
118 118 # def __init__(self, filename):
119 119 #
120 120 # QtCore.QThread.__init__(self)
121 121 # Project.__init__(self)
122 122 #
123 123 # self.filename = filename
124 124 #
125 125 # self.lock = threading.Lock()
126 126 # self.control = {'stop':False, 'pause':False}
127 127 #
128 128 # def __del__(self):
129 129 #
130 130 # self.control['stop'] = True
131 131 # self.wait()
132 132 #
133 133 # def stop(self):
134 134 #
135 135 # self.lock.acquire()
136 136 #
137 137 # self.control['stop'] = True
138 138 #
139 139 # self.lock.release()
140 140 #
141 141 # def pause(self):
142 142 #
143 143 # self.lock.acquire()
144 144 #
145 145 # self.control['pause'] = not(self.control['pause'])
146 146 # paused = self.control['pause']
147 147 #
148 148 # self.lock.release()
149 149 #
150 150 # return paused
151 151 #
152 152 # def isPaused(self):
153 153 #
154 154 # self.lock.acquire()
155 155 # paused = self.control['pause']
156 156 # self.lock.release()
157 157 #
158 158 # return paused
159 159 #
160 160 # def isStopped(self):
161 161 #
162 162 # self.lock.acquire()
163 163 # stopped = self.control['stop']
164 164 # self.lock.release()
165 165 #
166 166 # return stopped
167 167 #
168 168 # def run(self):
169 169 #
170 170 # self.control['stop'] = False
171 171 # self.control['pause'] = False
172 172 #
173 173 # self.readXml(self.filename)
174 174 # self.createObjects()
175 175 # self.connectObjects()
176 176 # self.emit( SIGNAL( "jobStarted( PyQt_PyObject )" ), 1)
177 177 # Project.run(self)
178 178 # self.emit( SIGNAL( "jobFinished( PyQt_PyObject )" ), 1)
179 179 #
Show file before | Show file after | Hide comments
1 NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff
Show file before | Show file after | Hide comments
@@ -1,12 +1,12
1 1 #from schainpy.model.data.jrodata import *
2 2 # from schainpy.model.io.jrodataIO import *
3 3 # from schainpy.model.proc.jroprocessing import *
4 4 # from schainpy.model.graphics.jroplot import *
5 5 # from schainpy.model.utils.jroutils import *
6 6 # from schainpy.serializer import *
7 7
8 8 from data import *
9 9 from io import *
10 10 from proc import *
11 11 from graphics import *
12 from utils import *
12 from utils import * No newline at end of file
Show file before | Show file after | Hide comments
@@ -1,1229 +1,1229
1 1 '''
2 2
3 3 $Author: murco $
4 4 $Id: JROData.py 173 2012-11-20 15:06:21Z murco $
5 5 '''
6 6
7 7 import copy
8 8 import numpy
9 9 import datetime
10 10
11 11 from jroheaderIO import SystemHeader, RadarControllerHeader
12 12 from schainpy import cSchain
13 13
14 14
15 15 def getNumpyDtype(dataTypeCode):
16 16
17 17 if dataTypeCode == 0:
18 18 numpyDtype = numpy.dtype([('real','<i1'),('imag','<i1')])
19 19 elif dataTypeCode == 1:
20 20 numpyDtype = numpy.dtype([('real','<i2'),('imag','<i2')])
21 21 elif dataTypeCode == 2:
22 22 numpyDtype = numpy.dtype([('real','<i4'),('imag','<i4')])
23 23 elif dataTypeCode == 3:
24 24 numpyDtype = numpy.dtype([('real','<i8'),('imag','<i8')])
25 25 elif dataTypeCode == 4:
26 26 numpyDtype = numpy.dtype([('real','<f4'),('imag','<f4')])
27 27 elif dataTypeCode == 5:
28 28 numpyDtype = numpy.dtype([('real','<f8'),('imag','<f8')])
29 29 else:
30 30 raise ValueError, 'dataTypeCode was not defined'
31 31
32 32 return numpyDtype
33 33
34 34 def getDataTypeCode(numpyDtype):
35 35
36 36 if numpyDtype == numpy.dtype([('real','<i1'),('imag','<i1')]):
37 37 datatype = 0
38 38 elif numpyDtype == numpy.dtype([('real','<i2'),('imag','<i2')]):
39 39 datatype = 1
40 40 elif numpyDtype == numpy.dtype([('real','<i4'),('imag','<i4')]):
41 41 datatype = 2
42 42 elif numpyDtype == numpy.dtype([('real','<i8'),('imag','<i8')]):
43 43 datatype = 3
44 44 elif numpyDtype == numpy.dtype([('real','<f4'),('imag','<f4')]):
45 45 datatype = 4
46 46 elif numpyDtype == numpy.dtype([('real','<f8'),('imag','<f8')]):
47 47 datatype = 5
48 48 else:
49 49 datatype = None
50 50
51 51 return datatype
52 52
53 53 def hildebrand_sekhon(data, navg):
54 54 """
55 55 This method is for the objective determination of the noise level in Doppler spectra. This
56 56 implementation technique is based on the fact that the standard deviation of the spectral
57 57 densities is equal to the mean spectral density for white Gaussian noise
58 58
59 59 Inputs:
60 60 Data : heights
61 61 navg : numbers of averages
62 62
63 63 Return:
64 64 -1 : any error
65 65 anoise : noise's level
66 66 """
67 67
68 68 sortdata = numpy.sort(data, axis=None)
69 69 # lenOfData = len(sortdata)
70 70 # nums_min = lenOfData*0.2
71 71 #
72 72 # if nums_min <= 5:
73 73 # nums_min = 5
74 74 #
75 75 # sump = 0.
76 76 #
77 77 # sumq = 0.
78 78 #
79 79 # j = 0
80 80 #
81 81 # cont = 1
82 82 #
83 83 # while((cont==1)and(j<lenOfData)):
84 84 #
85 85 # sump += sortdata[j]
86 86 #
87 87 # sumq += sortdata[j]**2
88 88 #
89 89 # if j > nums_min:
90 90 # rtest = float(j)/(j-1) + 1.0/navg
91 91 # if ((sumq*j) > (rtest*sump**2)):
92 92 # j = j - 1
93 93 # sump = sump - sortdata[j]
94 94 # sumq = sumq - sortdata[j]**2
95 95 # cont = 0
96 96 #
97 97 # j += 1
98 98 #
99 99 # lnoise = sump /j
100 100 #
101 101 # return lnoise
102 102
103 103 return cSchain.hildebrand_sekhon(sortdata, navg)
104 104
105 105
106 106 class Beam:
107 107
108 108 def __init__(self):
109 109 self.codeList = []
110 110 self.azimuthList = []
111 111 self.zenithList = []
112 112
113 113 class GenericData(object):
114 114
115 115 flagNoData = True
116 116
117 117 def copy(self, inputObj=None):
118 118
119 119 if inputObj == None:
120 120 return copy.deepcopy(self)
121 121
122 122 for key in inputObj.__dict__.keys():
123 123
124 124 attribute = inputObj.__dict__[key]
125 125
126 126 #If this attribute is a tuple or list
127 127 if type(inputObj.__dict__[key]) in (tuple, list):
128 128 self.__dict__[key] = attribute[:]
129 129 continue
130 130
131 131 #If this attribute is another object or instance
132 132 if hasattr(attribute, '__dict__'):
133 133 self.__dict__[key] = attribute.copy()
134 134 continue
135 135
136 136 self.__dict__[key] = inputObj.__dict__[key]
137 137
138 138 def deepcopy(self):
139 139
140 140 return copy.deepcopy(self)
141 141
142 142 def isEmpty(self):
143 143
144 144 return self.flagNoData
145 145
146 146 class JROData(GenericData):
147 147
148 148 # m_BasicHeader = BasicHeader()
149 149 # m_ProcessingHeader = ProcessingHeader()
150 150
151 151 systemHeaderObj = SystemHeader()
152 152
153 153 radarControllerHeaderObj = RadarControllerHeader()
154 154
155 155 # data = None
156 156
157 157 type = None
158 158
159 159 datatype = None #dtype but in string
160 160
161 161 # dtype = None
162 162
163 163 # nChannels = None
164 164
165 165 # nHeights = None
166 166
167 167 nProfiles = None
168 168
169 169 heightList = None
170 170
171 171 channelList = None
172 172
173 173 flagDiscontinuousBlock = False
174 174
175 175 useLocalTime = False
176 176
177 177 utctime = None
178 178
179 179 timeZone = None
180 180
181 181 dstFlag = None
182 182
183 183 errorCount = None
184 184
185 185 blocksize = None
186 186
187 187 # nCode = None
188 188 #
189 189 # nBaud = None
190 190 #
191 191 # code = None
192 192
193 193 flagDecodeData = False #asumo q la data no esta decodificada
194 194
195 195 flagDeflipData = False #asumo q la data no esta sin flip
196 196
197 197 flagShiftFFT = False
198 198
199 199 # ippSeconds = None
200 200
201 201 # timeInterval = None
202 202
203 203 nCohInt = None
204 204
205 205 # noise = None
206 206
207 207 windowOfFilter = 1
208 208
209 209 #Speed of ligth
210 210 C = 3e8
211 211
212 212 frequency = 49.92e6
213 213
214 214 realtime = False
215 215
216 216 beacon_heiIndexList = None
217 217
218 218 last_block = None
219 219
220 220 blocknow = None
221 221
222 222 azimuth = None
223 223
224 224 zenith = None
225 225
226 226 beam = Beam()
227 227
228 228 profileIndex = None
229 229
230 230 def getNoise(self):
231 231
232 232 raise NotImplementedError
233 233
234 234 def getNChannels(self):
235 235
236 236 return len(self.channelList)
237 237
238 238 def getChannelIndexList(self):
239 239
240 240 return range(self.nChannels)
241 241
242 242 def getNHeights(self):
243 243
244 244 return len(self.heightList)
245 245
246 246 def getHeiRange(self, extrapoints=0):
247 247
248 248 heis = self.heightList
249 249 # deltah = self.heightList[1] - self.heightList[0]
250 250 #
251 251 # heis.append(self.heightList[-1])
252 252
253 253 return heis
254 254
255 255 def getDeltaH(self):
256 256
257 257 delta = self.heightList[1] - self.heightList[0]
258 258
259 259 return delta
260 260
261 261 def getltctime(self):
262 262
263 263 if self.useLocalTime:
264 264 return self.utctime - self.timeZone*60
265 265
266 266 return self.utctime
267 267
268 268 def getDatatime(self):
269 269
270 270 datatimeValue = datetime.datetime.utcfromtimestamp(self.ltctime)
271 271 return datatimeValue
272 272
273 273 def getTimeRange(self):
274 274
275 275 datatime = []
276 276
277 277 datatime.append(self.ltctime)
278 278 datatime.append(self.ltctime + self.timeInterval+1)
279 279
280 280 datatime = numpy.array(datatime)
281 281
282 282 return datatime
283 283
284 284 def getFmaxTimeResponse(self):
285 285
286 286 period = (10**-6)*self.getDeltaH()/(0.15)
287 287
288 288 PRF = 1./(period * self.nCohInt)
289 289
290 290 fmax = PRF
291 291
292 292 return fmax
293 293
294 294 def getFmax(self):
295 295
296 296 PRF = 1./(self.ippSeconds * self.nCohInt)
297 297
298 298 fmax = PRF
299 299
300 300 return fmax
301 301
302 302 def getVmax(self):
303 303
304 304 _lambda = self.C/self.frequency
305 305
306 306 vmax = self.getFmax() * _lambda/2
307 307
308 308 return vmax
309 309
310 310 def get_ippSeconds(self):
311 311 '''
312 312 '''
313 313 return self.radarControllerHeaderObj.ippSeconds
314 314
315 315 def set_ippSeconds(self, ippSeconds):
316 316 '''
317 317 '''
318 318
319 319 self.radarControllerHeaderObj.ippSeconds = ippSeconds
320 320
321 321 return
322 322
323 323 def get_dtype(self):
324 324 '''
325 325 '''
326 326 return getNumpyDtype(self.datatype)
327 327
328 328 def set_dtype(self, numpyDtype):
329 329 '''
330 330 '''
331 331
332 332 self.datatype = getDataTypeCode(numpyDtype)
333 333
334 334 def get_code(self):
335 335 '''
336 336 '''
337 337 return self.radarControllerHeaderObj.code
338 338
339 339 def set_code(self, code):
340 340 '''
341 341 '''
342 342 self.radarControllerHeaderObj.code = code
343 343
344 344 return
345 345
346 346 def get_ncode(self):
347 347 '''
348 348 '''
349 349 return self.radarControllerHeaderObj.nCode
350 350
351 351 def set_ncode(self, nCode):
352 352 '''
353 353 '''
354 354 self.radarControllerHeaderObj.nCode = nCode
355 355
356 356 return
357 357
358 358 def get_nbaud(self):
359 359 '''
360 360 '''
361 361 return self.radarControllerHeaderObj.nBaud
362 362
363 363 def set_nbaud(self, nBaud):
364 364 '''
365 365 '''
366 366 self.radarControllerHeaderObj.nBaud = nBaud
367 367
368 368 return
369 369
370 370 nChannels = property(getNChannels, "I'm the 'nChannel' property.")
371 371 channelIndexList = property(getChannelIndexList, "I'm the 'channelIndexList' property.")
372 372 nHeights = property(getNHeights, "I'm the 'nHeights' property.")
373 373 #noise = property(getNoise, "I'm the 'nHeights' property.")
374 374 datatime = property(getDatatime, "I'm the 'datatime' property")
375 375 ltctime = property(getltctime, "I'm the 'ltctime' property")
376 376 ippSeconds = property(get_ippSeconds, set_ippSeconds)
377 377 dtype = property(get_dtype, set_dtype)
378 378 # timeInterval = property(getTimeInterval, "I'm the 'timeInterval' property")
379 379 code = property(get_code, set_code)
380 380 nCode = property(get_ncode, set_ncode)
381 381 nBaud = property(get_nbaud, set_nbaud)
382 382
383 383 class Voltage(JROData):
384 384
385 385 #data es un numpy array de 2 dmensiones (canales, alturas)
386 386 data = None
387 387
388 388 def __init__(self):
389 389 '''
390 390 Constructor
391 391 '''
392 392
393 393 self.useLocalTime = True
394 394
395 395 self.radarControllerHeaderObj = RadarControllerHeader()
396 396
397 397 self.systemHeaderObj = SystemHeader()
398 398
399 399 self.type = "Voltage"
400 400
401 401 self.data = None
402 402
403 403 # self.dtype = None
404 404
405 405 # self.nChannels = 0
406 406
407 407 # self.nHeights = 0
408 408
409 409 self.nProfiles = None
410 410
411 411 self.heightList = None
412 412
413 413 self.channelList = None
414 414
415 415 # self.channelIndexList = None
416 416
417 417 self.flagNoData = True
418 418
419 419 self.flagDiscontinuousBlock = False
420 420
421 421 self.utctime = None
422 422
423 423 self.timeZone = None
424 424
425 425 self.dstFlag = None
426 426
427 427 self.errorCount = None
428 428
429 429 self.nCohInt = None
430 430
431 431 self.blocksize = None
432 432
433 433 self.flagDecodeData = False #asumo q la data no esta decodificada
434 434
435 435 self.flagDeflipData = False #asumo q la data no esta sin flip
436 436
437 437 self.flagShiftFFT = False
438 438
439 439 self.flagDataAsBlock = False #Asumo que la data es leida perfil a perfil
440 440
441 441 self.profileIndex = 0
442 442
443 443 def getNoisebyHildebrand(self, channel = None):
444 444 """
445 445 Determino el nivel de ruido usando el metodo Hildebrand-Sekhon
446 446
447 447 Return:
448 448 noiselevel
449 449 """
450 450
451 451 if channel != None:
452 452 data = self.data[channel]
453 453 nChannels = 1
454 454 else:
455 455 data = self.data
456 456 nChannels = self.nChannels
457 457
458 458 noise = numpy.zeros(nChannels)
459 459 power = data * numpy.conjugate(data)
460 460
461 461 for thisChannel in range(nChannels):
462 462 if nChannels == 1:
463 463 daux = power[:].real
464 464 else:
465 465 daux = power[thisChannel,:].real
466 466 noise[thisChannel] = hildebrand_sekhon(daux, self.nCohInt)
467 467
468 468 return noise
469 469
470 470 def getNoise(self, type = 1, channel = None):
471 471
472 472 if type == 1:
473 473 noise = self.getNoisebyHildebrand(channel)
474 474
475 475 return noise
476 476
477 477 def getPower(self, channel = None):
478 478
479 479 if channel != None:
480 480 data = self.data[channel]
481 481 else:
482 482 data = self.data
483 483
484 484 power = data * numpy.conjugate(data)
485 485 powerdB = 10*numpy.log10(power.real)
486 486 powerdB = numpy.squeeze(powerdB)
487 487
488 488 return powerdB
489 489
490 490 def getTimeInterval(self):
491 491
492 492 timeInterval = self.ippSeconds * self.nCohInt
493 493
494 494 return timeInterval
495 495
496 496 noise = property(getNoise, "I'm the 'nHeights' property.")
497 497 timeInterval = property(getTimeInterval, "I'm the 'timeInterval' property")
498 498
499 499 class Spectra(JROData):
500 500
501 501 #data spc es un numpy array de 2 dmensiones (canales, perfiles, alturas)
502 502 data_spc = None
503 503
504 504 #data cspc es un numpy array de 2 dmensiones (canales, pares, alturas)
505 505 data_cspc = None
506 506
507 507 #data dc es un numpy array de 2 dmensiones (canales, alturas)
508 508 data_dc = None
509 509
510 510 #data power
511 511 data_pwr = None
512 512
513 513 nFFTPoints = None
514 514
515 515 # nPairs = None
516 516
517 517 pairsList = None
518 518
519 519 nIncohInt = None
520 520
521 521 wavelength = None #Necesario para cacular el rango de velocidad desde la frecuencia
522 522
523 523 nCohInt = None #se requiere para determinar el valor de timeInterval
524 524
525 525 ippFactor = None
526 526
527 527 profileIndex = 0
528 528
529 529 plotting = "spectra"
530 530
531 531 def __init__(self):
532 532 '''
533 533 Constructor
534 534 '''
535 535
536 536 self.useLocalTime = True
537 537
538 538 self.radarControllerHeaderObj = RadarControllerHeader()
539 539
540 540 self.systemHeaderObj = SystemHeader()
541 541
542 542 self.type = "Spectra"
543 543
544 544 # self.data = None
545 545
546 546 # self.dtype = None
547 547
548 548 # self.nChannels = 0
549 549
550 550 # self.nHeights = 0
551 551
552 552 self.nProfiles = None
553 553
554 554 self.heightList = None
555 555
556 556 self.channelList = None
557 557
558 558 # self.channelIndexList = None
559 559
560 560 self.pairsList = None
561 561
562 562 self.flagNoData = True
563 563
564 564 self.flagDiscontinuousBlock = False
565 565
566 566 self.utctime = None
567 567
568 568 self.nCohInt = None
569 569
570 570 self.nIncohInt = None
571 571
572 572 self.blocksize = None
573 573
574 574 self.nFFTPoints = None
575 575
576 576 self.wavelength = None
577 577
578 578 self.flagDecodeData = False #asumo q la data no esta decodificada
579 579
580 580 self.flagDeflipData = False #asumo q la data no esta sin flip
581 581
582 582 self.flagShiftFFT = False
583 583
584 584 self.ippFactor = 1
585 585
586 586 #self.noise = None
587 587
588 588 self.beacon_heiIndexList = []
589 589
590 590 self.noise_estimation = None
591 591
592 592
593 593 def getNoisebyHildebrand(self, xmin_index=None, xmax_index=None, ymin_index=None, ymax_index=None):
594 594 """
595 595 Determino el nivel de ruido usando el metodo Hildebrand-Sekhon
596 596
597 597 Return:
598 598 noiselevel
599 599 """
600 600
601 601 noise = numpy.zeros(self.nChannels)
602 602
603 603 for channel in range(self.nChannels):
604 604 daux = self.data_spc[channel,xmin_index:xmax_index,ymin_index:ymax_index]
605 605 noise[channel] = hildebrand_sekhon(daux, self.nIncohInt)
606 606
607 607 return noise
608 608
609 609 def getNoise(self, xmin_index=None, xmax_index=None, ymin_index=None, ymax_index=None):
610 610
611 611 if self.noise_estimation is not None:
612 612 return self.noise_estimation #this was estimated by getNoise Operation defined in jroproc_spectra.py
613 613 else:
614 614 noise = self.getNoisebyHildebrand(xmin_index, xmax_index, ymin_index, ymax_index)
615 615 return noise
616 616
617 617 def getFreqRangeTimeResponse(self, extrapoints=0):
618 618
619 619 deltafreq = self.getFmaxTimeResponse() / (self.nFFTPoints*self.ippFactor)
620 620 freqrange = deltafreq*(numpy.arange(self.nFFTPoints+extrapoints)-self.nFFTPoints/2.) - deltafreq/2
621 621
622 622 return freqrange
623 623
624 624 def getAcfRange(self, extrapoints=0):
625 625
626 626 deltafreq = 10./(self.getFmax() / (self.nFFTPoints*self.ippFactor))
627 627 freqrange = deltafreq*(numpy.arange(self.nFFTPoints+extrapoints)-self.nFFTPoints/2.) - deltafreq/2
628 628
629 629 return freqrange
630 630
631 631 def getFreqRange(self, extrapoints=0):
632 632
633 633 deltafreq = self.getFmax() / (self.nFFTPoints*self.ippFactor)
634 634 freqrange = deltafreq*(numpy.arange(self.nFFTPoints+extrapoints)-self.nFFTPoints/2.) - deltafreq/2
635 635
636 636 return freqrange
637 637
638 638 def getVelRange(self, extrapoints=0):
639 639
640 640 deltav = self.getVmax() / (self.nFFTPoints*self.ippFactor)
641 641 velrange = deltav*(numpy.arange(self.nFFTPoints+extrapoints)-self.nFFTPoints/2.) #- deltav/2
642 642
643 643 return velrange
644 644
645 645 def getNPairs(self):
646 646
647 647 return len(self.pairsList)
648 648
649 649 def getPairsIndexList(self):
650 650
651 651 return range(self.nPairs)
652 652
653 653 def getNormFactor(self):
654 654
655 655 pwcode = 1
656 656
657 657 if self.flagDecodeData:
658 658 pwcode = numpy.sum(self.code[0]**2)
659 659 #normFactor = min(self.nFFTPoints,self.nProfiles)*self.nIncohInt*self.nCohInt*pwcode*self.windowOfFilter
660 660 normFactor = self.nProfiles*self.nIncohInt*self.nCohInt*pwcode*self.windowOfFilter
661 661
662 662 return normFactor
663 663
664 664 def getFlagCspc(self):
665 665
666 666 if self.data_cspc is None:
667 667 return True
668 668
669 669 return False
670 670
671 671 def getFlagDc(self):
672 672
673 673 if self.data_dc is None:
674 674 return True
675 675
676 676 return False
677 677
678 678 def getTimeInterval(self):
679 679
680 680 timeInterval = self.ippSeconds * self.nCohInt * self.nIncohInt * self.nProfiles
681 681
682 682 return timeInterval
683 683
684 684 def getPower(self):
685 685
686 686 factor = self.normFactor
687 687 z = self.data_spc/factor
688 688 z = numpy.where(numpy.isfinite(z), z, numpy.NAN)
689 689 avg = numpy.average(z, axis=1)
690 690
691 691 return 10*numpy.log10(avg)
692 692
693 693 def getCoherence(self, pairsList=None, phase=False):
694 694
695 695 z = []
696 696 if pairsList is None:
697 697 pairsIndexList = self.pairsIndexList
698 698 else:
699 699 pairsIndexList = []
700 700 for pair in pairsList:
701 701 if pair not in self.pairsList:
702 702 raise ValueError, "Pair %s is not in dataOut.pairsList" %(pair)
703 pairsIndexList.append(self.pairsList.index(pair))
703 pairsIndexList.append(self.pairsList.index(pair))
704 704 for i in range(len(pairsIndexList)):
705 705 pair = self.pairsList[pairsIndexList[i]]
706 706 ccf = numpy.average(self.data_cspc[pairsIndexList[i], :, :], axis=0)
707 707 powa = numpy.average(self.data_spc[pair[0], :, :], axis=0)
708 708 powb = numpy.average(self.data_spc[pair[1], :, :], axis=0)
709 709 avgcoherenceComplex = ccf/numpy.sqrt(powa*powb)
710 710 if phase:
711 711 data = numpy.arctan2(avgcoherenceComplex.imag,
712 712 avgcoherenceComplex.real)*180/numpy.pi
713 713 else:
714 714 data = numpy.abs(avgcoherenceComplex)
715 715
716 716 z.append(data)
717 717
718 718 return numpy.array(z)
719 719
720 720 def setValue(self, value):
721 721
722 722 print "This property should not be initialized"
723 723
724 724 return
725 725
726 726 nPairs = property(getNPairs, setValue, "I'm the 'nPairs' property.")
727 727 pairsIndexList = property(getPairsIndexList, setValue, "I'm the 'pairsIndexList' property.")
728 728 normFactor = property(getNormFactor, setValue, "I'm the 'getNormFactor' property.")
729 729 flag_cspc = property(getFlagCspc, setValue)
730 730 flag_dc = property(getFlagDc, setValue)
731 731 noise = property(getNoise, setValue, "I'm the 'nHeights' property.")
732 732 timeInterval = property(getTimeInterval, setValue, "I'm the 'timeInterval' property")
733 733
734 734 class SpectraHeis(Spectra):
735 735
736 736 data_spc = None
737 737
738 738 data_cspc = None
739 739
740 740 data_dc = None
741 741
742 742 nFFTPoints = None
743 743
744 744 # nPairs = None
745 745
746 746 pairsList = None
747 747
748 748 nCohInt = None
749 749
750 750 nIncohInt = None
751 751
752 752 def __init__(self):
753 753
754 754 self.radarControllerHeaderObj = RadarControllerHeader()
755 755
756 756 self.systemHeaderObj = SystemHeader()
757 757
758 758 self.type = "SpectraHeis"
759 759
760 760 # self.dtype = None
761 761
762 762 # self.nChannels = 0
763 763
764 764 # self.nHeights = 0
765 765
766 766 self.nProfiles = None
767 767
768 768 self.heightList = None
769 769
770 770 self.channelList = None
771 771
772 772 # self.channelIndexList = None
773 773
774 774 self.flagNoData = True
775 775
776 776 self.flagDiscontinuousBlock = False
777 777
778 778 # self.nPairs = 0
779 779
780 780 self.utctime = None
781 781
782 782 self.blocksize = None
783 783
784 784 self.profileIndex = 0
785 785
786 786 self.nCohInt = 1
787 787
788 788 self.nIncohInt = 1
789 789
790 790 def getNormFactor(self):
791 791 pwcode = 1
792 792 if self.flagDecodeData:
793 793 pwcode = numpy.sum(self.code[0]**2)
794 794
795 795 normFactor = self.nIncohInt*self.nCohInt*pwcode
796 796
797 797 return normFactor
798 798
799 799 def getTimeInterval(self):
800 800
801 801 timeInterval = self.ippSeconds * self.nCohInt * self.nIncohInt
802 802
803 803 return timeInterval
804 804
805 805 normFactor = property(getNormFactor, "I'm the 'getNormFactor' property.")
806 806 timeInterval = property(getTimeInterval, "I'm the 'timeInterval' property")
807 807
808 808 class Fits(JROData):
809 809
810 810 heightList = None
811 811
812 812 channelList = None
813 813
814 814 flagNoData = True
815 815
816 816 flagDiscontinuousBlock = False
817 817
818 818 useLocalTime = False
819 819
820 820 utctime = None
821 821
822 822 timeZone = None
823 823
824 824 # ippSeconds = None
825 825
826 826 # timeInterval = None
827 827
828 828 nCohInt = None
829 829
830 830 nIncohInt = None
831 831
832 832 noise = None
833 833
834 834 windowOfFilter = 1
835 835
836 836 #Speed of ligth
837 837 C = 3e8
838 838
839 839 frequency = 49.92e6
840 840
841 841 realtime = False
842 842
843 843
844 844 def __init__(self):
845 845
846 846 self.type = "Fits"
847 847
848 848 self.nProfiles = None
849 849
850 850 self.heightList = None
851 851
852 852 self.channelList = None
853 853
854 854 # self.channelIndexList = None
855 855
856 856 self.flagNoData = True
857 857
858 858 self.utctime = None
859 859
860 860 self.nCohInt = 1
861 861
862 862 self.nIncohInt = 1
863 863
864 864 self.useLocalTime = True
865 865
866 866 self.profileIndex = 0
867 867
868 868 # self.utctime = None
869 869 # self.timeZone = None
870 870 # self.ltctime = None
871 871 # self.timeInterval = None
872 872 # self.header = None
873 873 # self.data_header = None
874 874 # self.data = None
875 875 # self.datatime = None
876 876 # self.flagNoData = False
877 877 # self.expName = ''
878 878 # self.nChannels = None
879 879 # self.nSamples = None
880 880 # self.dataBlocksPerFile = None
881 881 # self.comments = ''
882 882 #
883 883
884 884
885 885 def getltctime(self):
886 886
887 887 if self.useLocalTime:
888 888 return self.utctime - self.timeZone*60
889 889
890 890 return self.utctime
891 891
892 892 def getDatatime(self):
893 893
894 894 datatime = datetime.datetime.utcfromtimestamp(self.ltctime)
895 895 return datatime
896 896
897 897 def getTimeRange(self):
898 898
899 899 datatime = []
900 900
901 901 datatime.append(self.ltctime)
902 902 datatime.append(self.ltctime + self.timeInterval)
903 903
904 904 datatime = numpy.array(datatime)
905 905
906 906 return datatime
907 907
908 908 def getHeiRange(self):
909 909
910 910 heis = self.heightList
911 911
912 912 return heis
913 913
914 914 def getNHeights(self):
915 915
916 916 return len(self.heightList)
917 917
918 918 def getNChannels(self):
919 919
920 920 return len(self.channelList)
921 921
922 922 def getChannelIndexList(self):
923 923
924 924 return range(self.nChannels)
925 925
926 926 def getNoise(self, type = 1):
927 927
928 928 #noise = numpy.zeros(self.nChannels)
929 929
930 930 if type == 1:
931 931 noise = self.getNoisebyHildebrand()
932 932
933 933 if type == 2:
934 934 noise = self.getNoisebySort()
935 935
936 936 if type == 3:
937 937 noise = self.getNoisebyWindow()
938 938
939 939 return noise
940 940
941 941 def getTimeInterval(self):
942 942
943 943 timeInterval = self.ippSeconds * self.nCohInt * self.nIncohInt
944 944
945 945 return timeInterval
946 946
947 947 datatime = property(getDatatime, "I'm the 'datatime' property")
948 948 nHeights = property(getNHeights, "I'm the 'nHeights' property.")
949 949 nChannels = property(getNChannels, "I'm the 'nChannel' property.")
950 950 channelIndexList = property(getChannelIndexList, "I'm the 'channelIndexList' property.")
951 951 noise = property(getNoise, "I'm the 'nHeights' property.")
952 952
953 953 ltctime = property(getltctime, "I'm the 'ltctime' property")
954 954 timeInterval = property(getTimeInterval, "I'm the 'timeInterval' property")
955 955
956 956
957 957 class Correlation(JROData):
958 958
959 959 noise = None
960 960
961 961 SNR = None
962 962
963 963 #--------------------------------------------------
964 964
965 965 mode = None
966 966
967 967 split = False
968 968
969 969 data_cf = None
970 970
971 971 lags = None
972 972
973 973 lagRange = None
974 974
975 975 pairsList = None
976 976
977 977 normFactor = None
978 978
979 979 #--------------------------------------------------
980 980
981 981 # calculateVelocity = None
982 982
983 983 nLags = None
984 984
985 985 nPairs = None
986 986
987 987 nAvg = None
988 988
989 989
990 990 def __init__(self):
991 991 '''
992 992 Constructor
993 993 '''
994 994 self.radarControllerHeaderObj = RadarControllerHeader()
995 995
996 996 self.systemHeaderObj = SystemHeader()
997 997
998 998 self.type = "Correlation"
999 999
1000 1000 self.data = None
1001 1001
1002 1002 self.dtype = None
1003 1003
1004 1004 self.nProfiles = None
1005 1005
1006 1006 self.heightList = None
1007 1007
1008 1008 self.channelList = None
1009 1009
1010 1010 self.flagNoData = True
1011 1011
1012 1012 self.flagDiscontinuousBlock = False
1013 1013
1014 1014 self.utctime = None
1015 1015
1016 1016 self.timeZone = None
1017 1017
1018 1018 self.dstFlag = None
1019 1019
1020 1020 self.errorCount = None
1021 1021
1022 1022 self.blocksize = None
1023 1023
1024 1024 self.flagDecodeData = False #asumo q la data no esta decodificada
1025 1025
1026 1026 self.flagDeflipData = False #asumo q la data no esta sin flip
1027 1027
1028 1028 self.pairsList = None
1029 1029
1030 1030 self.nPoints = None
1031 1031
1032 1032 def getPairsList(self):
1033 1033
1034 1034 return self.pairsList
1035 1035
1036 1036 def getNoise(self, mode = 2):
1037 1037
1038 1038 indR = numpy.where(self.lagR == 0)[0][0]
1039 1039 indT = numpy.where(self.lagT == 0)[0][0]
1040 1040
1041 1041 jspectra0 = self.data_corr[:,:,indR,:]
1042 1042 jspectra = copy.copy(jspectra0)
1043 1043
1044 1044 num_chan = jspectra.shape[0]
1045 1045 num_hei = jspectra.shape[2]
1046 1046
1047 1047 freq_dc = jspectra.shape[1]/2
1048 1048 ind_vel = numpy.array([-2,-1,1,2]) + freq_dc
1049 1049
1050 1050 if ind_vel[0]<0:
1051 1051 ind_vel[range(0,1)] = ind_vel[range(0,1)] + self.num_prof
1052 1052
1053 1053 if mode == 1:
1054 1054 jspectra[:,freq_dc,:] = (jspectra[:,ind_vel[1],:] + jspectra[:,ind_vel[2],:])/2 #CORRECCION
1055 1055
1056 1056 if mode == 2:
1057 1057
1058 1058 vel = numpy.array([-2,-1,1,2])
1059 1059 xx = numpy.zeros([4,4])
1060 1060
1061 1061 for fil in range(4):
1062 1062 xx[fil,:] = vel[fil]**numpy.asarray(range(4))
1063 1063
1064 1064 xx_inv = numpy.linalg.inv(xx)
1065 1065 xx_aux = xx_inv[0,:]
1066 1066
1067 1067 for ich in range(num_chan):
1068 1068 yy = jspectra[ich,ind_vel,:]
1069 1069 jspectra[ich,freq_dc,:] = numpy.dot(xx_aux,yy)
1070 1070
1071 1071 junkid = jspectra[ich,freq_dc,:]<=0
1072 1072 cjunkid = sum(junkid)
1073 1073
1074 1074 if cjunkid.any():
1075 1075 jspectra[ich,freq_dc,junkid.nonzero()] = (jspectra[ich,ind_vel[1],junkid] + jspectra[ich,ind_vel[2],junkid])/2
1076 1076
1077 1077 noise = jspectra0[:,freq_dc,:] - jspectra[:,freq_dc,:]
1078 1078
1079 1079 return noise
1080 1080
1081 1081 def getTimeInterval(self):
1082 1082
1083 1083 timeInterval = self.ippSeconds * self.nCohInt * self.nProfiles
1084 1084
1085 1085 return timeInterval
1086 1086
1087 1087 def splitFunctions(self):
1088 1088
1089 1089 pairsList = self.pairsList
1090 1090 ccf_pairs = []
1091 1091 acf_pairs = []
1092 1092 ccf_ind = []
1093 1093 acf_ind = []
1094 1094 for l in range(len(pairsList)):
1095 1095 chan0 = pairsList[l][0]
1096 1096 chan1 = pairsList[l][1]
1097 1097
1098 1098 #Obteniendo pares de Autocorrelacion
1099 1099 if chan0 == chan1:
1100 1100 acf_pairs.append(chan0)
1101 1101 acf_ind.append(l)
1102 1102 else:
1103 1103 ccf_pairs.append(pairsList[l])
1104 1104 ccf_ind.append(l)
1105 1105
1106 1106 data_acf = self.data_cf[acf_ind]
1107 1107 data_ccf = self.data_cf[ccf_ind]
1108 1108
1109 1109 return acf_ind, ccf_ind, acf_pairs, ccf_pairs, data_acf, data_ccf
1110 1110
1111 1111 def getNormFactor(self):
1112 1112 acf_ind, ccf_ind, acf_pairs, ccf_pairs, data_acf, data_ccf = self.splitFunctions()
1113 1113 acf_pairs = numpy.array(acf_pairs)
1114 1114 normFactor = numpy.zeros((self.nPairs,self.nHeights))
1115 1115
1116 1116 for p in range(self.nPairs):
1117 1117 pair = self.pairsList[p]
1118 1118
1119 1119 ch0 = pair[0]
1120 1120 ch1 = pair[1]
1121 1121
1122 1122 ch0_max = numpy.max(data_acf[acf_pairs==ch0,:,:], axis=1)
1123 1123 ch1_max = numpy.max(data_acf[acf_pairs==ch1,:,:], axis=1)
1124 1124 normFactor[p,:] = numpy.sqrt(ch0_max*ch1_max)
1125 1125
1126 1126 return normFactor
1127 1127
1128 1128 timeInterval = property(getTimeInterval, "I'm the 'timeInterval' property")
1129 1129 normFactor = property(getNormFactor, "I'm the 'normFactor property'")
1130 1130
1131 1131 class Parameters(Spectra):
1132 1132
1133 1133 experimentInfo = None #Information about the experiment
1134 1134
1135 1135 #Information from previous data
1136 1136
1137 1137 inputUnit = None #Type of data to be processed
1138 1138
1139 1139 operation = None #Type of operation to parametrize
1140 1140
1141 1141 #normFactor = None #Normalization Factor
1142 1142
1143 1143 groupList = None #List of Pairs, Groups, etc
1144 1144
1145 1145 #Parameters
1146 1146
1147 1147 data_param = None #Parameters obtained
1148 1148
1149 1149 data_pre = None #Data Pre Parametrization
1150 1150
1151 1151 data_SNR = None #Signal to Noise Ratio
1152 1152
1153 1153 # heightRange = None #Heights
1154 1154
1155 1155 abscissaList = None #Abscissa, can be velocities, lags or time
1156 1156
1157 1157 # noise = None #Noise Potency
1158 1158
1159 1159 utctimeInit = None #Initial UTC time
1160 1160
1161 1161 paramInterval = None #Time interval to calculate Parameters in seconds
1162 1162
1163 1163 useLocalTime = True
1164 1164
1165 1165 #Fitting
1166 1166
1167 1167 data_error = None #Error of the estimation
1168 1168
1169 1169 constants = None
1170 1170
1171 1171 library = None
1172 1172
1173 1173 #Output signal
1174 1174
1175 1175 outputInterval = None #Time interval to calculate output signal in seconds
1176 1176
1177 1177 data_output = None #Out signal
1178 1178
1179 1179 nAvg = None
1180 1180
1181 1181 noise_estimation = None
1182 1182
1183 1183 GauSPC = None #Fit gaussian SPC
1184 1184
1185 1185
1186 1186 def __init__(self):
1187 1187 '''
1188 1188 Constructor
1189 1189 '''
1190 1190 self.radarControllerHeaderObj = RadarControllerHeader()
1191 1191
1192 1192 self.systemHeaderObj = SystemHeader()
1193 1193
1194 1194 self.type = "Parameters"
1195 1195
1196 1196 def getTimeRange1(self, interval):
1197 1197
1198 1198 datatime = []
1199 1199
1200 1200 if self.useLocalTime:
1201 1201 time1 = self.utctimeInit - self.timeZone*60
1202 1202 else:
1203 1203 time1 = self.utctimeInit
1204 1204
1205 1205 datatime.append(time1)
1206 1206 datatime.append(time1 + interval)
1207 1207 datatime = numpy.array(datatime)
1208 1208
1209 1209 return datatime
1210 1210
1211 1211 def getTimeInterval(self):
1212 1212
1213 1213 if hasattr(self, 'timeInterval1'):
1214 1214 return self.timeInterval1
1215 1215 else:
1216 1216 return self.paramInterval
1217 1217
1218 1218 def setValue(self, value):
1219 1219
1220 1220 print "This property should not be initialized"
1221 1221
1222 1222 return
1223 1223
1224 1224 def getNoise(self):
1225 1225
1226 1226 return self.spc_noise
1227 1227
1228 1228 timeInterval = property(getTimeInterval)
1229 1229 noise = property(getNoise, setValue, "I'm the 'Noise' property.")
Show file before | Show file after | Hide comments
@@ -1,657 +1,657
1 1 import os
2 2 import numpy
3 3 import time, datetime
4 4 import mpldriver
5 5
6 6 from schainpy.model.proc.jroproc_base import Operation
7 7
8 8 def isTimeInHourRange(datatime, xmin, xmax):
9 9
10 10 if xmin == None or xmax == None:
11 11 return 1
12 12 hour = datatime.hour + datatime.minute/60.0
13 13
14 14 if xmin < (xmax % 24):
15 15
16 16 if hour >= xmin and hour <= xmax:
17 17 return 1
18 18 else:
19 19 return 0
20 20
21 21 else:
22 22
23 23 if hour >= xmin or hour <= (xmax % 24):
24 24 return 1
25 25 else:
26 26 return 0
27 27
28 28 return 0
29 29
30 30 def isRealtime(utcdatatime):
31 31
32 32 utcnow = time.mktime(time.localtime())
33 33 delta = abs(utcnow - utcdatatime) # abs
34 34 if delta >= 30.:
35 35 return False
36 36 return True
37 37
38 38 class Figure(Operation):
39 39
40 40 __driver = mpldriver
41 41 fig = None
42 42
43 43 id = None
44 44 wintitle = None
45 45 width = None
46 46 height = None
47 47 nplots = None
48 48 timerange = None
49 49
50 50 axesObjList = []
51 51
52 52 WIDTH = 300
53 53 HEIGHT = 200
54 54 PREFIX = 'fig'
55 55
56 56 xmin = None
57 57 xmax = None
58 58
59 59 counter_imagwr = 0
60 60
61 61 figfile = None
62 62
63 63 created = False
64
64 parameters = {}
65 65 def __init__(self, **kwargs):
66 66
67 67 Operation.__init__(self, **kwargs)
68 68
69 69 def __del__(self):
70 70
71 71 self.__driver.closeFigure()
72 72
73 73 def getFilename(self, name, ext='.png'):
74 74
75 75 path = '%s%03d' %(self.PREFIX, self.id)
76 76 filename = '%s_%s%s' %(self.PREFIX, name, ext)
77 77 return os.path.join(path, filename)
78 78
79 79 def getAxesObjList(self):
80 80
81 81 return self.axesObjList
82 82
83 83 def getSubplots(self):
84 84
85 85 raise NotImplementedError
86 86
87 87 def getScreenDim(self, widthplot, heightplot):
88 88
89 89 nrow, ncol = self.getSubplots()
90 90
91 91 widthscreen = widthplot*ncol
92 92 heightscreen = heightplot*nrow
93 93
94 94 return widthscreen, heightscreen
95 95
96 96 def getTimeLim(self, x, xmin=None, xmax=None, timerange=None):
97 97
98 98 # if self.xmin != None and self.xmax != None:
99 99 # if timerange == None:
100 100 # timerange = self.xmax - self.xmin
101 101 # xmin = self.xmin + timerange
102 102 # xmax = self.xmax + timerange
103 103 #
104 104 # return xmin, xmax
105 105
106 106 if timerange == None and (xmin==None or xmax==None):
107 107 timerange = 14400 #seconds
108 108
109 109 if timerange != None:
110 110 txmin = x[0] #- x[0] % min(timerange/10, 10*60)
111 111 else:
112 112 txmin = x[0] #- x[0] % 10*60
113 113
114 114 thisdatetime = datetime.datetime.utcfromtimestamp(txmin)
115 115 thisdate = datetime.datetime.combine(thisdatetime.date(), datetime.time(0,0,0))
116 116
117 117 if timerange != None:
118 118 xmin = (thisdatetime - thisdate).seconds/(60*60.)
119 119 xmax = xmin + timerange/(60*60.)
120 120
121 121 d1970 = datetime.datetime(1970,1,1)
122 122
123 123 mindt = thisdate + datetime.timedelta(hours=xmin) #- datetime.timedelta(seconds=time.timezone)
124 124 xmin_sec = (mindt - d1970).total_seconds() #time.mktime(mindt.timetuple()) - time.timezone
125 125
126 126 maxdt = thisdate + datetime.timedelta(hours=xmax) #- datetime.timedelta(seconds=time.timezone)
127 127 xmax_sec = (maxdt - d1970).total_seconds() #time.mktime(maxdt.timetuple()) - time.timezone
128 128
129 129 return xmin_sec, xmax_sec
130 130
131 131 def init(self, id, nplots, wintitle):
132 132
133 133 raise NotImplementedError, "This method has been replaced by createFigure"
134 134
135 135 def createFigure(self, id, wintitle, widthplot=None, heightplot=None, show=True):
136 136
137 137 """
138 138 Crea la figura de acuerdo al driver y parametros seleccionados seleccionados.
139 139 Las dimensiones de la pantalla es calculada a partir de los atributos self.WIDTH
140 140 y self.HEIGHT y el numero de subplots (nrow, ncol)
141 141
142 142 Input:
143 143 id : Los parametros necesarios son
144 144 wintitle :
145 145
146 146 """
147 147
148 148 if widthplot == None:
149 149 widthplot = self.WIDTH
150 150
151 151 if heightplot == None:
152 152 heightplot = self.HEIGHT
153 153
154 154 self.id = id
155 155
156 156 self.wintitle = wintitle
157 157
158 158 self.widthscreen, self.heightscreen = self.getScreenDim(widthplot, heightplot)
159 159
160 160 # if self.created:
161 161 # self.__driver.closeFigure(self.fig)
162 162
163 163 if not self.created:
164 164 self.fig = self.__driver.createFigure(id=self.id,
165 165 wintitle=self.wintitle,
166 166 width=self.widthscreen,
167 167 height=self.heightscreen,
168 168 show=show)
169 169 else:
170 170 self.__driver.clearFigure(self.fig)
171 171
172 172 self.axesObjList = []
173 173 self.counter_imagwr = 0
174 174
175 175 self.created = True
176 176
177 177 def setDriver(self, driver=mpldriver):
178 178
179 179 self.__driver = driver
180 180
181 181 def setTitle(self, title):
182 182
183 183 self.__driver.setTitle(self.fig, title)
184 184
185 185 def setWinTitle(self, title):
186 186
187 187 self.__driver.setWinTitle(self.fig, title=title)
188 188
189 189 def setTextFromAxes(self, text):
190 190
191 191 raise NotImplementedError, "This method has been replaced with Axes.setText"
192 192
193 193 def makeAxes(self, nrow, ncol, xpos, ypos, colspan, rowspan):
194 194
195 195 raise NotImplementedError, "This method has been replaced with Axes.addAxes"
196 196
197 197 def addAxes(self, *args):
198 198 """
199 199
200 200 Input:
201 201 *args : Los parametros necesarios son
202 202 nrow, ncol, xpos, ypos, colspan, rowspan
203 203 """
204 204
205 205 axesObj = Axes(self.fig, *args)
206 206 self.axesObjList.append(axesObj)
207 207
208 208 def saveFigure(self, figpath, figfile, *args):
209 209
210 210 filename = os.path.join(figpath, figfile)
211 211
212 212 fullpath = os.path.split(filename)[0]
213 213
214 214 if not os.path.exists(fullpath):
215 215 subpath = os.path.split(fullpath)[0]
216 216
217 217 if not os.path.exists(subpath):
218 218 os.mkdir(subpath)
219 219
220 220 os.mkdir(fullpath)
221 221
222 222 self.__driver.saveFigure(self.fig, filename, *args)
223 223
224 224 def save(self, figpath, figfile=None, save=True, ftp=False, wr_period=1, thisDatetime=None, update_figfile=True):
225 225
226 226 self.counter_imagwr += 1
227 227 if self.counter_imagwr < wr_period:
228 228 return
229 229
230 230 self.counter_imagwr = 0
231 231
232 232 if save:
233 233
234 234 if not figfile:
235 235
236 236 if not thisDatetime:
237 237 raise ValueError, "Saving figure: figfile or thisDatetime should be defined"
238 238 return
239 239
240 240 str_datetime = thisDatetime.strftime("%Y%m%d_%H%M%S")
241 241 figfile = self.getFilename(name = str_datetime)
242 242
243 243 if self.figfile == None:
244 244 self.figfile = figfile
245 245
246 246 if update_figfile:
247 247 self.figfile = figfile
248 248
249 249 # store png plot to local folder
250 250 self.saveFigure(figpath, self.figfile)
251 251
252 252
253 253 if not ftp:
254 254 return
255 255
256 256 if not thisDatetime:
257 257 return
258 258
259 259 # store png plot to FTP server according to RT-Web format
260 260 ftp_filename = self.getNameToFtp(thisDatetime, self.FTP_WEI, self.EXP_CODE, self.SUB_EXP_CODE, self.PLOT_CODE, self.PLOT_POS)
261 261 # ftp_filename = os.path.join(figpath, name)
262 262 self.saveFigure(figpath, ftp_filename)
263 263
264 264 def getNameToFtp(self, thisDatetime, FTP_WEI, EXP_CODE, SUB_EXP_CODE, PLOT_CODE, PLOT_POS):
265 265 YEAR_STR = '%4.4d'%thisDatetime.timetuple().tm_year
266 266 DOY_STR = '%3.3d'%thisDatetime.timetuple().tm_yday
267 267 FTP_WEI = '%2.2d'%FTP_WEI
268 268 EXP_CODE = '%3.3d'%EXP_CODE
269 269 SUB_EXP_CODE = '%2.2d'%SUB_EXP_CODE
270 270 PLOT_CODE = '%2.2d'%PLOT_CODE
271 271 PLOT_POS = '%2.2d'%PLOT_POS
272 272 name = YEAR_STR + DOY_STR + FTP_WEI + EXP_CODE + SUB_EXP_CODE + PLOT_CODE + PLOT_POS
273 273 return name
274 274
275 275 def draw(self):
276 276
277 277 self.__driver.draw(self.fig)
278 278
279 279 def run(self):
280 280
281 281 raise NotImplementedError
282 282
283 283 def close(self, show=False):
284 284
285 285 self.__driver.closeFigure(show=show, fig=self.fig)
286 286
287 287 axesList = property(getAxesObjList)
288 288
289 289
290 290 class Axes:
291 291
292 292 __driver = mpldriver
293 293 fig = None
294 294 ax = None
295 295 plot = None
296 296 __missing = 1E30
297 297 __firsttime = None
298 298
299 299 __showprofile = False
300 300
301 301 xmin = None
302 302 xmax = None
303 303 ymin = None
304 304 ymax = None
305 305 zmin = None
306 306 zmax = None
307 307
308 308 x_buffer = None
309 309 z_buffer = None
310 310
311 311 decimationx = None
312 312 decimationy = None
313 313
314 314 __MAXNUMX = 200
315 315 __MAXNUMY = 400
316 316
317 317 __MAXNUMTIME = 500
318 318
319 319 def __init__(self, *args):
320 320
321 321 """
322 322
323 323 Input:
324 324 *args : Los parametros necesarios son
325 325 fig, nrow, ncol, xpos, ypos, colspan, rowspan
326 326 """
327 327
328 328 ax = self.__driver.createAxes(*args)
329 329 self.fig = args[0]
330 330 self.ax = ax
331 331 self.plot = None
332 332
333 333 self.__firsttime = True
334 334 self.idlineList = []
335 335
336 336 self.x_buffer = numpy.array([])
337 337 self.z_buffer = numpy.array([])
338 338
339 339 def setText(self, text):
340 340
341 341 self.__driver.setAxesText(self.ax, text)
342 342
343 343 def setXAxisAsTime(self):
344 344 pass
345 345
346 346 def pline(self, x, y,
347 347 xmin=None, xmax=None,
348 348 ymin=None, ymax=None,
349 349 xlabel='', ylabel='',
350 350 title='',
351 351 **kwargs):
352 352
353 353 """
354 354
355 355 Input:
356 356 x :
357 357 y :
358 358 xmin :
359 359 xmax :
360 360 ymin :
361 361 ymax :
362 362 xlabel :
363 363 ylabel :
364 364 title :
365 365 **kwargs : Los parametros aceptados son
366 366
367 367 ticksize
368 368 ytick_visible
369 369 """
370 370
371 371 if self.__firsttime:
372 372
373 373 if xmin == None: xmin = numpy.nanmin(x)
374 374 if xmax == None: xmax = numpy.nanmax(x)
375 375 if ymin == None: ymin = numpy.nanmin(y)
376 376 if ymax == None: ymax = numpy.nanmax(y)
377 377
378 378 self.plot = self.__driver.createPline(self.ax, x, y,
379 379 xmin, xmax,
380 380 ymin, ymax,
381 381 xlabel=xlabel,
382 382 ylabel=ylabel,
383 383 title=title,
384 384 **kwargs)
385 385
386 386 self.idlineList.append(0)
387 387 self.__firsttime = False
388 388 return
389 389
390 390 self.__driver.pline(self.plot, x, y, xlabel=xlabel,
391 391 ylabel=ylabel,
392 392 title=title)
393 393
394 394 # self.__driver.pause()
395 395
396 396 def addpline(self, x, y, idline, **kwargs):
397 397 lines = self.ax.lines
398 398
399 399 if idline in self.idlineList:
400 400 self.__driver.set_linedata(self.ax, x, y, idline)
401 401
402 402 if idline not in(self.idlineList):
403 403 self.__driver.addpline(self.ax, x, y, **kwargs)
404 404 self.idlineList.append(idline)
405 405
406 406 return
407 407
408 408 def pmultiline(self, x, y,
409 409 xmin=None, xmax=None,
410 410 ymin=None, ymax=None,
411 411 xlabel='', ylabel='',
412 412 title='',
413 413 **kwargs):
414 414
415 415 if self.__firsttime:
416 416
417 417 if xmin == None: xmin = numpy.nanmin(x)
418 418 if xmax == None: xmax = numpy.nanmax(x)
419 419 if ymin == None: ymin = numpy.nanmin(y)
420 420 if ymax == None: ymax = numpy.nanmax(y)
421 421
422 422 self.plot = self.__driver.createPmultiline(self.ax, x, y,
423 423 xmin, xmax,
424 424 ymin, ymax,
425 425 xlabel=xlabel,
426 426 ylabel=ylabel,
427 427 title=title,
428 428 **kwargs)
429 429 self.__firsttime = False
430 430 return
431 431
432 432 self.__driver.pmultiline(self.plot, x, y, xlabel=xlabel,
433 433 ylabel=ylabel,
434 434 title=title)
435 435
436 436 # self.__driver.pause()
437 437
438 438 def pmultilineyaxis(self, x, y,
439 439 xmin=None, xmax=None,
440 440 ymin=None, ymax=None,
441 441 xlabel='', ylabel='',
442 442 title='',
443 443 **kwargs):
444 444
445 445 if self.__firsttime:
446 446
447 447 if xmin == None: xmin = numpy.nanmin(x)
448 448 if xmax == None: xmax = numpy.nanmax(x)
449 449 if ymin == None: ymin = numpy.nanmin(y)
450 450 if ymax == None: ymax = numpy.nanmax(y)
451 451
452 452 self.plot = self.__driver.createPmultilineYAxis(self.ax, x, y,
453 453 xmin, xmax,
454 454 ymin, ymax,
455 455 xlabel=xlabel,
456 456 ylabel=ylabel,
457 457 title=title,
458 458 **kwargs)
459 459 if self.xmin == None: self.xmin = xmin
460 460 if self.xmax == None: self.xmax = xmax
461 461 if self.ymin == None: self.ymin = ymin
462 462 if self.ymax == None: self.ymax = ymax
463 463
464 464 self.__firsttime = False
465 465 return
466 466
467 467 self.__driver.pmultilineyaxis(self.plot, x, y, xlabel=xlabel,
468 468 ylabel=ylabel,
469 469 title=title)
470 470
471 471 # self.__driver.pause()
472 472
473 473 def pcolor(self, x, y, z,
474 474 xmin=None, xmax=None,
475 475 ymin=None, ymax=None,
476 476 zmin=None, zmax=None,
477 477 xlabel='', ylabel='',
478 478 title='', colormap='jet',
479 479 **kwargs):
480 480
481 481 """
482 482 Input:
483 483 x :
484 484 y :
485 485 x :
486 486 xmin :
487 487 xmax :
488 488 ymin :
489 489 ymax :
490 490 zmin :
491 491 zmax :
492 492 xlabel :
493 493 ylabel :
494 494 title :
495 495 **kwargs : Los parametros aceptados son
496 496 ticksize=9,
497 497 cblabel=''
498 498 """
499 499
500 500 #Decimating data
501 501 xlen = len(x)
502 502 ylen = len(y)
503 503
504 504 decimationx = int(xlen/self.__MAXNUMX) + 1
505 505 decimationy = int(ylen/self.__MAXNUMY) + 1
506 506
507 507
508 508 x_buffer = x#[::decimationx]
509 509 y_buffer = y#[::decimationy]
510 510 z_buffer = z#[::decimationx, ::decimationy]
511 511 #===================================================
512 512
513 513 if self.__firsttime:
514 514
515 515 if xmin == None: xmin = numpy.nanmin(x)
516 516 if xmax == None: xmax = numpy.nanmax(x)
517 517 if ymin == None: ymin = numpy.nanmin(y)
518 518 if ymax == None: ymax = numpy.nanmax(y)
519 519 if zmin == None: zmin = numpy.nanmin(z)
520 520 if zmax == None: zmax = numpy.nanmax(z)
521 521
522 522
523 523 self.plot = self.__driver.createPcolor(self.ax, x_buffer,
524 524 y_buffer,
525 525 z_buffer,
526 526 xmin, xmax,
527 527 ymin, ymax,
528 528 zmin, zmax,
529 529 xlabel=xlabel,
530 530 ylabel=ylabel,
531 531 title=title,
532 532 colormap=colormap,
533 533 **kwargs)
534 534
535 535 if self.xmin == None: self.xmin = xmin
536 536 if self.xmax == None: self.xmax = xmax
537 537 if self.ymin == None: self.ymin = ymin
538 538 if self.ymax == None: self.ymax = ymax
539 539 if self.zmin == None: self.zmin = zmin
540 540 if self.zmax == None: self.zmax = zmax
541 541
542 542 self.__firsttime = False
543 543 return
544 544
545 545 self.__driver.pcolor(self.plot,
546 546 z_buffer,
547 547 xlabel=xlabel,
548 548 ylabel=ylabel,
549 549 title=title)
550 550
551 551 # self.__driver.pause()
552 552
553 553 def pcolorbuffer(self, x, y, z,
554 554 xmin=None, xmax=None,
555 555 ymin=None, ymax=None,
556 556 zmin=None, zmax=None,
557 557 xlabel='', ylabel='',
558 558 title='', rti = True, colormap='jet',
559 559 maxNumX = None, maxNumY = None,
560 560 **kwargs):
561 561
562 562 if maxNumX == None:
563 563 maxNumX = self.__MAXNUMTIME
564 564
565 565 if maxNumY == None:
566 566 maxNumY = self.__MAXNUMY
567 567
568 568 if self.__firsttime:
569 569 self.z_buffer = z
570 570 self.x_buffer = numpy.hstack((self.x_buffer, x))
571 571
572 572 if xmin == None: xmin = numpy.nanmin(x)
573 573 if xmax == None: xmax = numpy.nanmax(x)
574 574 if ymin == None: ymin = numpy.nanmin(y)
575 575 if ymax == None: ymax = numpy.nanmax(y)
576 576 if zmin == None: zmin = numpy.nanmin(z)
577 577 if zmax == None: zmax = numpy.nanmax(z)
578 578
579 579 self.plot = self.__driver.createPcolor(self.ax, self.x_buffer, y, z,
580 580 xmin, xmax,
581 581 ymin, ymax,
582 582 zmin, zmax,
583 583 xlabel=xlabel,
584 584 ylabel=ylabel,
585 585 title=title,
586 586 colormap=colormap,
587 587 **kwargs)
588 588
589 589 if self.xmin == None: self.xmin = xmin
590 590 if self.xmax == None: self.xmax = xmax
591 591 if self.ymin == None: self.ymin = ymin
592 592 if self.ymax == None: self.ymax = ymax
593 593 if self.zmin == None: self.zmin = zmin
594 594 if self.zmax == None: self.zmax = zmax
595 595
596 596 self.__firsttime = False
597 597 return
598 598
599 599 self.x_buffer = numpy.hstack((self.x_buffer[:-1], x[0], x[-1]))
600 600 self.z_buffer = numpy.hstack((self.z_buffer, z))
601 601 z_buffer = self.z_buffer.reshape(-1,len(y))
602 602
603 603 #Decimating data
604 604 xlen = len(self.x_buffer)
605 605 ylen = len(y)
606 606
607 607 decimationx = int(xlen/maxNumX) + 1
608 608 decimationy = int(ylen/maxNumY) + 1
609 609
610 610 x_buffer = self.x_buffer#[::decimationx]
611 611 y_buffer = y#[::decimationy]
612 612 z_buffer = z_buffer#[::decimationx, ::decimationy]
613 613 #===================================================
614 614
615 615 x_buffer, y_buffer, z_buffer = self.__fillGaps(x_buffer, y_buffer, z_buffer)
616 616
617 617 self.__driver.addpcolorbuffer(self.ax, x_buffer, y_buffer, z_buffer, self.zmin, self.zmax,
618 618 xlabel=xlabel,
619 619 ylabel=ylabel,
620 620 title=title,
621 621 colormap=colormap)
622 622
623 623 # self.__driver.pause()
624 624
625 625 def polar(self, x, y,
626 626 title='', xlabel='',ylabel='',**kwargs):
627 627
628 628 if self.__firsttime:
629 629 self.plot = self.__driver.createPolar(self.ax, x, y, title = title, xlabel = xlabel, ylabel = ylabel)
630 630 self.__firsttime = False
631 631 self.x_buffer = x
632 632 self.y_buffer = y
633 633 return
634 634
635 635 self.x_buffer = numpy.hstack((self.x_buffer,x))
636 636 self.y_buffer = numpy.hstack((self.y_buffer,y))
637 637 self.__driver.polar(self.plot, self.x_buffer, self.y_buffer, xlabel=xlabel,
638 638 ylabel=ylabel,
639 639 title=title)
640 640
641 641 # self.__driver.pause()
642 642
643 643 def __fillGaps(self, x_buffer, y_buffer, z_buffer):
644 644
645 645 if x_buffer.shape[0] < 2:
646 646 return x_buffer, y_buffer, z_buffer
647 647
648 648 deltas = x_buffer[1:] - x_buffer[0:-1]
649 649 x_median = numpy.median(deltas)
650 650
651 651 index = numpy.where(deltas > 5*x_median)
652 652
653 653 if len(index[0]) != 0:
654 654 z_buffer[index[0],::] = self.__missing
655 655 z_buffer = numpy.ma.masked_inside(z_buffer,0.99*self.__missing,1.01*self.__missing)
656 656
657 657 return x_buffer, y_buffer, z_buffer
Show file before | Show file after | Hide comments
@@ -1,188 +1,187
1 1 import os
2 2 import datetime
3 3 import numpy
4 4 import copy
5
5 from schainpy.model import *
6 6 from figure import Figure, isRealtime
7 7
8 8 class CorrelationPlot(Figure):
9
10 9 isConfig = None
11 10 __nsubplots = None
12 11
13 12 WIDTHPROF = None
14 13 HEIGHTPROF = None
15 14 PREFIX = 'corr'
16 15
17 16 def __init__(self):
18 17
19 18 self.isConfig = False
20 19 self.__nsubplots = 1
21 20
22 21 self.WIDTH = 280
23 22 self.HEIGHT = 250
24 23 self.WIDTHPROF = 120
25 24 self.HEIGHTPROF = 0
26 25 self.counter_imagwr = 0
27 26
28 27 self.PLOT_CODE = 1
29 28 self.FTP_WEI = None
30 29 self.EXP_CODE = None
31 30 self.SUB_EXP_CODE = None
32 31 self.PLOT_POS = None
33 32
34 33 def getSubplots(self):
35 34
36 35 ncol = int(numpy.sqrt(self.nplots)+0.9)
37 36 nrow = int(self.nplots*1./ncol + 0.9)
38 37
39 38 return nrow, ncol
40 39
41 40 def setup(self, id, nplots, wintitle, showprofile=False, show=True):
42 41
43 42 showprofile = False
44 43 self.__showprofile = showprofile
45 44 self.nplots = nplots
46 45
47 46 ncolspan = 1
48 47 colspan = 1
49 48 if showprofile:
50 49 ncolspan = 3
51 50 colspan = 2
52 51 self.__nsubplots = 2
53 52
54 53 self.createFigure(id = id,
55 54 wintitle = wintitle,
56 55 widthplot = self.WIDTH + self.WIDTHPROF,
57 56 heightplot = self.HEIGHT + self.HEIGHTPROF,
58 57 show=show)
59 58
60 59 nrow, ncol = self.getSubplots()
61 60
62 61 counter = 0
63 62 for y in range(nrow):
64 63 for x in range(ncol):
65 64
66 65 if counter >= self.nplots:
67 66 break
68 67
69 68 self.addAxes(nrow, ncol*ncolspan, y, x*ncolspan, colspan, 1)
70 69
71 70 if showprofile:
72 71 self.addAxes(nrow, ncol*ncolspan, y, x*ncolspan+colspan, 1, 1)
73 72
74 73 counter += 1
75 74
76 75 def run(self, dataOut, id, wintitle="", channelList=None, showprofile=False,
77 76 xmin=None, xmax=None, ymin=None, ymax=None, zmin=None, zmax=None,
78 77 save=False, figpath='./', figfile=None, show=True, ftp=False, wr_period=1,
79 78 server=None, folder=None, username=None, password=None,
80 79 ftp_wei=0, exp_code=0, sub_exp_code=0, plot_pos=0, realtime=False):
81 80
82 81 """
83 82
84 83 Input:
85 84 dataOut :
86 85 id :
87 86 wintitle :
88 87 channelList :
89 88 showProfile :
90 89 xmin : None,
91 90 xmax : None,
92 91 ymin : None,
93 92 ymax : None,
94 93 zmin : None,
95 94 zmax : None
96 95 """
97 96
98 97 if dataOut.flagNoData:
99 98 return None
100 99
101 100 if realtime:
102 101 if not(isRealtime(utcdatatime = dataOut.utctime)):
103 102 print 'Skipping this plot function'
104 103 return
105 104
106 105 if channelList == None:
107 106 channelIndexList = dataOut.channelIndexList
108 107 else:
109 108 channelIndexList = []
110 109 for channel in channelList:
111 110 if channel not in dataOut.channelList:
112 111 raise ValueError, "Channel %d is not in dataOut.channelList"
113 112 channelIndexList.append(dataOut.channelList.index(channel))
114 113
115 114 factor = dataOut.normFactor
116 115 lenfactor = factor.shape[1]
117 116 x = dataOut.getLagTRange(1)
118 117 y = dataOut.getHeiRange()
119 118
120 119 z = copy.copy(dataOut.data_corr[:,:,0,:])
121 120 for i in range(dataOut.data_corr.shape[0]):
122 121 z[i,:,:] = z[i,:,:]/factor[i,:]
123 122 zdB = numpy.abs(z)
124 123
125 124 avg = numpy.average(z, axis=1)
126 125 # avg = numpy.nanmean(z, axis=1)
127 126 # noise = dataOut.noise/factor
128 127
129 128 #thisDatetime = dataOut.datatime
130 129 thisDatetime = datetime.datetime.utcfromtimestamp(dataOut.getTimeRange()[0])
131 130 title = wintitle + " Correlation"
132 131 xlabel = "Lag T (s)"
133 132 ylabel = "Range (Km)"
134 133
135 134 if not self.isConfig:
136 135
137 136 nplots = dataOut.data_corr.shape[0]
138 137
139 138 self.setup(id=id,
140 139 nplots=nplots,
141 140 wintitle=wintitle,
142 141 showprofile=showprofile,
143 142 show=show)
144 143
145 144 if xmin == None: xmin = numpy.nanmin(x)
146 145 if xmax == None: xmax = numpy.nanmax(x)
147 146 if ymin == None: ymin = numpy.nanmin(y)
148 147 if ymax == None: ymax = numpy.nanmax(y)
149 148 if zmin == None: zmin = 0
150 149 if zmax == None: zmax = 1
151 150
152 151 self.FTP_WEI = ftp_wei
153 152 self.EXP_CODE = exp_code
154 153 self.SUB_EXP_CODE = sub_exp_code
155 154 self.PLOT_POS = plot_pos
156 155
157 156 self.isConfig = True
158 157
159 158 self.setWinTitle(title)
160 159
161 160 for i in range(self.nplots):
162 161 str_datetime = '%s %s'%(thisDatetime.strftime("%Y/%m/%d"),thisDatetime.strftime("%H:%M:%S"))
163 162 title = "Channel %d and %d: : %s" %(dataOut.pairsList[i][0],dataOut.pairsList[i][1] , str_datetime)
164 163 axes = self.axesList[i*self.__nsubplots]
165 164 axes.pcolor(x, y, zdB[i,:,:],
166 165 xmin=xmin, xmax=xmax, ymin=ymin, ymax=ymax, zmin=zmin, zmax=zmax,
167 166 xlabel=xlabel, ylabel=ylabel, title=title,
168 167 ticksize=9, cblabel='')
169 168
170 169 # if self.__showprofile:
171 170 # axes = self.axesList[i*self.__nsubplots +1]
172 171 # axes.pline(avgdB[i], y,
173 172 # xmin=zmin, xmax=zmax, ymin=ymin, ymax=ymax,
174 173 # xlabel='dB', ylabel='', title='',
175 174 # ytick_visible=False,
176 175 # grid='x')
177 176 #
178 177 # noiseline = numpy.repeat(noisedB[i], len(y))
179 178 # axes.addpline(noiseline, y, idline=1, color="black", linestyle="dashed", lw=2)
180 179
181 180 self.draw()
182 181
183 182 self.save(figpath=figpath,
184 183 figfile=figfile,
185 184 save=save,
186 185 ftp=ftp,
187 186 wr_period=wr_period,
188 187 thisDatetime=thisDatetime)
Show file before | Show file after | Hide comments
This diff has been collapsed as it changes many lines, (1201 lines changed) Show them Hide them
@@ -1,955 +1,782
1 1
2 2 import os
3 import zmq
4 3 import time
5 import numpy
4 import glob
6 5 import datetime
7 import numpy as np
6 from multiprocessing import Process
7
8 import zmq
9 import numpy
8 10 import matplotlib
9 import glob
10 matplotlib.use('TkAgg')
11 11 import matplotlib.pyplot as plt
12 12 from mpl_toolkits.axes_grid1 import make_axes_locatable
13 from matplotlib.ticker import FuncFormatter, LinearLocator
14 from multiprocessing import Process
13 from matplotlib.ticker import FuncFormatter, LinearLocator, MultipleLocator
15 14
16 15 from schainpy.model.proc.jroproc_base import Operation
17
18 plt.ion()
16 from schainpy.utils import log
19 17
20 18 func = lambda x, pos: ('%s') %(datetime.datetime.fromtimestamp(x).strftime('%H:%M'))
21 19
22 d1970 = datetime.datetime(1970,1,1)
20 d1970 = datetime.datetime(1970, 1, 1)
21
23 22
24 23 class PlotData(Operation, Process):
24 '''
25 Base class for Schain plotting operations
26 '''
25 27
26 28 CODE = 'Figure'
27 29 colormap = 'jro'
30 bgcolor = 'white'
28 31 CONFLATE = False
29 32 __MAXNUMX = 80
30 33 __missing = 1E30
31 34
32 35 def __init__(self, **kwargs):
33 36
34 37 Operation.__init__(self, plot=True, **kwargs)
35 38 Process.__init__(self)
36 39 self.kwargs['code'] = self.CODE
37 40 self.mp = False
38 self.dataOut = None
39 self.isConfig = False
40 self.figure = None
41 self.data = None
42 self.isConfig = False
43 self.figures = []
41 44 self.axes = []
45 self.cb_axes = []
42 46 self.localtime = kwargs.pop('localtime', True)
43 47 self.show = kwargs.get('show', True)
44 48 self.save = kwargs.get('save', False)
45 49 self.colormap = kwargs.get('colormap', self.colormap)
46 50 self.colormap_coh = kwargs.get('colormap_coh', 'jet')
47 51 self.colormap_phase = kwargs.get('colormap_phase', 'RdBu_r')
48 self.showprofile = kwargs.get('showprofile', True)
49 self.title = kwargs.get('wintitle', '')
52 self.colormaps = kwargs.get('colormaps', None)
53 self.bgcolor = kwargs.get('bgcolor', self.bgcolor)
54 self.showprofile = kwargs.get('showprofile', False)
55 self.title = kwargs.get('wintitle', self.CODE.upper())
56 self.cb_label = kwargs.get('cb_label', None)
57 self.cb_labels = kwargs.get('cb_labels', None)
50 58 self.xaxis = kwargs.get('xaxis', 'frequency')
51 59 self.zmin = kwargs.get('zmin', None)
52 60 self.zmax = kwargs.get('zmax', None)
61 self.zlimits = kwargs.get('zlimits', None)
53 62 self.xmin = kwargs.get('xmin', None)
63 if self.xmin is not None:
64 self.xmin += 5
54 65 self.xmax = kwargs.get('xmax', None)
55 66 self.xrange = kwargs.get('xrange', 24)
56 67 self.ymin = kwargs.get('ymin', None)
57 68 self.ymax = kwargs.get('ymax', None)
58 self.__MAXNUMY = kwargs.get('decimation', 80)
59 self.throttle_value = 5
60 self.times = []
61 #self.interactive = self.kwargs['parent']
69 self.xlabel = kwargs.get('xlabel', None)
70 self.__MAXNUMY = kwargs.get('decimation', 100)
71 self.showSNR = kwargs.get('showSNR', False)
72 self.oneFigure = kwargs.get('oneFigure', True)
73 self.width = kwargs.get('width', None)
74 self.height = kwargs.get('height', None)
75 self.colorbar = kwargs.get('colorbar', True)
76 self.factors = kwargs.get('factors', [1, 1, 1, 1, 1, 1, 1, 1])
77 self.titles = ['' for __ in range(16)]
78
79 def __setup(self):
80 '''
81 Common setup for all figures, here figures and axes are created
82 '''
83
84 self.setup()
62 85
86 if self.width is None:
87 self.width = 8
88
89 self.figures = []
90 self.axes = []
91 self.cb_axes = []
92 self.pf_axes = []
93 self.cmaps = []
94
95 size = '15%' if self.ncols==1 else '30%'
96 pad = '4%' if self.ncols==1 else '8%'
97
98 if self.oneFigure:
99 if self.height is None:
100 self.height = 1.4*self.nrows + 1
101 fig = plt.figure(figsize=(self.width, self.height),
102 edgecolor='k',
103 facecolor='w')
104 self.figures.append(fig)
105 for n in range(self.nplots):
106 ax = fig.add_subplot(self.nrows, self.ncols, n+1)
107 ax.tick_params(labelsize=8)
108 ax.firsttime = True
109 self.axes.append(ax)
110 if self.showprofile:
111 cax = self.__add_axes(ax, size=size, pad=pad)
112 cax.tick_params(labelsize=8)
113 self.pf_axes.append(cax)
114 else:
115 if self.height is None:
116 self.height = 3
117 for n in range(self.nplots):
118 fig = plt.figure(figsize=(self.width, self.height),
119 edgecolor='k',
120 facecolor='w')
121 ax = fig.add_subplot(1, 1, 1)
122 ax.tick_params(labelsize=8)
123 ax.firsttime = True
124 self.figures.append(fig)
125 self.axes.append(ax)
126 if self.showprofile:
127 cax = self.__add_axes(ax, size=size, pad=pad)
128 cax.tick_params(labelsize=8)
129 self.pf_axes.append(cax)
130
131 for n in range(self.nrows):
132 if self.colormaps is not None:
133 cmap = plt.get_cmap(self.colormaps[n])
134 else:
135 cmap = plt.get_cmap(self.colormap)
136 cmap.set_bad(self.bgcolor, 1.)
137 self.cmaps.append(cmap)
138
139 def __add_axes(self, ax, size='30%', pad='8%'):
63 140 '''
64 this new parameter is created to plot data from varius channels at different figures
65 1. crear una lista de figuras donde se puedan plotear las figuras,
66 2. dar las opciones de configuracion a cada figura, estas opciones son iguales para ambas figuras
67 3. probar?
141 Add new axes to the given figure
68 142 '''
69 self.ind_plt_ch = kwargs.get('ind_plt_ch', False)
70 self.figurelist = None
143 divider = make_axes_locatable(ax)
144 nax = divider.new_horizontal(size=size, pad=pad)
145 ax.figure.add_axes(nax)
146 return nax
71 147
72 148
73 def fill_gaps(self, x_buffer, y_buffer, z_buffer):
149 def setup(self):
150 '''
151 This method should be implemented in the child class, the following
152 attributes should be set:
153
154 self.nrows: number of rows
155 self.ncols: number of cols
156 self.nplots: number of plots (channels or pairs)
157 self.ylabel: label for Y axes
158 self.titles: list of axes title
159
160 '''
161 raise(NotImplementedError, 'Implement this method in child class')
74 162
163 def fill_gaps(self, x_buffer, y_buffer, z_buffer):
164 '''
165 Create a masked array for missing data
166 '''
75 167 if x_buffer.shape[0] < 2:
76 168 return x_buffer, y_buffer, z_buffer
77 169
78 170 deltas = x_buffer[1:] - x_buffer[0:-1]
79 x_median = np.median(deltas)
171 x_median = numpy.median(deltas)
80 172
81 index = np.where(deltas > 5*x_median)
173 index = numpy.where(deltas > 5*x_median)
82 174
83 175 if len(index[0]) != 0:
84 176 z_buffer[::, index[0], ::] = self.__missing
85 z_buffer = np.ma.masked_inside(z_buffer,
177 z_buffer = numpy.ma.masked_inside(z_buffer,
86 178 0.99*self.__missing,
87 179 1.01*self.__missing)
88 180
89 181 return x_buffer, y_buffer, z_buffer
90 182
91 183 def decimate(self):
92 184
93 185 # dx = int(len(self.x)/self.__MAXNUMX) + 1
94 186 dy = int(len(self.y)/self.__MAXNUMY) + 1
95 187
96 188 # x = self.x[::dx]
97 189 x = self.x
98 190 y = self.y[::dy]
99 191 z = self.z[::, ::, ::dy]
100
192
101 193 return x, y, z
102 194
103 '''
104 JM:
105 elimana las otras imagenes generadas debido a que lso workers no llegan en orden y le pueden
106 poner otro tiempo a la figura q no necesariamente es el ultimo.
107 Solo se realiza cuando termina la imagen.
108 Problemas:
195 def format(self):
196 '''
197 Set min and max values, labels, ticks and titles
198 '''
109 199
110 File "/home/ci-81/workspace/schainv2.3/schainpy/model/graphics/jroplot_data.py", line 145, in __plot
111 for n, eachfigure in enumerate(self.figurelist):
112 TypeError: 'NoneType' object is not iterable
200 if self.xmin is None:
201 xmin = self.min_time
202 else:
203 if self.xaxis is 'time':
204 dt = datetime.datetime.fromtimestamp(self.min_time)
205 xmin = (datetime.datetime.combine(dt.date(),
206 datetime.time(int(self.xmin), 0, 0))-d1970).total_seconds()
207 else:
208 xmin = self.xmin
113 209
114 '''
115 def deleteanotherfiles(self):
116 figurenames=[]
117 if self.figurelist != None:
118 for n, eachfigure in enumerate(self.figurelist):
119 #add specific name for each channel in channelList
120 ghostfigname = os.path.join(self.save, '{}_{}_{}'.format(self.titles[n].replace(' ',''),self.CODE,
121 datetime.datetime.fromtimestamp(self.saveTime).strftime('%y%m%d')))
122 figname = os.path.join(self.save, '{}_{}_{}.png'.format(self.titles[n].replace(' ',''),self.CODE,
123 datetime.datetime.fromtimestamp(self.saveTime).strftime('%y%m%d_%H%M%S')))
124
125 for ghostfigure in glob.glob(ghostfigname+'*'): #ghostfigure will adopt all posible names of figures
126 if ghostfigure != figname:
127 os.remove(ghostfigure)
128 print 'Removing GhostFigures:' , figname
129 else :
130 '''Erasing ghost images for just on******************'''
131 ghostfigname = os.path.join(self.save, '{}_{}'.format(self.CODE,datetime.datetime.fromtimestamp(self.saveTime).strftime('%y%m%d')))
132 figname = os.path.join(self.save, '{}_{}.png'.format(self.CODE,datetime.datetime.fromtimestamp(self.saveTime).strftime('%y%m%d_%H%M%S')))
133 for ghostfigure in glob.glob(ghostfigname+'*'): #ghostfigure will adopt all posible names of figures
134 if ghostfigure != figname:
135 os.remove(ghostfigure)
136 print 'Removing GhostFigures:' , figname
210 if self.xmax is None:
211 xmax = xmin+self.xrange*60*60
212 else:
213 if self.xaxis is 'time':
214 dt = datetime.datetime.fromtimestamp(self.min_time)
215 xmax = (datetime.datetime.combine(dt.date(),
216 datetime.time(int(self.xmax), 0, 0))-d1970).total_seconds()
217 else:
218 xmax = self.xmax
219
220 ymin = self.ymin if self.ymin else numpy.nanmin(self.y)
221 ymax = self.ymax if self.ymax else numpy.nanmax(self.y)
222
223 ystep = 200 if ymax>= 800 else 100 if ymax>=400 else 50 if ymax>=200 else 20
224
225 for n, ax in enumerate(self.axes):
226 if ax.firsttime:
227 ax.set_facecolor(self.bgcolor)
228 ax.yaxis.set_major_locator(MultipleLocator(ystep))
229 if self.xaxis is 'time':
230 ax.xaxis.set_major_formatter(FuncFormatter(func))
231 ax.xaxis.set_major_locator(LinearLocator(9))
232 if self.xlabel is not None:
233 ax.set_xlabel(self.xlabel)
234 ax.set_ylabel(self.ylabel)
235 ax.firsttime = False
236 if self.showprofile:
237 self.pf_axes[n].set_ylim(ymin, ymax)
238 self.pf_axes[n].set_xlim(self.zmin, self.zmax)
239 self.pf_axes[n].set_xlabel('dB')
240 self.pf_axes[n].grid(b=True, axis='x')
241 [tick.set_visible(False) for tick in self.pf_axes[n].get_yticklabels()]
242 if self.colorbar:
243 cb = plt.colorbar(ax.plt, ax=ax, pad=0.02)
244 cb.ax.tick_params(labelsize=8)
245 if self.cb_label:
246 cb.set_label(self.cb_label, size=8)
247 elif self.cb_labels:
248 cb.set_label(self.cb_labels[n], size=8)
249
250 ax.set_title('{} - {} UTC'.format(
251 self.titles[n],
252 datetime.datetime.fromtimestamp(self.max_time).strftime('%H:%M:%S')),
253 size=8)
254 ax.set_xlim(xmin, xmax)
255 ax.set_ylim(ymin, ymax)
256
137 257
138 258 def __plot(self):
139
140 print 'plotting...{}'.format(self.CODE)
141 if self.ind_plt_ch is False : #standard
259 '''
260 '''
261 log.success('Plotting', self.name)
262
263 self.plot()
264 self.format()
265
266 for n, fig in enumerate(self.figures):
267 if self.nrows == 0 or self.nplots == 0:
268 log.warning('No data', self.name)
269 continue
142 270 if self.show:
143 self.figure.show()
144 self.plot()
145 plt.tight_layout()
146 self.figure.canvas.manager.set_window_title('{} {} - {}'.format(self.title, self.CODE.upper(),
147 datetime.datetime.fromtimestamp(self.max_time).strftime('%Y/%m/%d')))
148 else :
149 print 'len(self.figurelist): ',len(self.figurelist)
150 for n, eachfigure in enumerate(self.figurelist):
151 if self.show:
152 eachfigure.show()
153
154 self.plot()
155 eachfigure.tight_layout() # ajuste de cada subplot
156 eachfigure.canvas.manager.set_window_title('{} {} - {}'.format(self.title[n], self.CODE.upper(),
157 datetime.datetime.fromtimestamp(self.max_time).strftime('%Y/%m/%d')))
158
159 # if self.save:
160 # if self.ind_plt_ch is False : #standard
161 # figname = os.path.join(self.save, '{}_{}.png'.format(self.CODE,
162 # datetime.datetime.fromtimestamp(self.saveTime).strftime('%y%m%d_%H%M%S')))
163 # print 'Saving figure: {}'.format(figname)
164 # self.figure.savefig(figname)
165 # else :
166 # for n, eachfigure in enumerate(self.figurelist):
167 # #add specific name for each channel in channelList
168 # figname = os.path.join(self.save, '{}_{}_{}.png'.format(self.titles[n],self.CODE,
169 # datetime.datetime.fromtimestamp(self.saveTime).strftime('%y%m%d_%H%M%S')))
170 #
171 # print 'Saving figure: {}'.format(figname)
172 # eachfigure.savefig(figname)
173
174 if self.ind_plt_ch is False :
175 self.figure.canvas.draw()
176 else :
177 for eachfigure in self.figurelist:
178 eachfigure.canvas.draw()
179
180 if self.save:
181 if self.ind_plt_ch is False : #standard
182 figname = os.path.join(self.save, '{}_{}.png'.format(self.CODE,
183 datetime.datetime.fromtimestamp(self.saveTime).strftime('%y%m%d_%H%M%S')))
271 fig.show()
272
273 fig.tight_layout()
274 fig.canvas.manager.set_window_title('{} - {}'.format(self.title,
275 datetime.datetime.fromtimestamp(self.max_time).strftime('%Y/%m/%d')))
276 # fig.canvas.draw()
277
278 if self.save and self.data.ended:
279 channels = range(self.nrows)
280 if self.oneFigure:
281 label = ''
282 else:
283 label = '_{}'.format(channels[n])
284 figname = os.path.join(
285 self.save,
286 '{}{}_{}.png'.format(
287 self.CODE,
288 label,
289 datetime.datetime.fromtimestamp(self.saveTime).strftime('%y%m%d_%H%M%S')
290 )
291 )
184 292 print 'Saving figure: {}'.format(figname)
185 self.figure.savefig(figname)
186 else :
187 for n, eachfigure in enumerate(self.figurelist):
188 #add specific name for each channel in channelList
189 figname = os.path.join(self.save, '{}_{}_{}.png'.format(self.titles[n].replace(' ',''),self.CODE,
190 datetime.datetime.fromtimestamp(self.saveTime).strftime('%y%m%d_%H%M%S')))
191
192 print 'Saving figure: {}'.format(figname)
193 eachfigure.savefig(figname)
194
293 fig.savefig(figname)
195 294
196 295 def plot(self):
197
198 print 'plotting...{}'.format(self.CODE.upper())
199 return
296 '''
297 '''
298 raise(NotImplementedError, 'Implement this method in child class')
200 299
201 300 def run(self):
202 301
203 print '[Starting] {}'.format(self.name)
302 log.success('Starting', self.name)
204 303
205 304 context = zmq.Context()
206 305 receiver = context.socket(zmq.SUB)
207 306 receiver.setsockopt(zmq.SUBSCRIBE, '')
208 307 receiver.setsockopt(zmq.CONFLATE, self.CONFLATE)
209 308
210 309 if 'server' in self.kwargs['parent']:
211 310 receiver.connect('ipc:///tmp/{}.plots'.format(self.kwargs['parent']['server']))
212 311 else:
213 receiver.connect("ipc:///tmp/zmq.plots")
214
215 seconds_passed = 0
312 receiver.connect("ipc:///tmp/zmq.plots")
216 313
217 314 while True:
218 315 try:
219 self.data = receiver.recv_pyobj(flags=zmq.NOBLOCK)#flags=zmq.NOBLOCK
220 self.started = self.data['STARTED']
221 self.dataOut = self.data['dataOut']
222
223 if (len(self.times) < len(self.data['times']) and not self.started and self.data['ENDED']):
224 continue
225
226 self.times = self.data['times']
227 self.times.sort()
228 self.throttle_value = self.data['throttle']
229 self.min_time = self.times[0]
230 self.max_time = self.times[-1]
316 self.data = receiver.recv_pyobj(flags=zmq.NOBLOCK)
317
318 self.min_time = self.data.times[0]
319 self.max_time = self.data.times[-1]
231 320
232 321 if self.isConfig is False:
233 print 'setting up'
234 self.setup()
322 self.__setup()
235 323 self.isConfig = True
236 self.__plot()
237
238 if self.data['ENDED'] is True:
239 print '********GRAPHIC ENDED********'
240 self.ended = True
241 self.isConfig = False
242 self.__plot()
243 self.deleteanotherfiles() #CLPDG
244 elif seconds_passed >= self.data['throttle']:
245 print 'passed', seconds_passed
246 self.__plot()
247 seconds_passed = 0
324
325 self.__plot()
248 326
249 327 except zmq.Again as e:
250 print 'Waiting for data...'
251 plt.pause(2)
252 seconds_passed += 2
328 log.log('Waiting for data...')
329 if self.data:
330 plt.pause(self.data.throttle)
331 else:
332 time.sleep(2)
253 333
254 334 def close(self):
255 if self.dataOut:
335 if self.data:
256 336 self.__plot()
257 337
258 338
259 339 class PlotSpectraData(PlotData):
340 '''
341 Plot for Spectra data
342 '''
260 343
261 344 CODE = 'spc'
262 colormap = 'jro'
263 CONFLATE = False
345 colormap = 'jro'
264 346
265 347 def setup(self):
266
267 ncolspan = 1
268 colspan = 1
269 self.ncols = int(numpy.sqrt(self.dataOut.nChannels)+0.9)
270 self.nrows = int(self.dataOut.nChannels*1./self.ncols + 0.9)
271 self.width = 3.6*self.ncols
272 self.height = 3.2*self.nrows
273 if self.showprofile:
274 ncolspan = 3
275 colspan = 2
276 self.width += 1.2*self.ncols
348 self.nplots = len(self.data.channels)
349 self.ncols = int(numpy.sqrt(self.nplots)+ 0.9)
350 self.nrows = int((1.0*self.nplots/self.ncols) + 0.9)
351 self.width = 3.4*self.ncols
352 self.height = 3*self.nrows
353 self.cb_label = 'dB'
354 if self.showprofile:
355 self.width += 0.8*self.ncols
277 356
278 357 self.ylabel = 'Range [Km]'
279 self.titles = ['Channel {}'.format(x) for x in self.dataOut.channelList]
280
281 if self.figure is None:
282 self.figure = plt.figure(figsize=(self.width, self.height),
283 edgecolor='k',
284 facecolor='w')
285 else:
286 self.figure.clf()
287
288 n = 0
289 for y in range(self.nrows):
290 for x in range(self.ncols):
291 if n >= self.dataOut.nChannels:
292 break
293 ax = plt.subplot2grid((self.nrows, self.ncols*ncolspan), (y, x*ncolspan), 1, colspan)
294 if self.showprofile:
295 ax.ax_profile = plt.subplot2grid((self.nrows, self.ncols*ncolspan), (y, x*ncolspan+colspan), 1, 1)
296
297 ax.firsttime = True
298 self.axes.append(ax)
299 n += 1
300 358
301 359 def plot(self):
302
303 360 if self.xaxis == "frequency":
304 x = self.dataOut.getFreqRange(1)/1000.
305 xlabel = "Frequency (kHz)"
361 x = self.data.xrange[0]
362 self.xlabel = "Frequency (kHz)"
306 363 elif self.xaxis == "time":
307 x = self.dataOut.getAcfRange(1)
308 xlabel = "Time (ms)"
364 x = self.data.xrange[1]
365 self.xlabel = "Time (ms)"
309 366 else:
310 x = self.dataOut.getVelRange(1)
311 xlabel = "Velocity (m/s)"
367 x = self.data.xrange[2]
368 self.xlabel = "Velocity (m/s)"
312 369
313 y = self.dataOut.getHeiRange()
314 z = self.data[self.CODE]
370 if self.CODE == 'spc_mean':
371 x = self.data.xrange[2]
372 self.xlabel = "Velocity (m/s)"
315 373
374 self.titles = []
375
376 y = self.data.heights
377 self.y = y
378 z = self.data['spc']
379
316 380 for n, ax in enumerate(self.axes):
381 noise = self.data['noise'][n][-1]
382 if self.CODE == 'spc_mean':
383 mean = self.data['mean'][n][-1]
317 384 if ax.firsttime:
318 self.xmax = self.xmax if self.xmax else np.nanmax(x)
385 self.xmax = self.xmax if self.xmax else numpy.nanmax(x)
319 386 self.xmin = self.xmin if self.xmin else -self.xmax
320 self.ymin = self.ymin if self.ymin else np.nanmin(y)
321 self.ymax = self.ymax if self.ymax else np.nanmax(y)
322 self.zmin = self.zmin if self.zmin else np.nanmin(z)
323 self.zmax = self.zmax if self.zmax else np.nanmax(z)
324 ax.plot = ax.pcolormesh(x, y, z[n].T,
325 vmin=self.zmin,
326 vmax=self.zmax,
327 cmap=plt.get_cmap(self.colormap)
328 )
329 divider = make_axes_locatable(ax)
330 cax = divider.new_horizontal(size='3%', pad=0.05)
331 self.figure.add_axes(cax)
332 plt.colorbar(ax.plot, cax)
333
334 ax.set_xlim(self.xmin, self.xmax)
335 ax.set_ylim(self.ymin, self.ymax)
336
337 ax.set_ylabel(self.ylabel)
338 ax.set_xlabel(xlabel)
339
340 ax.firsttime = False
387 self.zmin = self.zmin if self.zmin else numpy.nanmin(z)
388 self.zmax = self.zmax if self.zmax else numpy.nanmax(z)
389 ax.plt = ax.pcolormesh(x, y, z[n].T,
390 vmin=self.zmin,
391 vmax=self.zmax,
392 cmap=plt.get_cmap(self.colormap)
393 )
341 394
342 395 if self.showprofile:
343 ax.plot_profile= ax.ax_profile.plot(self.data['rti'][self.max_time][n], y)[0]
344 ax.ax_profile.set_xlim(self.zmin, self.zmax)
345 ax.ax_profile.set_ylim(self.ymin, self.ymax)
346 ax.ax_profile.set_xlabel('dB')
347 ax.ax_profile.grid(b=True, axis='x')
348 ax.plot_noise = ax.ax_profile.plot(numpy.repeat(self.data['noise'][self.max_time][n], len(y)), y,
349 color="k", linestyle="dashed", lw=2)[0]
350 [tick.set_visible(False) for tick in ax.ax_profile.get_yticklabels()]
396 ax.plt_profile= self.pf_axes[n].plot(self.data['rti'][n][-1], y)[0]
397 ax.plt_noise = self.pf_axes[n].plot(numpy.repeat(noise, len(y)), y,
398 color="k", linestyle="dashed", lw=1)[0]
399 if self.CODE == 'spc_mean':
400 ax.plt_mean = ax.plot(mean, y, color='k')[0]
351 401 else:
352 ax.plot.set_array(z[n].T.ravel())
402 ax.plt.set_array(z[n].T.ravel())
353 403 if self.showprofile:
354 ax.plot_profile.set_data(self.data['rti'][self.max_time][n], y)
355 ax.plot_noise.set_data(numpy.repeat(self.data['noise'][self.max_time][n], len(y)), y)
404 ax.plt_profile.set_data(self.data['rti'][n][-1], y)
405 ax.plt_noise.set_data(numpy.repeat(noise, len(y)), y)
406 if self.CODE == 'spc_mean':
407 ax.plt_mean.set_data(mean, y)
356 408
357 ax.set_title('{} - Noise: {:.2f} dB'.format(self.titles[n], self.data['noise'][self.max_time][n]),
358 size=8)
409 self.titles.append('CH {}: {:3.2f}dB'.format(n, noise))
359 410 self.saveTime = self.max_time
360 411
361 412
362 413 class PlotCrossSpectraData(PlotData):
363 414
364 415 CODE = 'cspc'
365 416 zmin_coh = None
366 417 zmax_coh = None
367 418 zmin_phase = None
368 zmax_phase = None
369 CONFLATE = False
419 zmax_phase = None
370 420
371 421 def setup(self):
372 422
373 ncolspan = 1
374 colspan = 1
375 self.ncols = 2
376 self.nrows = self.dataOut.nPairs
377 self.width = 3.6*self.ncols
378 self.height = 3.2*self.nrows
379
423 self.ncols = 4
424 self.nrows = len(self.data.pairs)
425 self.nplots = self.nrows*4
426 self.width = 3.4*self.ncols
427 self.height = 3*self.nrows
380 428 self.ylabel = 'Range [Km]'
381 self.titles = ['Channel {}'.format(x) for x in self.dataOut.channelList]
382
383 if self.figure is None:
384 self.figure = plt.figure(figsize=(self.width, self.height),
385 edgecolor='k',
386 facecolor='w')
387 else:
388 self.figure.clf()
389
390 for y in range(self.nrows):
391 for x in range(self.ncols):
392 ax = plt.subplot2grid((self.nrows, self.ncols), (y, x), 1, 1)
393 ax.firsttime = True
394 self.axes.append(ax)
429 self.showprofile = False
395 430
396 431 def plot(self):
397 432
398 433 if self.xaxis == "frequency":
399 x = self.dataOut.getFreqRange(1)/1000.
400 xlabel = "Frequency (kHz)"
434 x = self.data.xrange[0]
435 self.xlabel = "Frequency (kHz)"
401 436 elif self.xaxis == "time":
402 x = self.dataOut.getAcfRange(1)
403 xlabel = "Time (ms)"
437 x = self.data.xrange[1]
438 self.xlabel = "Time (ms)"
404 439 else:
405 x = self.dataOut.getVelRange(1)
406 xlabel = "Velocity (m/s)"
440 x = self.data.xrange[2]
441 self.xlabel = "Velocity (m/s)"
407 442
408 y = self.dataOut.getHeiRange()
409 z_coh = self.data['cspc_coh']
410 z_phase = self.data['cspc_phase']
443 self.titles = []
444
445 y = self.data.heights
446 self.y = y
447 spc = self.data['spc']
448 cspc = self.data['cspc']
411 449
412 450 for n in range(self.nrows):
413 ax = self.axes[2*n]
414 ax1 = self.axes[2*n+1]
451 noise = self.data['noise'][n][-1]
452 pair = self.data.pairs[n]
453 ax = self.axes[4*n]
454 ax3 = self.axes[4*n+3]
415 455 if ax.firsttime:
416 self.xmax = self.xmax if self.xmax else np.nanmax(x)
456 self.xmax = self.xmax if self.xmax else numpy.nanmax(x)
417 457 self.xmin = self.xmin if self.xmin else -self.xmax
418 self.ymin = self.ymin if self.ymin else np.nanmin(y)
419 self.ymax = self.ymax if self.ymax else np.nanmax(y)
420 self.zmin_coh = self.zmin_coh if self.zmin_coh else 0.0
421 self.zmax_coh = self.zmax_coh if self.zmax_coh else 1.0
422 self.zmin_phase = self.zmin_phase if self.zmin_phase else -180
423 self.zmax_phase = self.zmax_phase if self.zmax_phase else 180
424
425 ax.plot = ax.pcolormesh(x, y, z_coh[n].T,
426 vmin=self.zmin_coh,
427 vmax=self.zmax_coh,
428 cmap=plt.get_cmap(self.colormap_coh)
429 )
430 divider = make_axes_locatable(ax)
431 cax = divider.new_horizontal(size='3%', pad=0.05)
432 self.figure.add_axes(cax)
433 plt.colorbar(ax.plot, cax)
434
435 ax.set_xlim(self.xmin, self.xmax)
436 ax.set_ylim(self.ymin, self.ymax)
437
438 ax.set_ylabel(self.ylabel)
439 ax.set_xlabel(xlabel)
440 ax.firsttime = False
441
442 ax1.plot = ax1.pcolormesh(x, y, z_phase[n].T,
443 vmin=self.zmin_phase,
444 vmax=self.zmax_phase,
445 cmap=plt.get_cmap(self.colormap_phase)
446 )
447 divider = make_axes_locatable(ax1)
448 cax = divider.new_horizontal(size='3%', pad=0.05)
449 self.figure.add_axes(cax)
450 plt.colorbar(ax1.plot, cax)
451
452 ax1.set_xlim(self.xmin, self.xmax)
453 ax1.set_ylim(self.ymin, self.ymax)
454
455 ax1.set_ylabel(self.ylabel)
456 ax1.set_xlabel(xlabel)
457 ax1.firsttime = False
458 self.zmin = self.zmin if self.zmin else numpy.nanmin(spc)
459 self.zmax = self.zmax if self.zmax else numpy.nanmax(spc)
460 ax.plt = ax.pcolormesh(x, y, spc[pair[0]].T,
461 vmin=self.zmin,
462 vmax=self.zmax,
463 cmap=plt.get_cmap(self.colormap)
464 )
458 465 else:
459 ax.plot.set_array(z_coh[n].T.ravel())
460 ax1.plot.set_array(z_phase[n].T.ravel())
461
462 ax.set_title('Coherence Ch{} * Ch{}'.format(self.dataOut.pairsList[n][0], self.dataOut.pairsList[n][1]), size=8)
463 ax1.set_title('Phase Ch{} * Ch{}'.format(self.dataOut.pairsList[n][0], self.dataOut.pairsList[n][1]), size=8)
464 self.saveTime = self.max_time
465
466
467 class PlotSpectraMeanData(PlotSpectraData):
466 ax.plt.set_array(spc[pair[0]].T.ravel())
467 self.titles.append('CH {}: {:3.2f}dB'.format(n, noise))
468 468
469 CODE = 'spc_mean'
470 colormap = 'jet'
471
472 def plot(self):
473
474 if self.xaxis == "frequency":
475 x = self.dataOut.getFreqRange(1)/1000.
476 xlabel = "Frequency (kHz)"
477 elif self.xaxis == "time":
478 x = self.dataOut.getAcfRange(1)
479 xlabel = "Time (ms)"
480 else:
481 x = self.dataOut.getVelRange(1)
482 xlabel = "Velocity (m/s)"
483
484 y = self.dataOut.getHeiRange()
485 z = self.data['spc']
486 mean = self.data['mean'][self.max_time]
487
488 for n, ax in enumerate(self.axes):
489
490 if ax.firsttime:
491 self.xmax = self.xmax if self.xmax else np.nanmax(x)
492 self.xmin = self.xmin if self.xmin else -self.xmax
493 self.ymin = self.ymin if self.ymin else np.nanmin(y)
494 self.ymax = self.ymax if self.ymax else np.nanmax(y)
495 self.zmin = self.zmin if self.zmin else np.nanmin(z)
496 self.zmax = self.zmax if self.zmax else np.nanmax(z)
497 ax.plt = ax.pcolormesh(x, y, z[n].T,
469 ax = self.axes[4*n+1]
470 if ax.firsttime:
471 ax.plt = ax.pcolormesh(x, y, spc[pair[1]].T,
498 472 vmin=self.zmin,
499 473 vmax=self.zmax,
500 474 cmap=plt.get_cmap(self.colormap)
501 475 )
502 ax.plt_dop = ax.plot(mean[n], y,
503 color='k')[0]
504
505 divider = make_axes_locatable(ax)
506 cax = divider.new_horizontal(size='3%', pad=0.05)
507 self.figure.add_axes(cax)
508 plt.colorbar(ax.plt, cax)
509
510 ax.set_xlim(self.xmin, self.xmax)
511 ax.set_ylim(self.ymin, self.ymax)
512
513 ax.set_ylabel(self.ylabel)
514 ax.set_xlabel(xlabel)
515
516 ax.firsttime = False
517
518 if self.showprofile:
519 ax.plt_profile= ax.ax_profile.plot(self.data['rti'][self.max_time][n], y)[0]
520 ax.ax_profile.set_xlim(self.zmin, self.zmax)
521 ax.ax_profile.set_ylim(self.ymin, self.ymax)
522 ax.ax_profile.set_xlabel('dB')
523 ax.ax_profile.grid(b=True, axis='x')
524 ax.plt_noise = ax.ax_profile.plot(numpy.repeat(self.data['noise'][self.max_time][n], len(y)), y,
525 color="k", linestyle="dashed", lw=2)[0]
526 [tick.set_visible(False) for tick in ax.ax_profile.get_yticklabels()]
527 476 else:
528 ax.plt.set_array(z[n].T.ravel())
529 ax.plt_dop.set_data(mean[n], y)
530 if self.showprofile:
531 ax.plt_profile.set_data(self.data['rti'][self.max_time][n], y)
532 ax.plt_noise.set_data(numpy.repeat(self.data['noise'][self.max_time][n], len(y)), y)
477 ax.plt.set_array(spc[pair[1]].T.ravel())
478 self.titles.append('CH {}: {:3.2f}dB'.format(n, noise))
479
480 out = cspc[n]/numpy.sqrt(spc[pair[0]]*spc[pair[1]])
481 coh = numpy.abs(out)
482 phase = numpy.arctan2(out.imag, out.real)*180/numpy.pi
483
484 ax = self.axes[4*n+2]
485 if ax.firsttime:
486 ax.plt = ax.pcolormesh(x, y, coh.T,
487 vmin=0,
488 vmax=1,
489 cmap=plt.get_cmap(self.colormap_coh)
490 )
491 else:
492 ax.plt.set_array(coh.T.ravel())
493 self.titles.append('Coherence Ch{} * Ch{}'.format(pair[0], pair[1]))
533 494
534 ax.set_title('{} - Noise: {:.2f} dB'.format(self.titles[n], self.data['noise'][self.max_time][n]),
535 size=8)
495 ax = self.axes[4*n+3]
496 if ax.firsttime:
497 ax.plt = ax.pcolormesh(x, y, phase.T,
498 vmin=-180,
499 vmax=180,
500 cmap=plt.get_cmap(self.colormap_phase)
501 )
502 else:
503 ax.plt.set_array(phase.T.ravel())
504 self.titles.append('Phase CH{} * CH{}'.format(pair[0], pair[1]))
505
536 506 self.saveTime = self.max_time
537 507
538 508
509 class PlotSpectraMeanData(PlotSpectraData):
510 '''
511 Plot for Spectra and Mean
512 '''
513 CODE = 'spc_mean'
514 colormap = 'jro'
515
516
539 517 class PlotRTIData(PlotData):
518 '''
519 Plot for RTI data
520 '''
540 521
541 522 CODE = 'rti'
542 523 colormap = 'jro'
543 524
544 525 def setup(self):
545 self.ncols = 1
546 self.nrows = self.dataOut.nChannels
547 self.width = 10
548 #TODO : arreglar la altura de la figura, esta hardcodeada.
549 #Se arreglo, testear!
550 if self.ind_plt_ch:
551 self.height = 3.2#*self.nrows if self.nrows<6 else 12
552 else:
553 self.height = 2.2*self.nrows if self.nrows<6 else 12
554
555 '''
556 if self.nrows==1:
557 self.height += 1
558 '''
526 self.xaxis = 'time'
527 self.ncols = 1
528 self.nrows = len(self.data.channels)
529 self.nplots = len(self.data.channels)
559 530 self.ylabel = 'Range [Km]'
560 self.titles = ['Channel {}'.format(x) for x in self.dataOut.channelList]
561
562 '''
563 Logica:
564 1) Si la variable ind_plt_ch es True, va a crear mas de 1 figura
565 2) guardamos "Figures" en una lista y "axes" en otra, quizas se deberia guardar el
566 axis dentro de "Figures" como un diccionario.
567 '''
568 if self.ind_plt_ch is False: #standard mode
569
570 if self.figure is None: #solo para la priemra vez
571 self.figure = plt.figure(figsize=(self.width, self.height),
572 edgecolor='k',
573 facecolor='w')
574 else:
575 self.figure.clf()
576 self.axes = []
577
578
579 for n in range(self.nrows):
580 ax = self.figure.add_subplot(self.nrows, self.ncols, n+1)
581 #ax = self.figure(n+1)
582 ax.firsttime = True
583 self.axes.append(ax)
584
585 else : #append one figure foreach channel in channelList
586 if self.figurelist == None:
587 self.figurelist = []
588 for n in range(self.nrows):
589 self.figure = plt.figure(figsize=(self.width, self.height),
590 edgecolor='k',
591 facecolor='w')
592 #add always one subplot
593 self.figurelist.append(self.figure)
594
595 else : # cada dia nuevo limpia el axes, pero mantiene el figure
596 for eachfigure in self.figurelist:
597 eachfigure.clf() # eliminaria todas las figuras de la lista?
598 self.axes = []
599
600 for eachfigure in self.figurelist:
601 ax = eachfigure.add_subplot(1,1,1) #solo 1 axis por figura
602 #ax = self.figure(n+1)
603 ax.firsttime = True
604 #Cada figura tiene un distinto puntero
605 self.axes.append(ax)
606 #plt.close(eachfigure)
607
531 self.cb_label = 'dB'
532 self.titles = ['{} Channel {}'.format(self.CODE.upper(), x) for x in range(self.nrows)]
608 533
609 534 def plot(self):
535 self.x = self.data.times
536 self.y = self.data.heights
537 self.z = self.data[self.CODE]
538 self.z = numpy.ma.masked_invalid(self.z)
610 539
611 if self.ind_plt_ch is False: #standard mode
612 self.x = np.array(self.times)
613 self.y = self.dataOut.getHeiRange()
614 self.z = []
615
616 for ch in range(self.nrows):
617 self.z.append([self.data[self.CODE][t][ch] for t in self.times])
618
619 self.z = np.array(self.z)
620 for n, ax in enumerate(self.axes):
621 x, y, z = self.fill_gaps(*self.decimate())
622 xmin = self.min_time
623 xmax = xmin+self.xrange*60*60
624 self.zmin = self.zmin if self.zmin else np.min(self.z)
625 self.zmax = self.zmax if self.zmax else np.max(self.z)
626 if ax.firsttime:
627 self.ymin = self.ymin if self.ymin else np.nanmin(self.y)
628 self.ymax = self.ymax if self.ymax else np.nanmax(self.y)
629 plot = ax.pcolormesh(x, y, z[n].T,
630 vmin=self.zmin,
631 vmax=self.zmax,
632 cmap=plt.get_cmap(self.colormap)
633 )
634 divider = make_axes_locatable(ax)
635 cax = divider.new_horizontal(size='2%', pad=0.05)
636 self.figure.add_axes(cax)
637 plt.colorbar(plot, cax)
638 ax.set_ylim(self.ymin, self.ymax)
639 ax.xaxis.set_major_formatter(FuncFormatter(func))
640 ax.xaxis.set_major_locator(LinearLocator(6))
641 ax.set_ylabel(self.ylabel)
642 if self.xmin is None:
643 xmin = self.min_time
644 else:
645 xmin = (datetime.datetime.combine(self.dataOut.datatime.date(),
646 datetime.time(self.xmin, 0, 0))-d1970).total_seconds()
647 ax.set_xlim(xmin, xmax)
648 ax.firsttime = False
649 else:
650 ax.collections.remove(ax.collections[0])
651 ax.set_xlim(xmin, xmax)
652 plot = ax.pcolormesh(x, y, z[n].T,
653 vmin=self.zmin,
654 vmax=self.zmax,
655 cmap=plt.get_cmap(self.colormap)
656 )
657 ax.set_title('{} {}'.format(self.titles[n],
658 datetime.datetime.fromtimestamp(self.max_time).strftime('%y/%m/%d %H:%M:%S')),
659 size=8)
660
661 self.saveTime = self.min_time
662 else :
663 self.x = np.array(self.times)
664 self.y = self.dataOut.getHeiRange()
665 self.z = []
666
667 for ch in range(self.nrows):
668 self.z.append([self.data[self.CODE][t][ch] for t in self.times])
669
670 self.z = np.array(self.z)
671 for n, eachfigure in enumerate(self.figurelist): #estaba ax in axes
672
673 x, y, z = self.fill_gaps(*self.decimate())
674 xmin = self.min_time
675 xmax = xmin+self.xrange*60*60
676 self.zmin = self.zmin if self.zmin else np.min(self.z)
677 self.zmax = self.zmax if self.zmax else np.max(self.z)
678 if self.axes[n].firsttime:
679 self.ymin = self.ymin if self.ymin else np.nanmin(self.y)
680 self.ymax = self.ymax if self.ymax else np.nanmax(self.y)
681 plot = self.axes[n].pcolormesh(x, y, z[n].T,
682 vmin=self.zmin,
683 vmax=self.zmax,
684 cmap=plt.get_cmap(self.colormap)
685 )
686 divider = make_axes_locatable(self.axes[n])
687 cax = divider.new_horizontal(size='2%', pad=0.05)
688 eachfigure.add_axes(cax)
689 #self.figure2.add_axes(cax)
690 plt.colorbar(plot, cax)
691 self.axes[n].set_ylim(self.ymin, self.ymax)
692
693 self.axes[n].xaxis.set_major_formatter(FuncFormatter(func))
694 self.axes[n].xaxis.set_major_locator(LinearLocator(6))
695
696 self.axes[n].set_ylabel(self.ylabel)
697
698 if self.xmin is None:
699 xmin = self.min_time
700 else:
701 xmin = (datetime.datetime.combine(self.dataOut.datatime.date(),
702 datetime.time(self.xmin, 0, 0))-d1970).total_seconds()
703
704 self.axes[n].set_xlim(xmin, xmax)
705 self.axes[n].firsttime = False
706 else:
707 self.axes[n].collections.remove(self.axes[n].collections[0])
708 self.axes[n].set_xlim(xmin, xmax)
709 plot = self.axes[n].pcolormesh(x, y, z[n].T,
710 vmin=self.zmin,
711 vmax=self.zmax,
712 cmap=plt.get_cmap(self.colormap)
713 )
714 self.axes[n].set_title('{} {}'.format(self.titles[n],
715 datetime.datetime.fromtimestamp(self.max_time).strftime('%y/%m/%d %H:%M:%S')),
716 size=8)
540 for n, ax in enumerate(self.axes):
541 x, y, z = self.fill_gaps(*self.decimate())
542 self.zmin = self.zmin if self.zmin else numpy.min(self.z)
543 self.zmax = self.zmax if self.zmax else numpy.max(self.z)
544 if ax.firsttime:
545 ax.plt = ax.pcolormesh(x, y, z[n].T,
546 vmin=self.zmin,
547 vmax=self.zmax,
548 cmap=plt.get_cmap(self.colormap)
549 )
550 if self.showprofile:
551 ax.plot_profile= self.pf_axes[n].plot(self.data['rti'][n][-1], self.y)[0]
552 ax.plot_noise = self.pf_axes[n].plot(numpy.repeat(self.data['noise'][n][-1], len(self.y)), self.y,
553 color="k", linestyle="dashed", lw=1)[0]
554 else:
555 ax.collections.remove(ax.collections[0])
556 ax.plt = ax.pcolormesh(x, y, z[n].T,
557 vmin=self.zmin,
558 vmax=self.zmax,
559 cmap=plt.get_cmap(self.colormap)
560 )
561 if self.showprofile:
562 ax.plot_profile.set_data(self.data['rti'][n][-1], self.y)
563 ax.plot_noise.set_data(numpy.repeat(self.data['noise'][n][-1], len(self.y)), self.y)
717 564
718 self.saveTime = self.min_time
565 self.saveTime = self.min_time
719 566
720 567
721 568 class PlotCOHData(PlotRTIData):
569 '''
570 Plot for Coherence data
571 '''
722 572
723 573 CODE = 'coh'
724 574
725 575 def setup(self):
726
576 self.xaxis = 'time'
727 577 self.ncols = 1
728 self.nrows = self.dataOut.nPairs
729 self.width = 10
730 self.height = 2.2*self.nrows if self.nrows<6 else 12
731 self.ind_plt_ch = False #just for coherence and phase
732 if self.nrows==1:
733 self.height += 1
734 self.ylabel = 'Range [Km]'
735 self.titles = ['{} Ch{} * Ch{}'.format(self.CODE.upper(), x[0], x[1]) for x in self.dataOut.pairsList]
736
737 if self.figure is None:
738 self.figure = plt.figure(figsize=(self.width, self.height),
739 edgecolor='k',
740 facecolor='w')
578 self.nrows = len(self.data.pairs)
579 self.nplots = len(self.data.pairs)
580 self.ylabel = 'Range [Km]'
581 if self.CODE == 'coh':
582 self.cb_label = ''
583 self.titles = ['Coherence Map Ch{} * Ch{}'.format(x[0], x[1]) for x in self.data.pairs]
741 584 else:
742 self.figure.clf()
743 self.axes = []
585 self.cb_label = 'Degrees'
586 self.titles = ['Phase Map Ch{} * Ch{}'.format(x[0], x[1]) for x in self.data.pairs]
744 587
745 for n in range(self.nrows):
746 ax = self.figure.add_subplot(self.nrows, self.ncols, n+1)
747 ax.firsttime = True
748 self.axes.append(ax)
588
589 class PlotPHASEData(PlotCOHData):
590 '''
591 Plot for Phase map data
592 '''
593
594 CODE = 'phase'
595 colormap = 'seismic'
749 596
750 597
751 598 class PlotNoiseData(PlotData):
599 '''
600 Plot for noise
601 '''
602
752 603 CODE = 'noise'
753 604
754 605 def setup(self):
755
606 self.xaxis = 'time'
756 607 self.ncols = 1
757 608 self.nrows = 1
758 self.width = 10
759 self.height = 3.2
609 self.nplots = 1
760 610 self.ylabel = 'Intensity [dB]'
761 611 self.titles = ['Noise']
762
763 if self.figure is None:
764 self.figure = plt.figure(figsize=(self.width, self.height),
765 edgecolor='k',
766 facecolor='w')
767 else:
768 self.figure.clf()
769 self.axes = []
770
771 self.ax = self.figure.add_subplot(self.nrows, self.ncols, 1)
772 self.ax.firsttime = True
612 self.colorbar = False
773 613
774 614 def plot(self):
775 615
776 x = self.times
616 x = self.data.times
777 617 xmin = self.min_time
778 618 xmax = xmin+self.xrange*60*60
779 if self.ax.firsttime:
780 for ch in self.dataOut.channelList:
781 y = [self.data[self.CODE][t][ch] for t in self.times]
782 self.ax.plot(x, y, lw=1, label='Ch{}'.format(ch))
783 self.ax.firsttime = False
784 self.ax.xaxis.set_major_formatter(FuncFormatter(func))
785 self.ax.xaxis.set_major_locator(LinearLocator(6))
786 self.ax.set_ylabel(self.ylabel)
619 Y = self.data[self.CODE]
620
621 if self.axes[0].firsttime:
622 for ch in self.data.channels:
623 y = Y[ch]
624 self.axes[0].plot(x, y, lw=1, label='Ch{}'.format(ch))
787 625 plt.legend()
788 626 else:
789 for ch in self.dataOut.channelList:
790 y = [self.data[self.CODE][t][ch] for t in self.times]
791 self.ax.lines[ch].set_data(x, y)
792
793 self.ax.set_xlim(xmin, xmax)
794 self.ax.set_ylim(min(y)-5, max(y)+5)
627 for ch in self.data.channels:
628 y = Y[ch]
629 self.axes[0].lines[ch].set_data(x, y)
630
631 self.ymin = numpy.nanmin(Y) - 5
632 self.ymax = numpy.nanmax(Y) + 5
795 633 self.saveTime = self.min_time
796 634
797 635
798 class PlotWindProfilerData(PlotRTIData):
799
800 CODE = 'wind'
801 colormap = 'seismic'
802
803 def setup(self):
804 self.ncols = 1
805 self.nrows = self.dataOut.data_output.shape[0]
806 self.width = 10
807 self.height = 2.2*self.nrows
808 self.ylabel = 'Height [Km]'
809 self.titles = ['Zonal Wind' ,'Meridional Wind', 'Vertical Wind']
810 self.clabels = ['Velocity (m/s)','Velocity (m/s)','Velocity (cm/s)']
811 self.windFactor = [1, 1, 100]
812
813 if self.figure is None:
814 self.figure = plt.figure(figsize=(self.width, self.height),
815 edgecolor='k',
816 facecolor='w')
817 else:
818 self.figure.clf()
819 self.axes = []
820
821 for n in range(self.nrows):
822 ax = self.figure.add_subplot(self.nrows, self.ncols, n+1)
823 ax.firsttime = True
824 self.axes.append(ax)
825
826 def plot(self):
827
828 self.x = np.array(self.times)
829 self.y = self.dataOut.heightList
830 self.z = []
831
832 for ch in range(self.nrows):
833 self.z.append([self.data['output'][t][ch] for t in self.times])
834
835 self.z = np.array(self.z)
836 self.z = numpy.ma.masked_invalid(self.z)
837
838 cmap=plt.get_cmap(self.colormap)
839 cmap.set_bad('black', 1.)
840
841 for n, ax in enumerate(self.axes):
842 x, y, z = self.fill_gaps(*self.decimate())
843 xmin = self.min_time
844 xmax = xmin+self.xrange*60*60
845 if ax.firsttime:
846 self.ymin = self.ymin if self.ymin else np.nanmin(self.y)
847 self.ymax = self.ymax if self.ymax else np.nanmax(self.y)
848 self.zmax = self.zmax if self.zmax else numpy.nanmax(abs(self.z[:-1, :]))
849 self.zmin = self.zmin if self.zmin else -self.zmax
850
851 plot = ax.pcolormesh(x, y, z[n].T*self.windFactor[n],
852 vmin=self.zmin,
853 vmax=self.zmax,
854 cmap=cmap
855 )
856 divider = make_axes_locatable(ax)
857 cax = divider.new_horizontal(size='2%', pad=0.05)
858 self.figure.add_axes(cax)
859 cb = plt.colorbar(plot, cax)
860 cb.set_label(self.clabels[n])
861 ax.set_ylim(self.ymin, self.ymax)
862
863 ax.xaxis.set_major_formatter(FuncFormatter(func))
864 ax.xaxis.set_major_locator(LinearLocator(6))
865
866 ax.set_ylabel(self.ylabel)
867
868 ax.set_xlim(xmin, xmax)
869 ax.firsttime = False
870 else:
871 ax.collections.remove(ax.collections[0])
872 ax.set_xlim(xmin, xmax)
873 plot = ax.pcolormesh(x, y, z[n].T*self.windFactor[n],
874 vmin=self.zmin,
875 vmax=self.zmax,
876 cmap=plt.get_cmap(self.colormap)
877 )
878 ax.set_title('{} {}'.format(self.titles[n],
879 datetime.datetime.fromtimestamp(self.max_time).strftime('%y/%m/%d %H:%M:%S')),
880 size=8)
881
882 self.saveTime = self.min_time
883
884
885 636 class PlotSNRData(PlotRTIData):
637 '''
638 Plot for SNR Data
639 '''
640
886 641 CODE = 'snr'
887 642 colormap = 'jet'
888 643
644
889 645 class PlotDOPData(PlotRTIData):
646 '''
647 Plot for DOPPLER Data
648 '''
649
890 650 CODE = 'dop'
891 651 colormap = 'jet'
892 652
893 653
894 class PlotPHASEData(PlotCOHData):
895 CODE = 'phase'
896 colormap = 'seismic'
897
898
899 654 class PlotSkyMapData(PlotData):
655 '''
656 Plot for meteors detection data
657 '''
900 658
901 659 CODE = 'met'
902 660
903 661 def setup(self):
904 662
905 663 self.ncols = 1
906 664 self.nrows = 1
907 665 self.width = 7.2
908 666 self.height = 7.2
909 667
910 668 self.xlabel = 'Zonal Zenith Angle (deg)'
911 669 self.ylabel = 'Meridional Zenith Angle (deg)'
912 670
913 671 if self.figure is None:
914 672 self.figure = plt.figure(figsize=(self.width, self.height),
915 673 edgecolor='k',
916 674 facecolor='w')
917 675 else:
918 676 self.figure.clf()
919 677
920 678 self.ax = plt.subplot2grid((self.nrows, self.ncols), (0, 0), 1, 1, polar=True)
921 679 self.ax.firsttime = True
922 680
923 681
924 682 def plot(self):
925 683
926 arrayParameters = np.concatenate([self.data['param'][t] for t in self.times])
684 arrayParameters = numpy.concatenate([self.data['param'][t] for t in self.data.times])
927 685 error = arrayParameters[:,-1]
928 686 indValid = numpy.where(error == 0)[0]
929 687 finalMeteor = arrayParameters[indValid,:]
930 688 finalAzimuth = finalMeteor[:,3]
931 689 finalZenith = finalMeteor[:,4]
932 690
933 691 x = finalAzimuth*numpy.pi/180
934 692 y = finalZenith
935 693
936 694 if self.ax.firsttime:
937 695 self.ax.plot = self.ax.plot(x, y, 'bo', markersize=5)[0]
938 696 self.ax.set_ylim(0,90)
939 697 self.ax.set_yticks(numpy.arange(0,90,20))
940 698 self.ax.set_xlabel(self.xlabel)
941 699 self.ax.set_ylabel(self.ylabel)
942 700 self.ax.yaxis.labelpad = 40
943 701 self.ax.firsttime = False
944 702 else:
945 703 self.ax.plot.set_data(x, y)
946 704
947 705
948 706 dt1 = datetime.datetime.fromtimestamp(self.min_time).strftime('%y/%m/%d %H:%M:%S')
949 707 dt2 = datetime.datetime.fromtimestamp(self.max_time).strftime('%y/%m/%d %H:%M:%S')
950 708 title = 'Meteor Detection Sky Map\n %s - %s \n Number of events: %5.0f\n' % (dt1,
951 709 dt2,
952 710 len(x))
953 711 self.ax.set_title(title, size=8)
954 712
955 713 self.saveTime = self.max_time
714
715 class PlotParamData(PlotRTIData):
716 '''
717 Plot for data_param object
718 '''
719
720 CODE = 'param'
721 colormap = 'seismic'
722
723 def setup(self):
724 self.xaxis = 'time'
725 self.ncols = 1
726 self.nrows = self.data.shape(self.CODE)[0]
727 self.nplots = self.nrows
728 if self.showSNR:
729 self.nrows += 1
730
731 self.ylabel = 'Height [Km]'
732 self.titles = self.data.parameters \
733 if self.data.parameters else ['Param {}'.format(x) for x in xrange(self.nrows)]
734 if self.showSNR:
735 self.titles.append('SNR')
736
737 def plot(self):
738 self.data.normalize_heights()
739 self.x = self.data.times
740 self.y = self.data.heights
741 if self.showSNR:
742 self.z = numpy.concatenate(
743 (self.data[self.CODE], self.data['snr'])
744 )
745 else:
746 self.z = self.data[self.CODE]
747
748 self.z = numpy.ma.masked_invalid(self.z)
749
750 for n, ax in enumerate(self.axes):
751
752 x, y, z = self.fill_gaps(*self.decimate())
753
754 if ax.firsttime:
755 if self.zlimits is not None:
756 self.zmin, self.zmax = self.zlimits[n]
757 self.zmax = self.zmax if self.zmax is not None else numpy.nanmax(abs(self.z[:-1, :]))
758 self.zmin = self.zmin if self.zmin is not None else -self.zmax
759 ax.plt = ax.pcolormesh(x, y, z[n, :, :].T*self.factors[n],
760 vmin=self.zmin,
761 vmax=self.zmax,
762 cmap=self.cmaps[n]
763 )
764 else:
765 if self.zlimits is not None:
766 self.zmin, self.zmax = self.zlimits[n]
767 ax.collections.remove(ax.collections[0])
768 ax.plt = ax.pcolormesh(x, y, z[n, :, :].T*self.factors[n],
769 vmin=self.zmin,
770 vmax=self.zmax,
771 cmap=self.cmaps[n]
772 )
773
774 self.saveTime = self.min_time
775
776 class PlotOuputData(PlotParamData):
777 '''
778 Plot data_output object
779 '''
780
781 CODE = 'output'
782 colormap = 'seismic' No newline at end of file
Show file before | Show file after | Hide comments
@@ -1,2154 +1,2154
1 1 import os
2 2 import datetime
3 3 import numpy
4 4 import inspect
5 5 from figure import Figure, isRealtime, isTimeInHourRange
6 6 from plotting_codes import *
7 7
8 8
9 9 class FitGauPlot(Figure):
10 10
11 11 isConfig = None
12 12 __nsubplots = None
13 13
14 14 WIDTHPROF = None
15 15 HEIGHTPROF = None
16 16 PREFIX = 'fitgau'
17 17
18 18 def __init__(self, **kwargs):
19 19 Figure.__init__(self, **kwargs)
20 20 self.isConfig = False
21 21 self.__nsubplots = 1
22 22
23 23 self.WIDTH = 250
24 24 self.HEIGHT = 250
25 25 self.WIDTHPROF = 120
26 26 self.HEIGHTPROF = 0
27 27 self.counter_imagwr = 0
28 28
29 29 self.PLOT_CODE = SPEC_CODE
30 30
31 31 self.FTP_WEI = None
32 32 self.EXP_CODE = None
33 33 self.SUB_EXP_CODE = None
34 34 self.PLOT_POS = None
35 35
36 36 self.__xfilter_ena = False
37 37 self.__yfilter_ena = False
38 38
39 39 def getSubplots(self):
40 40
41 41 ncol = int(numpy.sqrt(self.nplots)+0.9)
42 42 nrow = int(self.nplots*1./ncol + 0.9)
43 43
44 44 return nrow, ncol
45 45
46 46 def setup(self, id, nplots, wintitle, showprofile=True, show=True):
47 47
48 48 self.__showprofile = showprofile
49 49 self.nplots = nplots
50 50
51 51 ncolspan = 1
52 52 colspan = 1
53 53 if showprofile:
54 54 ncolspan = 3
55 55 colspan = 2
56 56 self.__nsubplots = 2
57 57
58 58 self.createFigure(id = id,
59 59 wintitle = wintitle,
60 60 widthplot = self.WIDTH + self.WIDTHPROF,
61 61 heightplot = self.HEIGHT + self.HEIGHTPROF,
62 62 show=show)
63 63
64 64 nrow, ncol = self.getSubplots()
65 65
66 66 counter = 0
67 67 for y in range(nrow):
68 68 for x in range(ncol):
69 69
70 70 if counter >= self.nplots:
71 71 break
72 72
73 73 self.addAxes(nrow, ncol*ncolspan, y, x*ncolspan, colspan, 1)
74 74
75 75 if showprofile:
76 76 self.addAxes(nrow, ncol*ncolspan, y, x*ncolspan+colspan, 1, 1)
77 77
78 78 counter += 1
79 79
80 80 def run(self, dataOut, id, wintitle="", channelList=None, showprofile=True,
81 81 xmin=None, xmax=None, ymin=None, ymax=None, zmin=None, zmax=None,
82 82 save=False, figpath='./', figfile=None, show=True, ftp=False, wr_period=1,
83 83 server=None, folder=None, username=None, password=None,
84 84 ftp_wei=0, exp_code=0, sub_exp_code=0, plot_pos=0, realtime=False,
85 85 xaxis="frequency", colormap='jet', normFactor=None , GauSelector = 1):
86 86
87 87 """
88 88
89 89 Input:
90 90 dataOut :
91 91 id :
92 92 wintitle :
93 93 channelList :
94 94 showProfile :
95 95 xmin : None,
96 96 xmax : None,
97 97 ymin : None,
98 98 ymax : None,
99 99 zmin : None,
100 100 zmax : None
101 101 """
102 102 if realtime:
103 103 if not(isRealtime(utcdatatime = dataOut.utctime)):
104 104 print 'Skipping this plot function'
105 105 return
106 106
107 107 if channelList == None:
108 108 channelIndexList = dataOut.channelIndexList
109 109 else:
110 110 channelIndexList = []
111 111 for channel in channelList:
112 112 if channel not in dataOut.channelList:
113 113 raise ValueError, "Channel %d is not in dataOut.channelList" %channel
114 114 channelIndexList.append(dataOut.channelList.index(channel))
115 115
116 116 # if normFactor is None:
117 117 # factor = dataOut.normFactor
118 118 # else:
119 119 # factor = normFactor
120 120 if xaxis == "frequency":
121 121 x = dataOut.spc_range[0]
122 122 xlabel = "Frequency (kHz)"
123 123
124 124 elif xaxis == "time":
125 125 x = dataOut.spc_range[1]
126 126 xlabel = "Time (ms)"
127 127
128 128 else:
129 129 x = dataOut.spc_range[2]
130 130 xlabel = "Velocity (m/s)"
131 131
132 132 ylabel = "Range (Km)"
133 133
134 134 y = dataOut.getHeiRange()
135 135
136 136 z = dataOut.GauSPC[:,GauSelector,:,:] #GauSelector] #dataOut.data_spc/factor
137 137 print 'GausSPC', z[0,32,10:40]
138 138 z = numpy.where(numpy.isfinite(z), z, numpy.NAN)
139 139 zdB = 10*numpy.log10(z)
140 140
141 141 avg = numpy.average(z, axis=1)
142 142 avgdB = 10*numpy.log10(avg)
143 143
144 144 noise = dataOut.spc_noise
145 145 noisedB = 10*numpy.log10(noise)
146 146
147 147 thisDatetime = datetime.datetime.utcfromtimestamp(dataOut.getTimeRange()[0])
148 148 title = wintitle + " Spectra"
149 149 if ((dataOut.azimuth!=None) and (dataOut.zenith!=None)):
150 150 title = title + '_' + 'azimuth,zenith=%2.2f,%2.2f'%(dataOut.azimuth, dataOut.zenith)
151 151
152 152 if not self.isConfig:
153 153
154 154 nplots = len(channelIndexList)
155 155
156 156 self.setup(id=id,
157 157 nplots=nplots,
158 158 wintitle=wintitle,
159 159 showprofile=showprofile,
160 160 show=show)
161 161
162 162 if xmin == None: xmin = numpy.nanmin(x)
163 163 if xmax == None: xmax = numpy.nanmax(x)
164 164 if ymin == None: ymin = numpy.nanmin(y)
165 165 if ymax == None: ymax = numpy.nanmax(y)
166 166 if zmin == None: zmin = numpy.floor(numpy.nanmin(noisedB)) - 3
167 167 if zmax == None: zmax = numpy.ceil(numpy.nanmax(avgdB)) + 3
168 168
169 169 self.FTP_WEI = ftp_wei
170 170 self.EXP_CODE = exp_code
171 171 self.SUB_EXP_CODE = sub_exp_code
172 172 self.PLOT_POS = plot_pos
173 173
174 174 self.isConfig = True
175 175
176 176 self.setWinTitle(title)
177 177
178 178 for i in range(self.nplots):
179 179 index = channelIndexList[i]
180 180 str_datetime = '%s %s'%(thisDatetime.strftime("%Y/%m/%d"),thisDatetime.strftime("%H:%M:%S"))
181 181 title = "Channel %d: %4.2fdB: %s" %(dataOut.channelList[index], noisedB[index], str_datetime)
182 182 if len(dataOut.beam.codeList) != 0:
183 183 title = "Ch%d:%4.2fdB,%2.2f,%2.2f:%s" %(dataOut.channelList[index], noisedB[index], dataOut.beam.azimuthList[index], dataOut.beam.zenithList[index], str_datetime)
184 184
185 185 axes = self.axesList[i*self.__nsubplots]
186 186 axes.pcolor(x, y, zdB[index,:,:],
187 187 xmin=xmin, xmax=xmax, ymin=ymin, ymax=ymax, zmin=zmin, zmax=zmax,
188 188 xlabel=xlabel, ylabel=ylabel, title=title, colormap=colormap,
189 189 ticksize=9, cblabel='')
190 190
191 191 if self.__showprofile:
192 192 axes = self.axesList[i*self.__nsubplots +1]
193 193 axes.pline(avgdB[index,:], y,
194 194 xmin=zmin, xmax=zmax, ymin=ymin, ymax=ymax,
195 195 xlabel='dB', ylabel='', title='',
196 196 ytick_visible=False,
197 197 grid='x')
198 198
199 199 noiseline = numpy.repeat(noisedB[index], len(y))
200 200 axes.addpline(noiseline, y, idline=1, color="black", linestyle="dashed", lw=2)
201 201
202 202 self.draw()
203 203
204 204 if figfile == None:
205 205 str_datetime = thisDatetime.strftime("%Y%m%d_%H%M%S")
206 206 name = str_datetime
207 207 if ((dataOut.azimuth!=None) and (dataOut.zenith!=None)):
208 208 name = name + '_az' + '_%2.2f'%(dataOut.azimuth) + '_zn' + '_%2.2f'%(dataOut.zenith)
209 209 figfile = self.getFilename(name)
210 210
211 211 self.save(figpath=figpath,
212 212 figfile=figfile,
213 213 save=save,
214 214 ftp=ftp,
215 215 wr_period=wr_period,
216 216 thisDatetime=thisDatetime)
217 217
218 218
219 219
220 220 class MomentsPlot(Figure):
221 221
222 222 isConfig = None
223 223 __nsubplots = None
224 224
225 225 WIDTHPROF = None
226 226 HEIGHTPROF = None
227 227 PREFIX = 'prm'
228
229 228 def __init__(self, **kwargs):
230 229 Figure.__init__(self, **kwargs)
231 230 self.isConfig = False
232 231 self.__nsubplots = 1
233 232
234 233 self.WIDTH = 280
235 234 self.HEIGHT = 250
236 235 self.WIDTHPROF = 120
237 236 self.HEIGHTPROF = 0
238 237 self.counter_imagwr = 0
239 238
240 239 self.PLOT_CODE = MOMENTS_CODE
241 240
242 241 self.FTP_WEI = None
243 242 self.EXP_CODE = None
244 243 self.SUB_EXP_CODE = None
245 244 self.PLOT_POS = None
246 245
247 246 def getSubplots(self):
248 247
249 248 ncol = int(numpy.sqrt(self.nplots)+0.9)
250 249 nrow = int(self.nplots*1./ncol + 0.9)
251 250
252 251 return nrow, ncol
253 252
254 253 def setup(self, id, nplots, wintitle, showprofile=True, show=True):
255 254
256 255 self.__showprofile = showprofile
257 256 self.nplots = nplots
258 257
259 258 ncolspan = 1
260 259 colspan = 1
261 260 if showprofile:
262 261 ncolspan = 3
263 262 colspan = 2
264 263 self.__nsubplots = 2
265 264
266 265 self.createFigure(id = id,
267 266 wintitle = wintitle,
268 267 widthplot = self.WIDTH + self.WIDTHPROF,
269 268 heightplot = self.HEIGHT + self.HEIGHTPROF,
270 269 show=show)
271 270
272 271 nrow, ncol = self.getSubplots()
273 272
274 273 counter = 0
275 274 for y in range(nrow):
276 275 for x in range(ncol):
277 276
278 277 if counter >= self.nplots:
279 278 break
280 279
281 280 self.addAxes(nrow, ncol*ncolspan, y, x*ncolspan, colspan, 1)
282 281
283 282 if showprofile:
284 283 self.addAxes(nrow, ncol*ncolspan, y, x*ncolspan+colspan, 1, 1)
285 284
286 285 counter += 1
287 286
288 287 def run(self, dataOut, id, wintitle="", channelList=None, showprofile=True,
289 288 xmin=None, xmax=None, ymin=None, ymax=None, zmin=None, zmax=None,
290 289 save=False, figpath='./', figfile=None, show=True, ftp=False, wr_period=1,
291 290 server=None, folder=None, username=None, password=None,
292 291 ftp_wei=0, exp_code=0, sub_exp_code=0, plot_pos=0, realtime=False):
293 292
294 293 """
295 294
296 295 Input:
297 296 dataOut :
298 297 id :
299 298 wintitle :
300 299 channelList :
301 300 showProfile :
302 301 xmin : None,
303 302 xmax : None,
304 303 ymin : None,
305 304 ymax : None,
306 305 zmin : None,
307 306 zmax : None
308 307 """
309 308
310 309 if dataOut.flagNoData:
311 310 return None
312 311
313 312 if realtime:
314 313 if not(isRealtime(utcdatatime = dataOut.utctime)):
315 314 print 'Skipping this plot function'
316 315 return
317 316
318 317 if channelList == None:
319 318 channelIndexList = dataOut.channelIndexList
320 319 else:
321 320 channelIndexList = []
322 321 for channel in channelList:
323 322 if channel not in dataOut.channelList:
324 323 raise ValueError, "Channel %d is not in dataOut.channelList"
325 324 channelIndexList.append(dataOut.channelList.index(channel))
326 325
327 326 factor = dataOut.normFactor
328 327 x = dataOut.abscissaList
329 328 y = dataOut.heightList
330 329
331 330 z = dataOut.data_pre[channelIndexList,:,:]/factor
332 331 z = numpy.where(numpy.isfinite(z), z, numpy.NAN)
333 332 avg = numpy.average(z, axis=1)
334 333 noise = dataOut.noise/factor
335 334
336 335 zdB = 10*numpy.log10(z)
337 336 avgdB = 10*numpy.log10(avg)
338 337 noisedB = 10*numpy.log10(noise)
339 338
340 339 #thisDatetime = dataOut.datatime
341 340 thisDatetime = datetime.datetime.utcfromtimestamp(dataOut.getTimeRange()[0])
342 341 title = wintitle + " Parameters"
343 342 xlabel = "Velocity (m/s)"
344 343 ylabel = "Range (Km)"
345 344
346 345 update_figfile = False
347 346
348 347 if not self.isConfig:
349 348
350 349 nplots = len(channelIndexList)
351 350
352 351 self.setup(id=id,
353 352 nplots=nplots,
354 353 wintitle=wintitle,
355 354 showprofile=showprofile,
356 355 show=show)
357 356
358 357 if xmin == None: xmin = numpy.nanmin(x)
359 358 if xmax == None: xmax = numpy.nanmax(x)
360 359 if ymin == None: ymin = numpy.nanmin(y)
361 360 if ymax == None: ymax = numpy.nanmax(y)
362 361 if zmin == None: zmin = numpy.nanmin(avgdB)*0.9
363 362 if zmax == None: zmax = numpy.nanmax(avgdB)*0.9
364 363
365 364 self.FTP_WEI = ftp_wei
366 365 self.EXP_CODE = exp_code
367 366 self.SUB_EXP_CODE = sub_exp_code
368 367 self.PLOT_POS = plot_pos
369 368
370 369 self.isConfig = True
371 370 update_figfile = True
372 371
373 372 self.setWinTitle(title)
374 373
375 374 for i in range(self.nplots):
376 375 str_datetime = '%s %s'%(thisDatetime.strftime("%Y/%m/%d"),thisDatetime.strftime("%H:%M:%S"))
377 376 title = "Channel %d: %4.2fdB: %s" %(dataOut.channelList[i], noisedB[i], str_datetime)
378 377 axes = self.axesList[i*self.__nsubplots]
379 378 axes.pcolor(x, y, zdB[i,:,:],
380 379 xmin=xmin, xmax=xmax, ymin=ymin, ymax=ymax, zmin=zmin, zmax=zmax,
381 380 xlabel=xlabel, ylabel=ylabel, title=title,
382 381 ticksize=9, cblabel='')
383 382 #Mean Line
384 383 mean = dataOut.data_param[i, 1, :]
385 384 axes.addpline(mean, y, idline=0, color="black", linestyle="solid", lw=1)
386 385
387 386 if self.__showprofile:
388 387 axes = self.axesList[i*self.__nsubplots +1]
389 388 axes.pline(avgdB[i], y,
390 389 xmin=zmin, xmax=zmax, ymin=ymin, ymax=ymax,
391 390 xlabel='dB', ylabel='', title='',
392 391 ytick_visible=False,
393 392 grid='x')
394 393
395 394 noiseline = numpy.repeat(noisedB[i], len(y))
396 395 axes.addpline(noiseline, y, idline=1, color="black", linestyle="dashed", lw=2)
397 396
398 397 self.draw()
399 398
400 399 self.save(figpath=figpath,
401 400 figfile=figfile,
402 401 save=save,
403 402 ftp=ftp,
404 403 wr_period=wr_period,
405 404 thisDatetime=thisDatetime)
406 405
407 406
408 407
409 408 class SkyMapPlot(Figure):
410 409
411 410 __isConfig = None
412 411 __nsubplots = None
413 412
414 413 WIDTHPROF = None
415 414 HEIGHTPROF = None
416 415 PREFIX = 'mmap'
417 416
418 417 def __init__(self, **kwargs):
419 418 Figure.__init__(self, **kwargs)
420 419 self.isConfig = False
421 420 self.__nsubplots = 1
422 421
423 422 # self.WIDTH = 280
424 423 # self.HEIGHT = 250
425 424 self.WIDTH = 600
426 425 self.HEIGHT = 600
427 426 self.WIDTHPROF = 120
428 427 self.HEIGHTPROF = 0
429 428 self.counter_imagwr = 0
430 429
431 430 self.PLOT_CODE = MSKYMAP_CODE
432 431
433 432 self.FTP_WEI = None
434 433 self.EXP_CODE = None
435 434 self.SUB_EXP_CODE = None
436 435 self.PLOT_POS = None
437 436
438 437 def getSubplots(self):
439 438
440 439 ncol = int(numpy.sqrt(self.nplots)+0.9)
441 440 nrow = int(self.nplots*1./ncol + 0.9)
442 441
443 442 return nrow, ncol
444 443
445 444 def setup(self, id, nplots, wintitle, showprofile=False, show=True):
446 445
447 446 self.__showprofile = showprofile
448 447 self.nplots = nplots
449 448
450 449 ncolspan = 1
451 450 colspan = 1
452 451
453 452 self.createFigure(id = id,
454 453 wintitle = wintitle,
455 454 widthplot = self.WIDTH, #+ self.WIDTHPROF,
456 455 heightplot = self.HEIGHT,# + self.HEIGHTPROF,
457 456 show=show)
458 457
459 458 nrow, ncol = 1,1
460 459 counter = 0
461 460 x = 0
462 461 y = 0
463 462 self.addAxes(1, 1, 0, 0, 1, 1, True)
464 463
465 464 def run(self, dataOut, id, wintitle="", channelList=None, showprofile=False,
466 465 tmin=0, tmax=24, timerange=None,
467 466 save=False, figpath='./', figfile=None, show=True, ftp=False, wr_period=1,
468 467 server=None, folder=None, username=None, password=None,
469 468 ftp_wei=0, exp_code=0, sub_exp_code=0, plot_pos=0, realtime=False):
470 469
471 470 """
472 471
473 472 Input:
474 473 dataOut :
475 474 id :
476 475 wintitle :
477 476 channelList :
478 477 showProfile :
479 478 xmin : None,
480 479 xmax : None,
481 480 ymin : None,
482 481 ymax : None,
483 482 zmin : None,
484 483 zmax : None
485 484 """
486 485
487 486 arrayParameters = dataOut.data_param
488 487 error = arrayParameters[:,-1]
489 488 indValid = numpy.where(error == 0)[0]
490 489 finalMeteor = arrayParameters[indValid,:]
491 490 finalAzimuth = finalMeteor[:,3]
492 491 finalZenith = finalMeteor[:,4]
493 492
494 493 x = finalAzimuth*numpy.pi/180
495 494 y = finalZenith
496 495 x1 = [dataOut.ltctime, dataOut.ltctime]
497 496
498 497 #thisDatetime = dataOut.datatime
499 498 thisDatetime = datetime.datetime.utcfromtimestamp(dataOut.ltctime)
500 499 title = wintitle + " Parameters"
501 500 xlabel = "Zonal Zenith Angle (deg) "
502 501 ylabel = "Meridional Zenith Angle (deg)"
503 502 update_figfile = False
504 503
505 504 if not self.isConfig:
506 505
507 506 nplots = 1
508 507
509 508 self.setup(id=id,
510 509 nplots=nplots,
511 510 wintitle=wintitle,
512 511 showprofile=showprofile,
513 512 show=show)
514 513
515 514 if self.xmin is None and self.xmax is None:
516 515 self.xmin, self.xmax = self.getTimeLim(x1, tmin, tmax, timerange)
517 516
518 517 if timerange != None:
519 518 self.timerange = timerange
520 519 else:
521 520 self.timerange = self.xmax - self.xmin
522 521
523 522 self.FTP_WEI = ftp_wei
524 523 self.EXP_CODE = exp_code
525 524 self.SUB_EXP_CODE = sub_exp_code
526 525 self.PLOT_POS = plot_pos
527 526 self.name = thisDatetime.strftime("%Y%m%d_%H%M%S")
528 527 self.firstdate = '%s %s'%(thisDatetime.strftime("%Y/%m/%d"),thisDatetime.strftime("%H:%M:%S"))
529 528 self.isConfig = True
530 529 update_figfile = True
531 530
532 531 self.setWinTitle(title)
533 532
534 533 i = 0
535 534 str_datetime = '%s %s'%(thisDatetime.strftime("%Y/%m/%d"),thisDatetime.strftime("%H:%M:%S"))
536 535
537 536 axes = self.axesList[i*self.__nsubplots]
538 537 nevents = axes.x_buffer.shape[0] + x.shape[0]
539 538 title = "Meteor Detection Sky Map\n %s - %s \n Number of events: %5.0f\n" %(self.firstdate,str_datetime,nevents)
540 539 axes.polar(x, y,
541 540 title=title, xlabel=xlabel, ylabel=ylabel,
542 541 ticksize=9, cblabel='')
543 542
544 543 self.draw()
545 544
546 545 self.save(figpath=figpath,
547 546 figfile=figfile,
548 547 save=save,
549 548 ftp=ftp,
550 549 wr_period=wr_period,
551 550 thisDatetime=thisDatetime,
552 551 update_figfile=update_figfile)
553 552
554 553 if dataOut.ltctime >= self.xmax:
555 554 self.isConfigmagwr = wr_period
556 555 self.isConfig = False
557 556 update_figfile = True
558 557 axes.__firsttime = True
559 558 self.xmin += self.timerange
560 559 self.xmax += self.timerange
561 560
562 561
563 562
564 563
565 564 class WindProfilerPlot(Figure):
566 565
567 566 __isConfig = None
568 567 __nsubplots = None
569 568
570 569 WIDTHPROF = None
571 570 HEIGHTPROF = None
572 571 PREFIX = 'wind'
573 572
574 573 def __init__(self, **kwargs):
575 574 Figure.__init__(self, **kwargs)
576 575 self.timerange = None
577 576 self.isConfig = False
578 577 self.__nsubplots = 1
579 578
580 579 self.WIDTH = 800
581 580 self.HEIGHT = 300
582 581 self.WIDTHPROF = 120
583 582 self.HEIGHTPROF = 0
584 583 self.counter_imagwr = 0
585 584
586 585 self.PLOT_CODE = WIND_CODE
587 586
588 587 self.FTP_WEI = None
589 588 self.EXP_CODE = None
590 589 self.SUB_EXP_CODE = None
591 590 self.PLOT_POS = None
592 591 self.tmin = None
593 592 self.tmax = None
594 593
595 594 self.xmin = None
596 595 self.xmax = None
597 596
598 597 self.figfile = None
599 598
600 599 def getSubplots(self):
601 600
602 601 ncol = 1
603 602 nrow = self.nplots
604 603
605 604 return nrow, ncol
606 605
607 606 def setup(self, id, nplots, wintitle, showprofile=True, show=True):
608 607
609 608 self.__showprofile = showprofile
610 609 self.nplots = nplots
611 610
612 611 ncolspan = 1
613 612 colspan = 1
614 613
615 614 self.createFigure(id = id,
616 615 wintitle = wintitle,
617 616 widthplot = self.WIDTH + self.WIDTHPROF,
618 617 heightplot = self.HEIGHT + self.HEIGHTPROF,
619 618 show=show)
620 619
621 620 nrow, ncol = self.getSubplots()
622 621
623 622 counter = 0
624 623 for y in range(nrow):
625 624 if counter >= self.nplots:
626 625 break
627 626
628 627 self.addAxes(nrow, ncol*ncolspan, y, 0, colspan, 1)
629 628 counter += 1
630 629
631 630 def run(self, dataOut, id, wintitle="", channelList=None, showprofile='False',
632 631 xmin=None, xmax=None, ymin=None, ymax=None, zmin=None, zmax=None,
633 632 zmax_ver = None, zmin_ver = None, SNRmin = None, SNRmax = None,
634 633 timerange=None, SNRthresh = None,
635 634 save=False, figpath='./', lastone=0,figfile=None, ftp=False, wr_period=1, show=True,
636 635 server=None, folder=None, username=None, password=None,
637 636 ftp_wei=0, exp_code=0, sub_exp_code=0, plot_pos=0):
638 637 """
639 638
640 639 Input:
641 640 dataOut :
642 641 id :
643 642 wintitle :
644 643 channelList :
645 644 showProfile :
646 645 xmin : None,
647 646 xmax : None,
648 647 ymin : None,
649 648 ymax : None,
650 649 zmin : None,
651 650 zmax : None
652 651 """
653 652
654 653 # if timerange is not None:
655 654 # self.timerange = timerange
656 655 #
657 656 # tmin = None
658 657 # tmax = None
659 658
660 659 x = dataOut.getTimeRange1(dataOut.paramInterval)
661 660 y = dataOut.heightList
662 661 z = dataOut.data_output.copy()
663 662 nplots = z.shape[0] #Number of wind dimensions estimated
664 663 nplotsw = nplots
665 664
666 665
667 666 #If there is a SNR function defined
668 667 if dataOut.data_SNR is not None:
669 668 nplots += 1
670 669 SNR = dataOut.data_SNR
671 670 SNRavg = numpy.average(SNR, axis=0)
672 671
673 672 SNRdB = 10*numpy.log10(SNR)
674 673 SNRavgdB = 10*numpy.log10(SNRavg)
675 674
676 675 if SNRthresh == None: SNRthresh = -5.0
677 676 ind = numpy.where(SNRavg < 10**(SNRthresh/10))[0]
678 677
679 678 for i in range(nplotsw):
680 679 z[i,ind] = numpy.nan
681 680
682 681 thisDatetime = datetime.datetime.utcfromtimestamp(dataOut.ltctime)
683 682 #thisDatetime = datetime.datetime.now()
684 683 title = wintitle + "Wind"
685 684 xlabel = ""
686 685 ylabel = "Height (km)"
687 686 update_figfile = False
688 687
689 688 if not self.isConfig:
690 689
691 690 self.setup(id=id,
692 691 nplots=nplots,
693 692 wintitle=wintitle,
694 693 showprofile=showprofile,
695 694 show=show)
696 695
697 696 if timerange is not None:
698 697 self.timerange = timerange
699 698
700 699 self.xmin, self.xmax = self.getTimeLim(x, xmin, xmax, timerange)
701 700
702 701 if ymin == None: ymin = numpy.nanmin(y)
703 702 if ymax == None: ymax = numpy.nanmax(y)
704 703
705 704 if zmax == None: zmax = numpy.nanmax(abs(z[range(2),:]))
706 705 #if numpy.isnan(zmax): zmax = 50
707 706 if zmin == None: zmin = -zmax
708 707
709 708 if nplotsw == 3:
710 709 if zmax_ver == None: zmax_ver = numpy.nanmax(abs(z[2,:]))
711 710 if zmin_ver == None: zmin_ver = -zmax_ver
712 711
713 712 if dataOut.data_SNR is not None:
714 713 if SNRmin == None: SNRmin = numpy.nanmin(SNRavgdB)
715 714 if SNRmax == None: SNRmax = numpy.nanmax(SNRavgdB)
716 715
717 716
718 717 self.FTP_WEI = ftp_wei
719 718 self.EXP_CODE = exp_code
720 719 self.SUB_EXP_CODE = sub_exp_code
721 720 self.PLOT_POS = plot_pos
722 721
723 722 self.name = thisDatetime.strftime("%Y%m%d_%H%M%S")
724 723 self.isConfig = True
725 724 self.figfile = figfile
726 725 update_figfile = True
727 726
728 727 self.setWinTitle(title)
729 728
730 729 if ((self.xmax - x[1]) < (x[1]-x[0])):
731 730 x[1] = self.xmax
732 731
733 732 strWind = ['Zonal', 'Meridional', 'Vertical']
734 733 strCb = ['Velocity (m/s)','Velocity (m/s)','Velocity (cm/s)']
735 734 zmaxVector = [zmax, zmax, zmax_ver]
736 735 zminVector = [zmin, zmin, zmin_ver]
737 736 windFactor = [1,1,100]
738 737
739 738 for i in range(nplotsw):
740 739
741 740 title = "%s Wind: %s" %(strWind[i], thisDatetime.strftime("%Y/%m/%d %H:%M:%S"))
742 741 axes = self.axesList[i*self.__nsubplots]
743 742
744 743 z1 = z[i,:].reshape((1,-1))*windFactor[i]
745 744 #z1=numpy.ma.masked_where(z1==0.,z1)
746 745
747 746 axes.pcolorbuffer(x, y, z1,
748 747 xmin=self.xmin, xmax=self.xmax, ymin=ymin, ymax=ymax, zmin=zminVector[i], zmax=zmaxVector[i],
749 748 xlabel=xlabel, ylabel=ylabel, title=title, rti=True, XAxisAsTime=True,
750 749 ticksize=9, cblabel=strCb[i], cbsize="1%", colormap="seismic" )
751 750
752 751 if dataOut.data_SNR is not None:
753 752 i += 1
754 753 title = "Signal Noise Ratio (SNR): %s" %(thisDatetime.strftime("%Y/%m/%d %H:%M:%S"))
755 754 axes = self.axesList[i*self.__nsubplots]
756 755 SNRavgdB = SNRavgdB.reshape((1,-1))
757 756 axes.pcolorbuffer(x, y, SNRavgdB,
758 757 xmin=self.xmin, xmax=self.xmax, ymin=ymin, ymax=ymax, zmin=SNRmin, zmax=SNRmax,
759 758 xlabel=xlabel, ylabel=ylabel, title=title, rti=True, XAxisAsTime=True,
760 759 ticksize=9, cblabel='', cbsize="1%", colormap="jet")
761 760
762 761 self.draw()
763 762
764 763 self.save(figpath=figpath,
765 764 figfile=figfile,
766 765 save=save,
767 766 ftp=ftp,
768 767 wr_period=wr_period,
769 768 thisDatetime=thisDatetime,
770 769 update_figfile=update_figfile)
771 770
772 771 if dataOut.ltctime + dataOut.paramInterval >= self.xmax:
773 772 self.counter_imagwr = wr_period
774 773 self.isConfig = False
775 774 update_figfile = True
776 775
777 776
778 777 class ParametersPlot(Figure):
779 778
780 779 __isConfig = None
781 780 __nsubplots = None
782 781
783 782 WIDTHPROF = None
784 783 HEIGHTPROF = None
785 784 PREFIX = 'param'
786 785
787 786 nplots = None
788 787 nchan = None
789 788
790 789 def __init__(self, **kwargs):
791 790 Figure.__init__(self, **kwargs)
792 791 self.timerange = None
793 792 self.isConfig = False
794 793 self.__nsubplots = 1
795 794
796 795 self.WIDTH = 800
797 796 self.HEIGHT = 180
798 797 self.WIDTHPROF = 120
799 798 self.HEIGHTPROF = 0
800 799 self.counter_imagwr = 0
801 800
802 801 self.PLOT_CODE = RTI_CODE
803 802
804 803 self.FTP_WEI = None
805 804 self.EXP_CODE = None
806 805 self.SUB_EXP_CODE = None
807 806 self.PLOT_POS = None
808 807 self.tmin = None
809 808 self.tmax = None
810 809
811 810 self.xmin = None
812 811 self.xmax = None
813 812
814 813 self.figfile = None
815 814
816 815 def getSubplots(self):
817 816
818 817 ncol = 1
819 818 nrow = self.nplots
820 819
821 820 return nrow, ncol
822 821
823 822 def setup(self, id, nplots, wintitle, show=True):
824 823
825 824 self.nplots = nplots
826 825
827 826 ncolspan = 1
828 827 colspan = 1
829 828
830 829 self.createFigure(id = id,
831 830 wintitle = wintitle,
832 831 widthplot = self.WIDTH + self.WIDTHPROF,
833 832 heightplot = self.HEIGHT + self.HEIGHTPROF,
834 833 show=show)
835 834
836 835 nrow, ncol = self.getSubplots()
837 836
838 837 counter = 0
839 838 for y in range(nrow):
840 839 for x in range(ncol):
841 840
842 841 if counter >= self.nplots:
843 842 break
844 843
845 844 self.addAxes(nrow, ncol*ncolspan, y, x*ncolspan, colspan, 1)
846 845
847 846 counter += 1
848 847
849 848 def run(self, dataOut, id, wintitle="", channelList=None, paramIndex = 0, colormap="jet",
850 849 xmin=None, xmax=None, ymin=None, ymax=None, zmin=None, zmax=None, timerange=None,
851 850 showSNR=False, SNRthresh = -numpy.inf, SNRmin=None, SNRmax=None,
852 851 save=False, figpath='./', lastone=0,figfile=None, ftp=False, wr_period=1, show=True,
853 852 server=None, folder=None, username=None, password=None,
854 853 ftp_wei=0, exp_code=0, sub_exp_code=0, plot_pos=0, HEIGHT=None):
855 854 """
856 855
857 856 Input:
858 857 dataOut :
859 858 id :
860 859 wintitle :
861 860 channelList :
862 861 showProfile :
863 862 xmin : None,
864 863 xmax : None,
865 864 ymin : None,
866 865 ymax : None,
867 866 zmin : None,
868 867 zmax : None
869 868 """
870 869
871 870 if HEIGHT is not None:
872 871 self.HEIGHT = HEIGHT
873 872
874 873
875 874 if not isTimeInHourRange(dataOut.datatime, xmin, xmax):
876 875 return
877 876
878 877 if channelList == None:
879 878 channelIndexList = range(dataOut.data_param.shape[0])
880 879 else:
881 880 channelIndexList = []
882 881 for channel in channelList:
883 882 if channel not in dataOut.channelList:
884 883 raise ValueError, "Channel %d is not in dataOut.channelList"
885 884 channelIndexList.append(dataOut.channelList.index(channel))
886 885
887 886 x = dataOut.getTimeRange1(dataOut.paramInterval)
888 887 y = dataOut.getHeiRange()
889 888
890 889 if dataOut.data_param.ndim == 3:
891 890 z = dataOut.data_param[channelIndexList,paramIndex,:]
892 891 else:
893 892 z = dataOut.data_param[channelIndexList,:]
894 893
895 894 if showSNR:
896 895 #SNR data
897 896 SNRarray = dataOut.data_SNR[channelIndexList,:]
898 897 SNRdB = 10*numpy.log10(SNRarray)
899 898 ind = numpy.where(SNRdB < SNRthresh)
900 899 z[ind] = numpy.nan
901 900
902 901 thisDatetime = dataOut.datatime
903 902 # thisDatetime = datetime.datetime.utcfromtimestamp(dataOut.getTimeRange()[0])
904 903 title = wintitle + " Parameters Plot" #: %s" %(thisDatetime.strftime("%d-%b-%Y"))
905 904 xlabel = ""
906 905 ylabel = "Range (Km)"
907 906
908 907 update_figfile = False
909 908
910 909 if not self.isConfig:
911 910
912 911 nchan = len(channelIndexList)
913 912 self.nchan = nchan
914 913 self.plotFact = 1
915 914 nplots = nchan
916 915
917 916 if showSNR:
918 917 nplots = nchan*2
919 918 self.plotFact = 2
920 919 if SNRmin == None: SNRmin = numpy.nanmin(SNRdB)
921 920 if SNRmax == None: SNRmax = numpy.nanmax(SNRdB)
922 921
923 922 self.setup(id=id,
924 923 nplots=nplots,
925 924 wintitle=wintitle,
926 925 show=show)
927 926
928 927 if timerange != None:
929 928 self.timerange = timerange
930 929
931 930 self.xmin, self.xmax = self.getTimeLim(x, xmin, xmax, timerange)
932 931
933 932 if ymin == None: ymin = numpy.nanmin(y)
934 933 if ymax == None: ymax = numpy.nanmax(y)
935 934 if zmin == None: zmin = numpy.nanmin(z)
936 935 if zmax == None: zmax = numpy.nanmax(z)
937 936
938 937 self.FTP_WEI = ftp_wei
939 938 self.EXP_CODE = exp_code
940 939 self.SUB_EXP_CODE = sub_exp_code
941 940 self.PLOT_POS = plot_pos
942 941
943 942 self.name = thisDatetime.strftime("%Y%m%d_%H%M%S")
944 943 self.isConfig = True
945 944 self.figfile = figfile
946 945 update_figfile = True
947 946
948 947 self.setWinTitle(title)
949 948
950 949 for i in range(self.nchan):
951 950 index = channelIndexList[i]
952 951 title = "Channel %d: %s" %(dataOut.channelList[index], thisDatetime.strftime("%Y/%m/%d %H:%M:%S"))
953 952 axes = self.axesList[i*self.plotFact]
954 953 z1 = z[i,:].reshape((1,-1))
955 954 axes.pcolorbuffer(x, y, z1,
956 955 xmin=self.xmin, xmax=self.xmax, ymin=ymin, ymax=ymax, zmin=zmin, zmax=zmax,
957 956 xlabel=xlabel, ylabel=ylabel, title=title, rti=True, XAxisAsTime=True,
958 957 ticksize=9, cblabel='', cbsize="1%",colormap=colormap)
959 958
960 959 if showSNR:
961 960 title = "Channel %d SNR: %s" %(dataOut.channelList[index], thisDatetime.strftime("%Y/%m/%d %H:%M:%S"))
962 961 axes = self.axesList[i*self.plotFact + 1]
963 962 SNRdB1 = SNRdB[i,:].reshape((1,-1))
964 963 axes.pcolorbuffer(x, y, SNRdB1,
965 964 xmin=self.xmin, xmax=self.xmax, ymin=ymin, ymax=ymax, zmin=SNRmin, zmax=SNRmax,
966 965 xlabel=xlabel, ylabel=ylabel, title=title, rti=True, XAxisAsTime=True,
967 966 ticksize=9, cblabel='', cbsize="1%",colormap='jet')
968 967
969 968
970 969 self.draw()
971 970
972 971 if dataOut.ltctime >= self.xmax:
973 972 self.counter_imagwr = wr_period
974 973 self.isConfig = False
975 974 update_figfile = True
976 975
977 976 self.save(figpath=figpath,
978 977 figfile=figfile,
979 978 save=save,
980 979 ftp=ftp,
981 980 wr_period=wr_period,
982 981 thisDatetime=thisDatetime,
983 982 update_figfile=update_figfile)
984 983
985 984
986 985
987 986 class Parameters1Plot(Figure):
988 987
989 988 __isConfig = None
990 989 __nsubplots = None
991 990
992 991 WIDTHPROF = None
993 992 HEIGHTPROF = None
994 993 PREFIX = 'prm'
995 994
996 995 def __init__(self, **kwargs):
997 996 Figure.__init__(self, **kwargs)
998 997 self.timerange = 2*60*60
999 998 self.isConfig = False
1000 999 self.__nsubplots = 1
1001 1000
1002 1001 self.WIDTH = 800
1003 1002 self.HEIGHT = 180
1004 1003 self.WIDTHPROF = 120
1005 1004 self.HEIGHTPROF = 0
1006 1005 self.counter_imagwr = 0
1007 1006
1008 1007 self.PLOT_CODE = PARMS_CODE
1009 1008
1010 1009 self.FTP_WEI = None
1011 1010 self.EXP_CODE = None
1012 1011 self.SUB_EXP_CODE = None
1013 1012 self.PLOT_POS = None
1014 1013 self.tmin = None
1015 1014 self.tmax = None
1016 1015
1017 1016 self.xmin = None
1018 1017 self.xmax = None
1019 1018
1020 1019 self.figfile = None
1021 1020
1022 1021 def getSubplots(self):
1023 1022
1024 1023 ncol = 1
1025 1024 nrow = self.nplots
1026 1025
1027 1026 return nrow, ncol
1028 1027
1029 1028 def setup(self, id, nplots, wintitle, showprofile=True, show=True):
1030 1029
1031 1030 self.__showprofile = showprofile
1032 1031 self.nplots = nplots
1033 1032
1034 1033 ncolspan = 1
1035 1034 colspan = 1
1036 1035
1037 1036 self.createFigure(id = id,
1038 1037 wintitle = wintitle,
1039 1038 widthplot = self.WIDTH + self.WIDTHPROF,
1040 1039 heightplot = self.HEIGHT + self.HEIGHTPROF,
1041 1040 show=show)
1042 1041
1043 1042 nrow, ncol = self.getSubplots()
1044 1043
1045 1044 counter = 0
1046 1045 for y in range(nrow):
1047 1046 for x in range(ncol):
1048 1047
1049 1048 if counter >= self.nplots:
1050 1049 break
1051 1050
1052 1051 self.addAxes(nrow, ncol*ncolspan, y, x*ncolspan, colspan, 1)
1053 1052
1054 1053 if showprofile:
1055 1054 self.addAxes(nrow, ncol*ncolspan, y, x*ncolspan+colspan, 1, 1)
1056 1055
1057 1056 counter += 1
1058 1057
1059 1058 def run(self, dataOut, id, wintitle="", channelList=None, showprofile=False,
1060 1059 xmin=None, xmax=None, ymin=None, ymax=None, zmin=None, zmax=None,timerange=None,
1061 1060 parameterIndex = None, onlyPositive = False,
1062 1061 SNRthresh = -numpy.inf, SNR = True, SNRmin = None, SNRmax = None, onlySNR = False,
1063 1062 DOP = True,
1064 1063 zlabel = "", parameterName = "", parameterObject = "data_param",
1065 1064 save=False, figpath='./', lastone=0,figfile=None, ftp=False, wr_period=1, show=True,
1066 1065 server=None, folder=None, username=None, password=None,
1067 1066 ftp_wei=0, exp_code=0, sub_exp_code=0, plot_pos=0):
1068 1067 #print inspect.getargspec(self.run).args
1069 1068 """
1070 1069
1071 1070 Input:
1072 1071 dataOut :
1073 1072 id :
1074 1073 wintitle :
1075 1074 channelList :
1076 1075 showProfile :
1077 1076 xmin : None,
1078 1077 xmax : None,
1079 1078 ymin : None,
1080 1079 ymax : None,
1081 1080 zmin : None,
1082 1081 zmax : None
1083 1082 """
1084 1083
1085 1084 data_param = getattr(dataOut, parameterObject)
1086 1085
1087 1086 if channelList == None:
1088 1087 channelIndexList = numpy.arange(data_param.shape[0])
1089 1088 else:
1090 1089 channelIndexList = numpy.array(channelList)
1091 1090
1092 1091 nchan = len(channelIndexList) #Number of channels being plotted
1093 1092
1094 1093 if nchan < 1:
1095 1094 return
1096 1095
1097 1096 nGraphsByChannel = 0
1098 1097
1099 1098 if SNR:
1100 1099 nGraphsByChannel += 1
1101 1100 if DOP:
1102 1101 nGraphsByChannel += 1
1103 1102
1104 1103 if nGraphsByChannel < 1:
1105 1104 return
1106 1105
1107 1106 nplots = nGraphsByChannel*nchan
1108 1107
1109 1108 if timerange is not None:
1110 1109 self.timerange = timerange
1111 1110
1112 1111 #tmin = None
1113 1112 #tmax = None
1114 1113 if parameterIndex == None:
1115 1114 parameterIndex = 1
1116 1115
1117 1116 x = dataOut.getTimeRange1(dataOut.paramInterval)
1118 1117 y = dataOut.heightList
1119 1118 z = data_param[channelIndexList,parameterIndex,:].copy()
1120 1119
1121 1120 zRange = dataOut.abscissaList
1122 1121 # nChannels = z.shape[0] #Number of wind dimensions estimated
1123 1122 # thisDatetime = dataOut.datatime
1124 1123
1125 1124 if dataOut.data_SNR is not None:
1126 1125 SNRarray = dataOut.data_SNR[channelIndexList,:]
1127 1126 SNRdB = 10*numpy.log10(SNRarray)
1128 1127 # SNRavgdB = 10*numpy.log10(SNRavg)
1129 1128 ind = numpy.where(SNRdB < 10**(SNRthresh/10))
1130 1129 z[ind] = numpy.nan
1131 1130
1132 1131 thisDatetime = datetime.datetime.utcfromtimestamp(dataOut.getTimeRange()[0])
1133 1132 title = wintitle + " Parameters Plot" #: %s" %(thisDatetime.strftime("%d-%b-%Y"))
1134 1133 xlabel = ""
1135 1134 ylabel = "Range (Km)"
1136 1135
1137 1136 if (SNR and not onlySNR): nplots = 2*nplots
1138 1137
1139 1138 if onlyPositive:
1140 1139 colormap = "jet"
1141 1140 zmin = 0
1142 1141 else: colormap = "RdBu_r"
1143 1142
1144 1143 if not self.isConfig:
1145 1144
1146 1145 self.setup(id=id,
1147 1146 nplots=nplots,
1148 1147 wintitle=wintitle,
1149 1148 showprofile=showprofile,
1150 1149 show=show)
1151 1150
1152 1151 self.xmin, self.xmax = self.getTimeLim(x, xmin, xmax, timerange)
1153 1152
1154 1153 if ymin == None: ymin = numpy.nanmin(y)
1155 1154 if ymax == None: ymax = numpy.nanmax(y)
1156 1155 if zmin == None: zmin = numpy.nanmin(zRange)
1157 1156 if zmax == None: zmax = numpy.nanmax(zRange)
1158 1157
1159 1158 if SNR:
1160 1159 if SNRmin == None: SNRmin = numpy.nanmin(SNRdB)
1161 1160 if SNRmax == None: SNRmax = numpy.nanmax(SNRdB)
1162 1161
1163 1162 self.FTP_WEI = ftp_wei
1164 1163 self.EXP_CODE = exp_code
1165 1164 self.SUB_EXP_CODE = sub_exp_code
1166 1165 self.PLOT_POS = plot_pos
1167 1166
1168 1167 self.name = thisDatetime.strftime("%Y%m%d_%H%M%S")
1169 1168 self.isConfig = True
1170 1169 self.figfile = figfile
1171 1170
1172 1171 self.setWinTitle(title)
1173 1172
1174 1173 if ((self.xmax - x[1]) < (x[1]-x[0])):
1175 1174 x[1] = self.xmax
1176 1175
1177 1176 for i in range(nchan):
1178 1177
1179 1178 if (SNR and not onlySNR): j = 2*i
1180 1179 else: j = i
1181 1180
1182 1181 j = nGraphsByChannel*i
1183 1182
1184 1183 if ((dataOut.azimuth!=None) and (dataOut.zenith!=None)):
1185 1184 title = title + '_' + 'azimuth,zenith=%2.2f,%2.2f'%(dataOut.azimuth, dataOut.zenith)
1186 1185
1187 1186 if not onlySNR:
1188 1187 axes = self.axesList[j*self.__nsubplots]
1189 1188 z1 = z[i,:].reshape((1,-1))
1190 1189 axes.pcolorbuffer(x, y, z1,
1191 1190 xmin=self.xmin, xmax=self.xmax, ymin=ymin, ymax=ymax, zmin=zmin, zmax=zmax,
1192 1191 xlabel=xlabel, ylabel=ylabel, title=title, rti=True, XAxisAsTime=True,colormap=colormap,
1193 1192 ticksize=9, cblabel=zlabel, cbsize="1%")
1194 1193
1195 1194 if DOP:
1196 1195 title = "%s Channel %d: %s" %(parameterName, channelIndexList[i], thisDatetime.strftime("%Y/%m/%d %H:%M:%S"))
1197 1196
1198 1197 if ((dataOut.azimuth!=None) and (dataOut.zenith!=None)):
1199 1198 title = title + '_' + 'azimuth,zenith=%2.2f,%2.2f'%(dataOut.azimuth, dataOut.zenith)
1200 1199 axes = self.axesList[j]
1201 1200 z1 = z[i,:].reshape((1,-1))
1202 1201 axes.pcolorbuffer(x, y, z1,
1203 1202 xmin=self.xmin, xmax=self.xmax, ymin=ymin, ymax=ymax, zmin=zmin, zmax=zmax,
1204 1203 xlabel=xlabel, ylabel=ylabel, title=title, rti=True, XAxisAsTime=True,colormap=colormap,
1205 1204 ticksize=9, cblabel=zlabel, cbsize="1%")
1206 1205
1207 1206 if SNR:
1208 1207 title = "Channel %d Signal Noise Ratio (SNR): %s" %(channelIndexList[i], thisDatetime.strftime("%Y/%m/%d %H:%M:%S"))
1209 1208 axes = self.axesList[(j)*self.__nsubplots]
1210 1209 if not onlySNR:
1211 1210 axes = self.axesList[(j + 1)*self.__nsubplots]
1212 1211
1213 1212 axes = self.axesList[(j + nGraphsByChannel-1)]
1214 1213
1215 1214 z1 = SNRdB[i,:].reshape((1,-1))
1216 1215 axes.pcolorbuffer(x, y, z1,
1217 1216 xmin=self.xmin, xmax=self.xmax, ymin=ymin, ymax=ymax, zmin=SNRmin, zmax=SNRmax,
1218 1217 xlabel=xlabel, ylabel=ylabel, title=title, rti=True, XAxisAsTime=True,colormap="jet",
1219 1218 ticksize=9, cblabel=zlabel, cbsize="1%")
1220 1219
1221 1220
1222 1221
1223 1222 self.draw()
1224 1223
1225 1224 if x[1] >= self.axesList[0].xmax:
1226 1225 self.counter_imagwr = wr_period
1227 1226 self.isConfig = False
1228 1227 self.figfile = None
1229 1228
1230 1229 self.save(figpath=figpath,
1231 1230 figfile=figfile,
1232 1231 save=save,
1233 1232 ftp=ftp,
1234 1233 wr_period=wr_period,
1235 1234 thisDatetime=thisDatetime,
1236 1235 update_figfile=False)
1237 1236
1238 1237 class SpectralFittingPlot(Figure):
1239 1238
1240 1239 __isConfig = None
1241 1240 __nsubplots = None
1242 1241
1243 1242 WIDTHPROF = None
1244 1243 HEIGHTPROF = None
1245 1244 PREFIX = 'prm'
1246 1245
1247 1246
1248 1247 N = None
1249 1248 ippSeconds = None
1250 1249
1251 1250 def __init__(self, **kwargs):
1252 1251 Figure.__init__(self, **kwargs)
1253 1252 self.isConfig = False
1254 1253 self.__nsubplots = 1
1255 1254
1256 1255 self.PLOT_CODE = SPECFIT_CODE
1257 1256
1258 1257 self.WIDTH = 450
1259 1258 self.HEIGHT = 250
1260 1259 self.WIDTHPROF = 0
1261 1260 self.HEIGHTPROF = 0
1262 1261
1263 1262 def getSubplots(self):
1264 1263
1265 1264 ncol = int(numpy.sqrt(self.nplots)+0.9)
1266 1265 nrow = int(self.nplots*1./ncol + 0.9)
1267 1266
1268 1267 return nrow, ncol
1269 1268
1270 1269 def setup(self, id, nplots, wintitle, showprofile=False, show=True):
1271 1270
1272 1271 showprofile = False
1273 1272 self.__showprofile = showprofile
1274 1273 self.nplots = nplots
1275 1274
1276 1275 ncolspan = 5
1277 1276 colspan = 4
1278 1277 if showprofile:
1279 1278 ncolspan = 5
1280 1279 colspan = 4
1281 1280 self.__nsubplots = 2
1282 1281
1283 1282 self.createFigure(id = id,
1284 1283 wintitle = wintitle,
1285 1284 widthplot = self.WIDTH + self.WIDTHPROF,
1286 1285 heightplot = self.HEIGHT + self.HEIGHTPROF,
1287 1286 show=show)
1288 1287
1289 1288 nrow, ncol = self.getSubplots()
1290 1289
1291 1290 counter = 0
1292 1291 for y in range(nrow):
1293 1292 for x in range(ncol):
1294 1293
1295 1294 if counter >= self.nplots:
1296 1295 break
1297 1296
1298 1297 self.addAxes(nrow, ncol*ncolspan, y, x*ncolspan, colspan, 1)
1299 1298
1300 1299 if showprofile:
1301 1300 self.addAxes(nrow, ncol*ncolspan, y, x*ncolspan+colspan, 1, 1)
1302 1301
1303 1302 counter += 1
1304 1303
1305 1304 def run(self, dataOut, id, cutHeight=None, fit=False, wintitle="", channelList=None, showprofile=True,
1306 1305 xmin=None, xmax=None, ymin=None, ymax=None,
1307 1306 save=False, figpath='./', figfile=None, show=True):
1308 1307
1309 1308 """
1310 1309
1311 1310 Input:
1312 1311 dataOut :
1313 1312 id :
1314 1313 wintitle :
1315 1314 channelList :
1316 1315 showProfile :
1317 1316 xmin : None,
1318 1317 xmax : None,
1319 1318 zmin : None,
1320 1319 zmax : None
1321 1320 """
1322 1321
1323 1322 if cutHeight==None:
1324 1323 h=270
1325 1324 heightindex = numpy.abs(cutHeight - dataOut.heightList).argmin()
1326 1325 cutHeight = dataOut.heightList[heightindex]
1327 1326
1328 1327 factor = dataOut.normFactor
1329 1328 x = dataOut.abscissaList[:-1]
1330 1329 #y = dataOut.getHeiRange()
1331 1330
1332 1331 z = dataOut.data_pre[:,:,heightindex]/factor
1333 1332 z = numpy.where(numpy.isfinite(z), z, numpy.NAN)
1334 1333 avg = numpy.average(z, axis=1)
1335 1334 listChannels = z.shape[0]
1336 1335
1337 1336 #Reconstruct Function
1338 1337 if fit==True:
1339 1338 groupArray = dataOut.groupList
1340 1339 listChannels = groupArray.reshape((groupArray.size))
1341 1340 listChannels.sort()
1342 1341 spcFitLine = numpy.zeros(z.shape)
1343 1342 constants = dataOut.constants
1344 1343
1345 1344 nGroups = groupArray.shape[0]
1346 1345 nChannels = groupArray.shape[1]
1347 1346 nProfiles = z.shape[1]
1348 1347
1349 1348 for f in range(nGroups):
1350 1349 groupChann = groupArray[f,:]
1351 1350 p = dataOut.data_param[f,:,heightindex]
1352 1351 # p = numpy.array([ 89.343967,0.14036615,0.17086219,18.89835291,1.58388365,1.55099167])
1353 1352 fitLineAux = dataOut.library.modelFunction(p, constants)*nProfiles
1354 1353 fitLineAux = fitLineAux.reshape((nChannels,nProfiles))
1355 1354 spcFitLine[groupChann,:] = fitLineAux
1356 1355 # spcFitLine = spcFitLine/factor
1357 1356
1358 1357 z = z[listChannels,:]
1359 1358 spcFitLine = spcFitLine[listChannels,:]
1360 1359 spcFitLinedB = 10*numpy.log10(spcFitLine)
1361 1360
1362 1361 zdB = 10*numpy.log10(z)
1363 1362 #thisDatetime = dataOut.datatime
1364 1363 thisDatetime = datetime.datetime.utcfromtimestamp(dataOut.getTimeRange()[0])
1365 1364 title = wintitle + " Doppler Spectra: %s" %(thisDatetime.strftime("%d-%b-%Y %H:%M:%S"))
1366 1365 xlabel = "Velocity (m/s)"
1367 1366 ylabel = "Spectrum"
1368 1367
1369 1368 if not self.isConfig:
1370 1369
1371 1370 nplots = listChannels.size
1372 1371
1373 1372 self.setup(id=id,
1374 1373 nplots=nplots,
1375 1374 wintitle=wintitle,
1376 1375 showprofile=showprofile,
1377 1376 show=show)
1378 1377
1379 1378 if xmin == None: xmin = numpy.nanmin(x)
1380 1379 if xmax == None: xmax = numpy.nanmax(x)
1381 1380 if ymin == None: ymin = numpy.nanmin(zdB)
1382 1381 if ymax == None: ymax = numpy.nanmax(zdB)+2
1383 1382
1384 1383 self.isConfig = True
1385 1384
1386 1385 self.setWinTitle(title)
1387 1386 for i in range(self.nplots):
1388 1387 # title = "Channel %d: %4.2fdB" %(dataOut.channelList[i]+1, noisedB[i])
1389 1388 title = "Height %4.1f km\nChannel %d:" %(cutHeight, listChannels[i])
1390 1389 axes = self.axesList[i*self.__nsubplots]
1391 1390 if fit == False:
1392 1391 axes.pline(x, zdB[i,:],
1393 1392 xmin=xmin, xmax=xmax, ymin=ymin, ymax=ymax,
1394 1393 xlabel=xlabel, ylabel=ylabel, title=title
1395 1394 )
1396 1395 if fit == True:
1397 1396 fitline=spcFitLinedB[i,:]
1398 1397 y=numpy.vstack([zdB[i,:],fitline] )
1399 1398 legendlabels=['Data','Fitting']
1400 1399 axes.pmultilineyaxis(x, y,
1401 1400 xmin=xmin, xmax=xmax, ymin=ymin, ymax=ymax,
1402 1401 xlabel=xlabel, ylabel=ylabel, title=title,
1403 1402 legendlabels=legendlabels, marker=None,
1404 1403 linestyle='solid', grid='both')
1405 1404
1406 1405 self.draw()
1407 1406
1408 1407 self.save(figpath=figpath,
1409 1408 figfile=figfile,
1410 1409 save=save,
1411 1410 ftp=ftp,
1412 1411 wr_period=wr_period,
1413 1412 thisDatetime=thisDatetime)
1414 1413
1415 1414
1416 1415 class EWDriftsPlot(Figure):
1417 1416
1418 1417 __isConfig = None
1419 1418 __nsubplots = None
1420 1419
1421 1420 WIDTHPROF = None
1422 1421 HEIGHTPROF = None
1423 1422 PREFIX = 'drift'
1424 1423
1425 1424 def __init__(self, **kwargs):
1426 1425 Figure.__init__(self, **kwargs)
1427 1426 self.timerange = 2*60*60
1428 1427 self.isConfig = False
1429 1428 self.__nsubplots = 1
1430 1429
1431 1430 self.WIDTH = 800
1432 1431 self.HEIGHT = 150
1433 1432 self.WIDTHPROF = 120
1434 1433 self.HEIGHTPROF = 0
1435 1434 self.counter_imagwr = 0
1436 1435
1437 1436 self.PLOT_CODE = EWDRIFT_CODE
1438 1437
1439 1438 self.FTP_WEI = None
1440 1439 self.EXP_CODE = None
1441 1440 self.SUB_EXP_CODE = None
1442 1441 self.PLOT_POS = None
1443 1442 self.tmin = None
1444 1443 self.tmax = None
1445 1444
1446 1445 self.xmin = None
1447 1446 self.xmax = None
1448 1447
1449 1448 self.figfile = None
1450 1449
1451 1450 def getSubplots(self):
1452 1451
1453 1452 ncol = 1
1454 1453 nrow = self.nplots
1455 1454
1456 1455 return nrow, ncol
1457 1456
1458 1457 def setup(self, id, nplots, wintitle, showprofile=True, show=True):
1459 1458
1460 1459 self.__showprofile = showprofile
1461 1460 self.nplots = nplots
1462 1461
1463 1462 ncolspan = 1
1464 1463 colspan = 1
1465 1464
1466 1465 self.createFigure(id = id,
1467 1466 wintitle = wintitle,
1468 1467 widthplot = self.WIDTH + self.WIDTHPROF,
1469 1468 heightplot = self.HEIGHT + self.HEIGHTPROF,
1470 1469 show=show)
1471 1470
1472 1471 nrow, ncol = self.getSubplots()
1473 1472
1474 1473 counter = 0
1475 1474 for y in range(nrow):
1476 1475 if counter >= self.nplots:
1477 1476 break
1478 1477
1479 1478 self.addAxes(nrow, ncol*ncolspan, y, 0, colspan, 1)
1480 1479 counter += 1
1481 1480
1482 1481 def run(self, dataOut, id, wintitle="", channelList=None,
1483 1482 xmin=None, xmax=None, ymin=None, ymax=None, zmin=None, zmax=None,
1484 1483 zmaxVertical = None, zminVertical = None, zmaxZonal = None, zminZonal = None,
1485 1484 timerange=None, SNRthresh = -numpy.inf, SNRmin = None, SNRmax = None, SNR_1 = False,
1486 1485 save=False, figpath='./', lastone=0,figfile=None, ftp=False, wr_period=1, show=True,
1487 1486 server=None, folder=None, username=None, password=None,
1488 1487 ftp_wei=0, exp_code=0, sub_exp_code=0, plot_pos=0):
1489 1488 """
1490 1489
1491 1490 Input:
1492 1491 dataOut :
1493 1492 id :
1494 1493 wintitle :
1495 1494 channelList :
1496 1495 showProfile :
1497 1496 xmin : None,
1498 1497 xmax : None,
1499 1498 ymin : None,
1500 1499 ymax : None,
1501 1500 zmin : None,
1502 1501 zmax : None
1503 1502 """
1504 1503
1505 1504 if timerange is not None:
1506 1505 self.timerange = timerange
1507 1506
1508 1507 tmin = None
1509 1508 tmax = None
1510 1509
1511 1510 x = dataOut.getTimeRange1(dataOut.outputInterval)
1512 1511 # y = dataOut.heightList
1513 1512 y = dataOut.heightList
1514 1513
1515 1514 z = dataOut.data_output
1516 1515 nplots = z.shape[0] #Number of wind dimensions estimated
1517 1516 nplotsw = nplots
1518 1517
1519 1518 #If there is a SNR function defined
1520 1519 if dataOut.data_SNR is not None:
1521 1520 nplots += 1
1522 1521 SNR = dataOut.data_SNR
1523 1522
1524 1523 if SNR_1:
1525 1524 SNR += 1
1526 1525
1527 1526 SNRavg = numpy.average(SNR, axis=0)
1528 1527
1529 1528 SNRdB = 10*numpy.log10(SNR)
1530 1529 SNRavgdB = 10*numpy.log10(SNRavg)
1531 1530
1532 1531 ind = numpy.where(SNRavg < 10**(SNRthresh/10))[0]
1533 1532
1534 1533 for i in range(nplotsw):
1535 1534 z[i,ind] = numpy.nan
1536 1535
1537 1536
1538 1537 showprofile = False
1539 1538 # thisDatetime = dataOut.datatime
1540 1539 thisDatetime = datetime.datetime.utcfromtimestamp(x[1])
1541 1540 title = wintitle + " EW Drifts"
1542 1541 xlabel = ""
1543 1542 ylabel = "Height (Km)"
1544 1543
1545 1544 if not self.isConfig:
1546 1545
1547 1546 self.setup(id=id,
1548 1547 nplots=nplots,
1549 1548 wintitle=wintitle,
1550 1549 showprofile=showprofile,
1551 1550 show=show)
1552 1551
1553 1552 self.xmin, self.xmax = self.getTimeLim(x, xmin, xmax, timerange)
1554 1553
1555 1554 if ymin == None: ymin = numpy.nanmin(y)
1556 1555 if ymax == None: ymax = numpy.nanmax(y)
1557 1556
1558 1557 if zmaxZonal == None: zmaxZonal = numpy.nanmax(abs(z[0,:]))
1559 1558 if zminZonal == None: zminZonal = -zmaxZonal
1560 1559 if zmaxVertical == None: zmaxVertical = numpy.nanmax(abs(z[1,:]))
1561 1560 if zminVertical == None: zminVertical = -zmaxVertical
1562 1561
1563 1562 if dataOut.data_SNR is not None:
1564 1563 if SNRmin == None: SNRmin = numpy.nanmin(SNRavgdB)
1565 1564 if SNRmax == None: SNRmax = numpy.nanmax(SNRavgdB)
1566 1565
1567 1566 self.FTP_WEI = ftp_wei
1568 1567 self.EXP_CODE = exp_code
1569 1568 self.SUB_EXP_CODE = sub_exp_code
1570 1569 self.PLOT_POS = plot_pos
1571 1570
1572 1571 self.name = thisDatetime.strftime("%Y%m%d_%H%M%S")
1573 1572 self.isConfig = True
1574 1573
1575 1574
1576 1575 self.setWinTitle(title)
1577 1576
1578 1577 if ((self.xmax - x[1]) < (x[1]-x[0])):
1579 1578 x[1] = self.xmax
1580 1579
1581 1580 strWind = ['Zonal','Vertical']
1582 1581 strCb = 'Velocity (m/s)'
1583 1582 zmaxVector = [zmaxZonal, zmaxVertical]
1584 1583 zminVector = [zminZonal, zminVertical]
1585 1584
1586 1585 for i in range(nplotsw):
1587 1586
1588 1587 title = "%s Drifts: %s" %(strWind[i], thisDatetime.strftime("%Y/%m/%d %H:%M:%S"))
1589 1588 axes = self.axesList[i*self.__nsubplots]
1590 1589
1591 1590 z1 = z[i,:].reshape((1,-1))
1592 1591
1593 1592 axes.pcolorbuffer(x, y, z1,
1594 1593 xmin=self.xmin, xmax=self.xmax, ymin=ymin, ymax=ymax, zmin=zminVector[i], zmax=zmaxVector[i],
1595 1594 xlabel=xlabel, ylabel=ylabel, title=title, rti=True, XAxisAsTime=True,
1596 1595 ticksize=9, cblabel=strCb, cbsize="1%", colormap="RdBu_r")
1597 1596
1598 1597 if dataOut.data_SNR is not None:
1599 1598 i += 1
1600 1599 if SNR_1:
1601 1600 title = "Signal Noise Ratio + 1 (SNR+1): %s" %(thisDatetime.strftime("%Y/%m/%d %H:%M:%S"))
1602 1601 else:
1603 1602 title = "Signal Noise Ratio (SNR): %s" %(thisDatetime.strftime("%Y/%m/%d %H:%M:%S"))
1604 1603 axes = self.axesList[i*self.__nsubplots]
1605 1604 SNRavgdB = SNRavgdB.reshape((1,-1))
1606 1605
1607 1606 axes.pcolorbuffer(x, y, SNRavgdB,
1608 1607 xmin=self.xmin, xmax=self.xmax, ymin=ymin, ymax=ymax, zmin=SNRmin, zmax=SNRmax,
1609 1608 xlabel=xlabel, ylabel=ylabel, title=title, rti=True, XAxisAsTime=True,
1610 1609 ticksize=9, cblabel='', cbsize="1%", colormap="jet")
1611 1610
1612 1611 self.draw()
1613 1612
1614 1613 if x[1] >= self.axesList[0].xmax:
1615 1614 self.counter_imagwr = wr_period
1616 1615 self.isConfig = False
1617 1616 self.figfile = None
1618 1617
1619 1618
1620 1619
1621 1620
1622 1621 class PhasePlot(Figure):
1623 1622
1624 1623 __isConfig = None
1625 1624 __nsubplots = None
1626 1625
1627 1626 PREFIX = 'mphase'
1628 1627
1628
1629 1629 def __init__(self, **kwargs):
1630 1630 Figure.__init__(self, **kwargs)
1631 1631 self.timerange = 24*60*60
1632 1632 self.isConfig = False
1633 1633 self.__nsubplots = 1
1634 1634 self.counter_imagwr = 0
1635 1635 self.WIDTH = 600
1636 1636 self.HEIGHT = 300
1637 1637 self.WIDTHPROF = 120
1638 1638 self.HEIGHTPROF = 0
1639 1639 self.xdata = None
1640 1640 self.ydata = None
1641 1641
1642 1642 self.PLOT_CODE = MPHASE_CODE
1643 1643
1644 1644 self.FTP_WEI = None
1645 1645 self.EXP_CODE = None
1646 1646 self.SUB_EXP_CODE = None
1647 1647 self.PLOT_POS = None
1648 1648
1649 1649
1650 1650 self.filename_phase = None
1651 1651
1652 1652 self.figfile = None
1653 1653
1654 1654 def getSubplots(self):
1655 1655
1656 1656 ncol = 1
1657 1657 nrow = 1
1658 1658
1659 1659 return nrow, ncol
1660 1660
1661 1661 def setup(self, id, nplots, wintitle, showprofile=True, show=True):
1662 1662
1663 1663 self.__showprofile = showprofile
1664 1664 self.nplots = nplots
1665 1665
1666 1666 ncolspan = 7
1667 1667 colspan = 6
1668 1668 self.__nsubplots = 2
1669 1669
1670 1670 self.createFigure(id = id,
1671 1671 wintitle = wintitle,
1672 1672 widthplot = self.WIDTH+self.WIDTHPROF,
1673 1673 heightplot = self.HEIGHT+self.HEIGHTPROF,
1674 1674 show=show)
1675 1675
1676 1676 nrow, ncol = self.getSubplots()
1677 1677
1678 1678 self.addAxes(nrow, ncol*ncolspan, 0, 0, colspan, 1)
1679 1679
1680 1680
1681 1681 def run(self, dataOut, id, wintitle="", pairsList=None, showprofile='True',
1682 1682 xmin=None, xmax=None, ymin=None, ymax=None,
1683 1683 timerange=None,
1684 1684 save=False, figpath='./', figfile=None, show=True, ftp=False, wr_period=1,
1685 1685 server=None, folder=None, username=None, password=None,
1686 1686 ftp_wei=0, exp_code=0, sub_exp_code=0, plot_pos=0):
1687 1687
1688 1688
1689 1689 tmin = None
1690 1690 tmax = None
1691 1691 x = dataOut.getTimeRange1(dataOut.outputInterval)
1692 1692 y = dataOut.getHeiRange()
1693 1693
1694 1694
1695 1695 #thisDatetime = dataOut.datatime
1696 1696 thisDatetime = datetime.datetime.utcfromtimestamp(dataOut.ltctime)
1697 1697 title = wintitle + " Phase of Beacon Signal" # : %s" %(thisDatetime.strftime("%d-%b-%Y"))
1698 1698 xlabel = "Local Time"
1699 1699 ylabel = "Phase"
1700 1700
1701 1701
1702 1702 #phase = numpy.zeros((len(pairsIndexList),len(dataOut.beacon_heiIndexList)))
1703 1703 phase_beacon = dataOut.data_output
1704 1704 update_figfile = False
1705 1705
1706 1706 if not self.isConfig:
1707 1707
1708 1708 self.nplots = phase_beacon.size
1709 1709
1710 1710 self.setup(id=id,
1711 1711 nplots=self.nplots,
1712 1712 wintitle=wintitle,
1713 1713 showprofile=showprofile,
1714 1714 show=show)
1715 1715
1716 1716 if timerange is not None:
1717 1717 self.timerange = timerange
1718 1718
1719 1719 self.xmin, self.xmax = self.getTimeLim(x, xmin, xmax, timerange)
1720 1720
1721 1721 if ymin == None: ymin = numpy.nanmin(phase_beacon) - 10.0
1722 1722 if ymax == None: ymax = numpy.nanmax(phase_beacon) + 10.0
1723 1723
1724 1724 self.FTP_WEI = ftp_wei
1725 1725 self.EXP_CODE = exp_code
1726 1726 self.SUB_EXP_CODE = sub_exp_code
1727 1727 self.PLOT_POS = plot_pos
1728 1728
1729 1729 self.name = thisDatetime.strftime("%Y%m%d_%H%M%S")
1730 1730 self.isConfig = True
1731 1731 self.figfile = figfile
1732 1732 self.xdata = numpy.array([])
1733 1733 self.ydata = numpy.array([])
1734 1734
1735 1735 #open file beacon phase
1736 1736 path = '%s%03d' %(self.PREFIX, self.id)
1737 1737 beacon_file = os.path.join(path,'%s.txt'%self.name)
1738 1738 self.filename_phase = os.path.join(figpath,beacon_file)
1739 1739 update_figfile = True
1740 1740
1741 1741
1742 1742 #store data beacon phase
1743 1743 #self.save_data(self.filename_phase, phase_beacon, thisDatetime)
1744 1744
1745 1745 self.setWinTitle(title)
1746 1746
1747 1747
1748 1748 title = "Phase Offset %s" %(thisDatetime.strftime("%Y/%m/%d %H:%M:%S"))
1749 1749
1750 1750 legendlabels = ["phase %d"%(chan) for chan in numpy.arange(self.nplots)]
1751 1751
1752 1752 axes = self.axesList[0]
1753 1753
1754 1754 self.xdata = numpy.hstack((self.xdata, x[0:1]))
1755 1755
1756 1756 if len(self.ydata)==0:
1757 1757 self.ydata = phase_beacon.reshape(-1,1)
1758 1758 else:
1759 1759 self.ydata = numpy.hstack((self.ydata, phase_beacon.reshape(-1,1)))
1760 1760
1761 1761
1762 1762 axes.pmultilineyaxis(x=self.xdata, y=self.ydata,
1763 1763 xmin=self.xmin, xmax=self.xmax, ymin=ymin, ymax=ymax,
1764 1764 xlabel=xlabel, ylabel=ylabel, title=title, legendlabels=legendlabels, marker='x', markersize=8, linestyle="solid",
1765 1765 XAxisAsTime=True, grid='both'
1766 1766 )
1767 1767
1768 1768 self.draw()
1769 1769
1770 1770 self.save(figpath=figpath,
1771 1771 figfile=figfile,
1772 1772 save=save,
1773 1773 ftp=ftp,
1774 1774 wr_period=wr_period,
1775 1775 thisDatetime=thisDatetime,
1776 1776 update_figfile=update_figfile)
1777 1777
1778 1778 if dataOut.ltctime + dataOut.outputInterval >= self.xmax:
1779 1779 self.counter_imagwr = wr_period
1780 1780 self.isConfig = False
1781 1781 update_figfile = True
1782 1782
1783 1783
1784 1784
1785 1785 class NSMeteorDetection1Plot(Figure):
1786 1786
1787 1787 isConfig = None
1788 1788 __nsubplots = None
1789 1789
1790 1790 WIDTHPROF = None
1791 1791 HEIGHTPROF = None
1792 1792 PREFIX = 'nsm'
1793 1793
1794 1794 zminList = None
1795 1795 zmaxList = None
1796 1796 cmapList = None
1797 1797 titleList = None
1798 1798 nPairs = None
1799 1799 nChannels = None
1800 1800 nParam = None
1801 1801
1802 1802 def __init__(self, **kwargs):
1803 1803 Figure.__init__(self, **kwargs)
1804 1804 self.isConfig = False
1805 1805 self.__nsubplots = 1
1806 1806
1807 1807 self.WIDTH = 750
1808 1808 self.HEIGHT = 250
1809 1809 self.WIDTHPROF = 120
1810 1810 self.HEIGHTPROF = 0
1811 1811 self.counter_imagwr = 0
1812 1812
1813 1813 self.PLOT_CODE = SPEC_CODE
1814 1814
1815 1815 self.FTP_WEI = None
1816 1816 self.EXP_CODE = None
1817 1817 self.SUB_EXP_CODE = None
1818 1818 self.PLOT_POS = None
1819 1819
1820 1820 self.__xfilter_ena = False
1821 1821 self.__yfilter_ena = False
1822 1822
1823 1823 def getSubplots(self):
1824 1824
1825 1825 ncol = 3
1826 1826 nrow = int(numpy.ceil(self.nplots/3.0))
1827 1827
1828 1828 return nrow, ncol
1829 1829
1830 1830 def setup(self, id, nplots, wintitle, show=True):
1831 1831
1832 1832 self.nplots = nplots
1833 1833
1834 1834 ncolspan = 1
1835 1835 colspan = 1
1836 1836
1837 1837 self.createFigure(id = id,
1838 1838 wintitle = wintitle,
1839 1839 widthplot = self.WIDTH + self.WIDTHPROF,
1840 1840 heightplot = self.HEIGHT + self.HEIGHTPROF,
1841 1841 show=show)
1842 1842
1843 1843 nrow, ncol = self.getSubplots()
1844 1844
1845 1845 counter = 0
1846 1846 for y in range(nrow):
1847 1847 for x in range(ncol):
1848 1848
1849 1849 if counter >= self.nplots:
1850 1850 break
1851 1851
1852 1852 self.addAxes(nrow, ncol*ncolspan, y, x*ncolspan, colspan, 1)
1853 1853
1854 1854 counter += 1
1855 1855
1856 1856 def run(self, dataOut, id, wintitle="", channelList=None, showprofile=True,
1857 1857 xmin=None, xmax=None, ymin=None, ymax=None, SNRmin=None, SNRmax=None,
1858 1858 vmin=None, vmax=None, wmin=None, wmax=None, mode = 'SA',
1859 1859 save=False, figpath='./', figfile=None, show=True, ftp=False, wr_period=1,
1860 1860 server=None, folder=None, username=None, password=None,
1861 1861 ftp_wei=0, exp_code=0, sub_exp_code=0, plot_pos=0, realtime=False,
1862 1862 xaxis="frequency"):
1863 1863
1864 1864 """
1865 1865
1866 1866 Input:
1867 1867 dataOut :
1868 1868 id :
1869 1869 wintitle :
1870 1870 channelList :
1871 1871 showProfile :
1872 1872 xmin : None,
1873 1873 xmax : None,
1874 1874 ymin : None,
1875 1875 ymax : None,
1876 1876 zmin : None,
1877 1877 zmax : None
1878 1878 """
1879 1879 #SEPARAR EN DOS PLOTS
1880 1880 nParam = dataOut.data_param.shape[1] - 3
1881 1881
1882 1882 utctime = dataOut.data_param[0,0]
1883 1883 tmet = dataOut.data_param[:,1].astype(int)
1884 1884 hmet = dataOut.data_param[:,2].astype(int)
1885 1885
1886 1886 x = dataOut.abscissaList
1887 1887 y = dataOut.heightList
1888 1888
1889 1889 z = numpy.zeros((nParam, y.size, x.size - 1))
1890 1890 z[:,:] = numpy.nan
1891 1891 z[:,hmet,tmet] = dataOut.data_param[:,3:].T
1892 1892 z[0,:,:] = 10*numpy.log10(z[0,:,:])
1893 1893
1894 1894 xlabel = "Time (s)"
1895 1895 ylabel = "Range (km)"
1896 1896
1897 1897 thisDatetime = datetime.datetime.utcfromtimestamp(dataOut.ltctime)
1898 1898
1899 1899 if not self.isConfig:
1900 1900
1901 1901 nplots = nParam
1902 1902
1903 1903 self.setup(id=id,
1904 1904 nplots=nplots,
1905 1905 wintitle=wintitle,
1906 1906 show=show)
1907 1907
1908 1908 if xmin is None: xmin = numpy.nanmin(x)
1909 1909 if xmax is None: xmax = numpy.nanmax(x)
1910 1910 if ymin is None: ymin = numpy.nanmin(y)
1911 1911 if ymax is None: ymax = numpy.nanmax(y)
1912 1912 if SNRmin is None: SNRmin = numpy.nanmin(z[0,:])
1913 1913 if SNRmax is None: SNRmax = numpy.nanmax(z[0,:])
1914 1914 if vmax is None: vmax = numpy.nanmax(numpy.abs(z[1,:]))
1915 1915 if vmin is None: vmin = -vmax
1916 1916 if wmin is None: wmin = 0
1917 1917 if wmax is None: wmax = 50
1918 1918
1919 1919 pairsList = dataOut.groupList
1920 1920 self.nPairs = len(dataOut.groupList)
1921 1921
1922 1922 zminList = [SNRmin, vmin, cmin] + [pmin]*self.nPairs
1923 1923 zmaxList = [SNRmax, vmax, cmax] + [pmax]*self.nPairs
1924 1924 titleList = ["SNR","Radial Velocity","Coherence"]
1925 1925 cmapList = ["jet","RdBu_r","jet"]
1926 1926
1927 1927 for i in range(self.nPairs):
1928 1928 strAux1 = "Phase Difference "+ str(pairsList[i][0]) + str(pairsList[i][1])
1929 1929 titleList = titleList + [strAux1]
1930 1930 cmapList = cmapList + ["RdBu_r"]
1931 1931
1932 1932 self.zminList = zminList
1933 1933 self.zmaxList = zmaxList
1934 1934 self.cmapList = cmapList
1935 1935 self.titleList = titleList
1936 1936
1937 1937 self.FTP_WEI = ftp_wei
1938 1938 self.EXP_CODE = exp_code
1939 1939 self.SUB_EXP_CODE = sub_exp_code
1940 1940 self.PLOT_POS = plot_pos
1941 1941
1942 1942 self.isConfig = True
1943 1943
1944 1944 str_datetime = '%s %s'%(thisDatetime.strftime("%Y/%m/%d"),thisDatetime.strftime("%H:%M:%S"))
1945 1945
1946 1946 for i in range(nParam):
1947 1947 title = self.titleList[i] + ": " +str_datetime
1948 1948 axes = self.axesList[i]
1949 1949 axes.pcolor(x, y, z[i,:].T,
1950 1950 xmin=xmin, xmax=xmax, ymin=ymin, ymax=ymax, zmin=self.zminList[i], zmax=self.zmaxList[i],
1951 1951 xlabel=xlabel, ylabel=ylabel, title=title, colormap=self.cmapList[i],ticksize=9, cblabel='')
1952 1952 self.draw()
1953 1953
1954 1954 if figfile == None:
1955 1955 str_datetime = thisDatetime.strftime("%Y%m%d_%H%M%S")
1956 1956 name = str_datetime
1957 1957 if ((dataOut.azimuth!=None) and (dataOut.zenith!=None)):
1958 1958 name = name + '_az' + '_%2.2f'%(dataOut.azimuth) + '_zn' + '_%2.2f'%(dataOut.zenith)
1959 1959 figfile = self.getFilename(name)
1960 1960
1961 1961 self.save(figpath=figpath,
1962 1962 figfile=figfile,
1963 1963 save=save,
1964 1964 ftp=ftp,
1965 1965 wr_period=wr_period,
1966 1966 thisDatetime=thisDatetime)
1967 1967
1968 1968
1969 1969 class NSMeteorDetection2Plot(Figure):
1970 1970
1971 1971 isConfig = None
1972 1972 __nsubplots = None
1973 1973
1974 1974 WIDTHPROF = None
1975 1975 HEIGHTPROF = None
1976 1976 PREFIX = 'nsm'
1977 1977
1978 1978 zminList = None
1979 1979 zmaxList = None
1980 1980 cmapList = None
1981 1981 titleList = None
1982 1982 nPairs = None
1983 1983 nChannels = None
1984 1984 nParam = None
1985 1985
1986 1986 def __init__(self, **kwargs):
1987 1987 Figure.__init__(self, **kwargs)
1988 1988 self.isConfig = False
1989 1989 self.__nsubplots = 1
1990 1990
1991 1991 self.WIDTH = 750
1992 1992 self.HEIGHT = 250
1993 1993 self.WIDTHPROF = 120
1994 1994 self.HEIGHTPROF = 0
1995 1995 self.counter_imagwr = 0
1996 1996
1997 1997 self.PLOT_CODE = SPEC_CODE
1998 1998
1999 1999 self.FTP_WEI = None
2000 2000 self.EXP_CODE = None
2001 2001 self.SUB_EXP_CODE = None
2002 2002 self.PLOT_POS = None
2003 2003
2004 2004 self.__xfilter_ena = False
2005 2005 self.__yfilter_ena = False
2006 2006
2007 2007 def getSubplots(self):
2008 2008
2009 2009 ncol = 3
2010 2010 nrow = int(numpy.ceil(self.nplots/3.0))
2011 2011
2012 2012 return nrow, ncol
2013 2013
2014 2014 def setup(self, id, nplots, wintitle, show=True):
2015 2015
2016 2016 self.nplots = nplots
2017 2017
2018 2018 ncolspan = 1
2019 2019 colspan = 1
2020 2020
2021 2021 self.createFigure(id = id,
2022 2022 wintitle = wintitle,
2023 2023 widthplot = self.WIDTH + self.WIDTHPROF,
2024 2024 heightplot = self.HEIGHT + self.HEIGHTPROF,
2025 2025 show=show)
2026 2026
2027 2027 nrow, ncol = self.getSubplots()
2028 2028
2029 2029 counter = 0
2030 2030 for y in range(nrow):
2031 2031 for x in range(ncol):
2032 2032
2033 2033 if counter >= self.nplots:
2034 2034 break
2035 2035
2036 2036 self.addAxes(nrow, ncol*ncolspan, y, x*ncolspan, colspan, 1)
2037 2037
2038 2038 counter += 1
2039 2039
2040 2040 def run(self, dataOut, id, wintitle="", channelList=None, showprofile=True,
2041 2041 xmin=None, xmax=None, ymin=None, ymax=None, SNRmin=None, SNRmax=None,
2042 2042 vmin=None, vmax=None, wmin=None, wmax=None, mode = 'SA',
2043 2043 save=False, figpath='./', figfile=None, show=True, ftp=False, wr_period=1,
2044 2044 server=None, folder=None, username=None, password=None,
2045 2045 ftp_wei=0, exp_code=0, sub_exp_code=0, plot_pos=0, realtime=False,
2046 2046 xaxis="frequency"):
2047 2047
2048 2048 """
2049 2049
2050 2050 Input:
2051 2051 dataOut :
2052 2052 id :
2053 2053 wintitle :
2054 2054 channelList :
2055 2055 showProfile :
2056 2056 xmin : None,
2057 2057 xmax : None,
2058 2058 ymin : None,
2059 2059 ymax : None,
2060 2060 zmin : None,
2061 2061 zmax : None
2062 2062 """
2063 2063 #Rebuild matrix
2064 2064 utctime = dataOut.data_param[0,0]
2065 2065 cmet = dataOut.data_param[:,1].astype(int)
2066 2066 tmet = dataOut.data_param[:,2].astype(int)
2067 2067 hmet = dataOut.data_param[:,3].astype(int)
2068 2068
2069 2069 nParam = 3
2070 2070 nChan = len(dataOut.groupList)
2071 2071 x = dataOut.abscissaList
2072 2072 y = dataOut.heightList
2073 2073
2074 2074 z = numpy.full((nChan, nParam, y.size, x.size - 1),numpy.nan)
2075 2075 z[cmet,:,hmet,tmet] = dataOut.data_param[:,4:]
2076 2076 z[:,0,:,:] = 10*numpy.log10(z[:,0,:,:]) #logarithmic scale
2077 2077 z = numpy.reshape(z, (nChan*nParam, y.size, x.size-1))
2078 2078
2079 2079 xlabel = "Time (s)"
2080 2080 ylabel = "Range (km)"
2081 2081
2082 2082 thisDatetime = datetime.datetime.utcfromtimestamp(dataOut.ltctime)
2083 2083
2084 2084 if not self.isConfig:
2085 2085
2086 2086 nplots = nParam*nChan
2087 2087
2088 2088 self.setup(id=id,
2089 2089 nplots=nplots,
2090 2090 wintitle=wintitle,
2091 2091 show=show)
2092 2092
2093 2093 if xmin is None: xmin = numpy.nanmin(x)
2094 2094 if xmax is None: xmax = numpy.nanmax(x)
2095 2095 if ymin is None: ymin = numpy.nanmin(y)
2096 2096 if ymax is None: ymax = numpy.nanmax(y)
2097 2097 if SNRmin is None: SNRmin = numpy.nanmin(z[0,:])
2098 2098 if SNRmax is None: SNRmax = numpy.nanmax(z[0,:])
2099 2099 if vmax is None: vmax = numpy.nanmax(numpy.abs(z[1,:]))
2100 2100 if vmin is None: vmin = -vmax
2101 2101 if wmin is None: wmin = 0
2102 2102 if wmax is None: wmax = 50
2103 2103
2104 2104 self.nChannels = nChan
2105 2105
2106 2106 zminList = []
2107 2107 zmaxList = []
2108 2108 titleList = []
2109 2109 cmapList = []
2110 2110 for i in range(self.nChannels):
2111 2111 strAux1 = "SNR Channel "+ str(i)
2112 2112 strAux2 = "Radial Velocity Channel "+ str(i)
2113 2113 strAux3 = "Spectral Width Channel "+ str(i)
2114 2114
2115 2115 titleList = titleList + [strAux1,strAux2,strAux3]
2116 2116 cmapList = cmapList + ["jet","RdBu_r","jet"]
2117 2117 zminList = zminList + [SNRmin,vmin,wmin]
2118 2118 zmaxList = zmaxList + [SNRmax,vmax,wmax]
2119 2119
2120 2120 self.zminList = zminList
2121 2121 self.zmaxList = zmaxList
2122 2122 self.cmapList = cmapList
2123 2123 self.titleList = titleList
2124 2124
2125 2125 self.FTP_WEI = ftp_wei
2126 2126 self.EXP_CODE = exp_code
2127 2127 self.SUB_EXP_CODE = sub_exp_code
2128 2128 self.PLOT_POS = plot_pos
2129 2129
2130 2130 self.isConfig = True
2131 2131
2132 2132 str_datetime = '%s %s'%(thisDatetime.strftime("%Y/%m/%d"),thisDatetime.strftime("%H:%M:%S"))
2133 2133
2134 2134 for i in range(self.nplots):
2135 2135 title = self.titleList[i] + ": " +str_datetime
2136 2136 axes = self.axesList[i]
2137 2137 axes.pcolor(x, y, z[i,:].T,
2138 2138 xmin=xmin, xmax=xmax, ymin=ymin, ymax=ymax, zmin=self.zminList[i], zmax=self.zmaxList[i],
2139 2139 xlabel=xlabel, ylabel=ylabel, title=title, colormap=self.cmapList[i],ticksize=9, cblabel='')
2140 2140 self.draw()
2141 2141
2142 2142 if figfile == None:
2143 2143 str_datetime = thisDatetime.strftime("%Y%m%d_%H%M%S")
2144 2144 name = str_datetime
2145 2145 if ((dataOut.azimuth!=None) and (dataOut.zenith!=None)):
2146 2146 name = name + '_az' + '_%2.2f'%(dataOut.azimuth) + '_zn' + '_%2.2f'%(dataOut.zenith)
2147 2147 figfile = self.getFilename(name)
2148 2148
2149 2149 self.save(figpath=figpath,
2150 2150 figfile=figfile,
2151 2151 save=save,
2152 2152 ftp=ftp,
2153 2153 wr_period=wr_period,
2154 2154 thisDatetime=thisDatetime)
Show file before | Show file after | Hide comments
1 NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff
Show file before | Show file after | Hide comments
1 NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff
Show file before | Show file after | Hide comments
1 NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff
Show file before | Show file after | Hide comments
1 NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff
Show file before | Show file after | Hide comments
1 NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff
Show file before | Show file after | Hide comments
1 NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff
Show file before | Show file after | Hide comments
1 NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff
Show file before | Show file after | Hide comments
1 NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff
Show file before | Show file after | Hide comments
1 NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff
Show file before | Show file after | Hide comments
1 NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff
Show file before | Show file after | Hide comments
1 NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff
Show file before | Show file after | Hide comments
1 NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff
Show file before | Show file after | Hide comments
1 NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff
Show file before | Show file after | Hide comments
1 NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff
Show file before | Show file after | Hide comments
1 NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff
Show file before | Show file after | Hide comments
1 NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff
Show file before | Show file after | Hide comments
1 NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff
Show file before | Show file after | Hide comments
1 NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff
Show file before | Show file after | Hide comments
1 NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff
Show file before | Show file after | Hide comments
1 NO CONTENT: modified file
The requested commit or file is too big and content was truncated. Show full diff
Show file before | Show file after | Hide comments
1 NO CONTENT: file was removed
General Comments 0
You need to be logged in to leave comments. Login now