@@ -0,0 +1,108 | |||||
|
1 | ## CHANGELOG: | |||
|
2 | ||||
|
3 | ### 2.3 | |||
|
4 | * Added high order function `multiSchain` for multiprocessing scripts. | |||
|
5 | * Added two new Processing Units `PublishData` and `ReceiverData` for receiving and sending dataOut through multiple ways (tcp, ipc, inproc). | |||
|
6 | * Added a new graphics Processing Unit `PlotterReceiver`. It is decoupled from normal processing sequence with support for data generated by multiprocessing scripts. | |||
|
7 | * Added support for sending realtime graphic to web server. | |||
|
8 | * XML command `schain` is now `schain --xml`. | |||
|
9 | * Added a CLI tool named `schain`. | |||
|
10 | * Scripts templates can be now generated with `schain generate`. | |||
|
11 | * Now it is possible to search Processing Units and Operations with `schain search [module]` to get the right name and its allowed parameters. | |||
|
12 | * Added suggestions when parameters are poorly written. | |||
|
13 | * `Controller.start()` now runs in a different process than the process calling it. | |||
|
14 | * Added `schainpy.utils.log` for log standarization. | |||
|
15 | * Running script on online mode no longer ignores date and hour. Issue #1109. | |||
|
16 | * Added support for receving voltage data directly from JARS (tcp, ipc). | |||
|
17 | * Updated README for compatibility with MAC OS GUI installation. | |||
|
18 | * Setup now installs numpy. | |||
|
19 | ||||
|
20 | ### 2.2.6 | |||
|
21 | * Graphics generated by the GUI are now the same as generated by scripts. Issue #1074. | |||
|
22 | * Added support for C extensions. | |||
|
23 | * Function `hildebrand_sehkon` optimized with a C wrapper. | |||
|
24 | * Numpy version updated. | |||
|
25 | * Migration to GIT. | |||
|
26 | ||||
|
27 | ### 2.2.5: | |||
|
28 | * splitProfiles and combineProfiles modules were added to VoltageProc and Signal Chain GUI. | |||
|
29 | * nProfiles of USRP data (hdf5) is the number of profiles thera are in one second. | |||
|
30 | * jroPlotter works directly with data objects instead of dictionaries | |||
|
31 | * script "schain" was added to Signal Chain installer | |||
|
32 | ||||
|
33 | ### 2.2.4.1: | |||
|
34 | * jroIO_usrp.py is update to read Sandra's data | |||
|
35 | * decimation in Spectra and RTI plots is always enabled. | |||
|
36 | * time* window option added to GUI | |||
|
37 | ||||
|
38 | ### 2.2.4: | |||
|
39 | * jroproc_spectra_lags.py added to schainpy | |||
|
40 | * Bug fixed in schainGUI: ProcUnit was created with the same id in some cases. | |||
|
41 | * Bug fixed in jroHeaderIO: Header size validation. | |||
|
42 | ||||
|
43 | ### 2.2.3.1: | |||
|
44 | * Filtering block by time has been added. | |||
|
45 | * Bug fixed plotting RTI, CoherenceMap and others using xmin and xmax parameters. The first day worked | |||
|
46 | properly but the next days did not. | |||
|
47 | ||||
|
48 | ### 2.2.3: | |||
|
49 | * Bug fixed in GUI: Error getting(reading) Code value | |||
|
50 | * Bug fixed in GUI: Flip option always needs channelList field | |||
|
51 | * Bug fixed in jrodata: when one branch modified a value in "dataOut" (example: dataOut.code) this value | |||
|
52 | was modified for every branch (because this was a reference). It was modified in data.copy() | |||
|
53 | * Bug fixed in jroproc_voltage.profileSelector(): rangeList replaces to profileRangeList. | |||
|
54 | ||||
|
55 | ### 2.2.2: | |||
|
56 | * VoltageProc: ProfileSelector, Reshape, Decoder with nTxs!=1 and getblock=True was tested | |||
|
57 | * Rawdata and testRawdata.py added to Signal Chain project | |||
|
58 | ||||
|
59 | ### 2.2.1: | |||
|
60 | * Bugs fixed in GUI | |||
|
61 | * Views were improved in GUI | |||
|
62 | * Support to MST* ISR experiments | |||
|
63 | * Bug fixed getting noise using hyldebrant. (minimum number of points > 20%) | |||
|
64 | * handleError added to jroplotter.py | |||
|
65 | ||||
|
66 | ### 2.2.0: | |||
|
67 | * GUI: use of external plotter | |||
|
68 | * Compatible with matplotlib 1.5.0 | |||
|
69 | ||||
|
70 | ### 2.1.5: | |||
|
71 | * serializer module added to Signal Chain | |||
|
72 | * jroplotter.py added to Signal Chain | |||
|
73 | ||||
|
74 | ### 2.1.4.2: | |||
|
75 | * A new Plotter Class was added | |||
|
76 | * Project.start() does not accept filename as a parameter anymore | |||
|
77 | ||||
|
78 | ### 2.1.4.1: | |||
|
79 | * Send notifications when an error different to ValueError is detected | |||
|
80 | ||||
|
81 | ### 2.1.4: | |||
|
82 | * Sending error notifications to signal chain administrator | |||
|
83 | * Login to email server added | |||
|
84 | ||||
|
85 | ### 2.1.3.3: | |||
|
86 | * Colored Button Icons were added to GUI | |||
|
87 | ||||
|
88 | ### 2.1.3.2: | |||
|
89 | * GUI: user interaction enhanced | |||
|
90 | * controller_api.py: Safe access to ControllerThead | |||
|
91 | ||||
|
92 | ### 2.1.3.1: | |||
|
93 | * GUI: every icon were resized | |||
|
94 | * jroproc_voltage.py: Print a message when "Read from code" option is selected and the code is not defined inside data file | |||
|
95 | ||||
|
96 | ### 2.1.3: | |||
|
97 | * jroplot_heispectra.py: SpectraHeisScope was not showing the right channels | |||
|
98 | * jroproc_voltage.py: Bug fixed selecting profiles (self.nProfiles took a wrong value), | |||
|
99 | Bug fixed selecting heights by block (selecting profiles instead heights) | |||
|
100 | * jroproc_voltage.py: New feature added: decoding data by block using FFT. | |||
|
101 | * jroIO_heispectra.py: Bug fixed in FitsReader. Using local Fits instance instead schainpy.mode.data.jrodata.Fits. | |||
|
102 | * jroIO_heispectra.py: Channel index list does not exist. | |||
|
103 | ||||
|
104 | ### 2.1.2: | |||
|
105 | * jroutils_ftp.py: Bug fixed, Any error sending file stopped the Server Thread | |||
|
106 | Server thread opens and closes remote server each time file list is sent | |||
|
107 | * jroplot_spectra.py: Noise path was not being created when noise data is saved. | |||
|
108 | * jroIO_base.py: startTime can be greater than endTime. Example: SpreadF [18:00 * 07:00] No newline at end of file |
@@ -100,10 +100,20 ENV/ | |||||
100 | # eclipse |
|
100 | # eclipse | |
101 | .project |
|
101 | .project | |
102 | .pydevproject |
|
102 | .pydevproject | |
|
103 | <<<<<<< HEAD | |||
103 |
|
104 | |||
104 | # vscode |
|
105 | # vscode | |
105 |
|
106 | |||
106 | .vscode |
|
107 | .vscode | |
107 |
|
108 | |||
108 | schaingui/node_modules/ |
|
109 | schaingui/node_modules/ | |
109 | schainpy/scripts/ No newline at end of file |
|
110 | schainpy/scripts/ | |
|
111 | ======= | |||
|
112 | .svn/ | |||
|
113 | *.png | |||
|
114 | *.pyc | |||
|
115 | schainpy/scripts | |||
|
116 | .vscode | |||
|
117 | schaingui/node_modules | |||
|
118 | trash | |||
|
119 | >>>>>>> master |
@@ -6,24 +6,40 Signal Chain (SCh) is a radar data processing library developed using [Python](w | |||||
6 |
|
6 | |||
7 | ## Installation |
|
7 | ## Installation | |
8 |
|
8 | |||
9 |
Install system dependencies, |
|
9 | Install system dependencies, clone the latest version from [git](http://jro-dev.igp.gob.pe/rhodecode/schain/) and install it as a normal python package. | |
10 |
|
10 | |||
|
11 | ### Linux based system | |||
11 | ``` |
|
12 | ``` | |
12 | $ sudo apt-get install python-pip python-dev gfortran libpng-dev freetype* libblas-dev liblapack-dev libatlas-base-dev python-qt4 python-tk libssl-dev libhdf5-dev |
|
13 | $ sudo apt-get install python-pip python-dev gfortran libpng-dev freetype* libblas-dev liblapack-dev libatlas-base-dev python-qt4 python-tk libssl-dev libhdf5-dev | |
13 | $ tar xvzf schainpy-2.2.5.tar.gz |
|
14 | $ sudo pip install numpy | |
14 | $ cd schainpy-2.2.5 |
|
15 | $ git clone http://jro-dev.igp.gob.pe/rhodecode/schain/ | |
|
16 | $ cd schain | |||
15 | $ sudo pip install ./ |
|
17 | $ sudo pip install ./ | |
16 | ``` |
|
|||
17 |
|
||||
18 | **Its recommended to install schain in a virtual environment** |
|
|||
19 |
|
|
18 | ||
20 | ``` |
|
19 | ``` | |
|
20 | **It is recommended to install schain in a virtual environment** | |||
|
21 | ``` | |||
21 | $ sudo pip install virtualenv |
|
22 | $ sudo pip install virtualenv | |
22 | $ virtualenv /path/to/virtual --system-site-packages |
|
23 | $ virtualenv /path/to/virtual --system-site-packages | |
23 | $ source /path/to/virtual/bin/activate |
|
24 | $ source /path/to/virtual/bin/activate | |
24 |
(virtual) $ cd schain |
|
25 | (virtual) $ cd schain | |
25 | (virtual) $ pip install ./ |
|
26 | (virtual) $ pip install ./ | |
|
27 | ||||
|
28 | ``` | |||
|
29 | ||||
|
30 | ### MAC Os | |||
26 |
``` |
|
31 | ``` | |
|
32 | $ brew install cartr/qt4/pyqt | |||
|
33 | $ git clone http://jro-dev.igp.gob.pe/rhodecode/schain/ | |||
|
34 | $ cd schain | |||
|
35 | $ pip install ./ | |||
|
36 | ``` | |||
|
37 | ||||
|
38 | if ```pip install ./``` does not work, install a proper python enviroment, and repeat the steps. | |||
|
39 | ``` | |||
|
40 | $ brew install python | |||
|
41 | ``` | |||
|
42 | ||||
27 |
|
43 | |||
28 | ## First Script |
|
44 | ## First Script | |
29 |
|
45 | |||
@@ -128,4 +144,4 op.addParameter(name='wintitle', value='RTI', format='str') | |||||
128 |
|
144 | |||
129 | controller.start() |
|
145 | controller.start() | |
130 |
|
146 | |||
131 |
``` |
|
147 | ``` No newline at end of file |
@@ -4,4 +4,5 Created on Feb 7, 2012 | |||||
4 | @author $Author$ |
|
4 | @author $Author$ | |
5 | @version $Id$ |
|
5 | @version $Id$ | |
6 | ''' |
|
6 | ''' | |
7 | __version__ = "2.3" No newline at end of file |
|
7 | ||
|
8 | __version__ = "2.3" |
@@ -326,10 +326,10 class OperationConf(): | |||||
326 | def getParameterValue(self, parameterName): |
|
326 | def getParameterValue(self, parameterName): | |
327 |
|
327 | |||
328 | parameterObj = self.getParameterObj(parameterName) |
|
328 | parameterObj = self.getParameterObj(parameterName) | |
329 |
|
329 | |||
330 | # if not parameterObj: |
|
330 | # if not parameterObj: | |
331 | # return None |
|
331 | # return None | |
332 |
|
332 | |||
333 | value = parameterObj.getValue() |
|
333 | value = parameterObj.getValue() | |
334 |
|
334 | |||
335 | return value |
|
335 | return value | |
@@ -647,11 +647,11 class ProcUnitConf(): | |||||
647 | def printattr(self): |
|
647 | def printattr(self): | |
648 |
|
648 | |||
649 | print "%s[%s]: name = %s, datatype = %s, inputId = %s" %(self.ELEMENTNAME, |
|
649 | print "%s[%s]: name = %s, datatype = %s, inputId = %s" %(self.ELEMENTNAME, | |
650 | self.id, |
|
650 | self.id, | |
651 | self.name, |
|
651 | self.name, | |
652 | self.datatype, |
|
652 | self.datatype, | |
653 | self.inputId) |
|
653 | self.inputId) | |
654 |
|
654 | |||
655 | for opConfObj in self.opConfObjList: |
|
655 | for opConfObj in self.opConfObjList: | |
656 | opConfObj.printattr() |
|
656 | opConfObj.printattr() | |
657 |
|
657 | |||
@@ -706,13 +706,13 class ProcUnitConf(): | |||||
706 | sts = self.procUnitObj.call(opType = opConfObj.type, |
|
706 | sts = self.procUnitObj.call(opType = opConfObj.type, | |
707 | opName = opConfObj.name, |
|
707 | opName = opConfObj.name, | |
708 | opId = opConfObj.id, |
|
708 | opId = opConfObj.id, | |
709 | ) |
|
709 | **kwargs) | |
710 |
|
710 | |||
711 | # total_time = time.time() - ini |
|
711 | # total_time = time.time() - ini | |
712 | # |
|
712 | # | |
713 | # if total_time > 0.002: |
|
713 | # if total_time > 0.002: | |
714 | # print "%s::%s took %f seconds" %(self.name, opConfObj.name, total_time) |
|
714 | # print "%s::%s took %f seconds" %(self.name, opConfObj.name, total_time) | |
715 |
|
715 | |||
716 | is_ok = is_ok or sts |
|
716 | is_ok = is_ok or sts | |
717 |
|
717 | |||
718 | return is_ok |
|
718 | return is_ok | |
@@ -858,18 +858,18 class ReadUnitConf(ProcUnitConf): | |||||
858 | opObj.addParameter(name=key, value=value, format=type(value).__name__) |
|
858 | opObj.addParameter(name=key, value=value, format=type(value).__name__) | |
859 |
|
859 | |||
860 | return opObj |
|
860 | return opObj | |
861 |
|
861 | |||
862 | # def makeXml(self, projectElement): |
|
862 | # def makeXml(self, projectElement): | |
863 | # |
|
863 | # | |
864 | # procUnitElement = SubElement(projectElement, self.ELEMENTNAME) |
|
864 | # procUnitElement = SubElement(projectElement, self.ELEMENTNAME) | |
865 | # procUnitElement.set('id', str(self.id)) |
|
865 | # procUnitElement.set('id', str(self.id)) | |
866 | # procUnitElement.set('name', self.name) |
|
866 | # procUnitElement.set('name', self.name) | |
867 | # procUnitElement.set('datatype', self.datatype) |
|
867 | # procUnitElement.set('datatype', self.datatype) | |
868 | # procUnitElement.set('inputId', str(self.inputId)) |
|
868 | # procUnitElement.set('inputId', str(self.inputId)) | |
869 | # |
|
869 | # | |
870 | # for opConfObj in self.opConfObjList: |
|
870 | # for opConfObj in self.opConfObjList: | |
871 | # opConfObj.makeXml(procUnitElement) |
|
871 | # opConfObj.makeXml(procUnitElement) | |
872 |
|
872 | |||
873 | def readXml(self, upElement): |
|
873 | def readXml(self, upElement): | |
874 |
|
874 | |||
875 | self.id = upElement.get('id') |
|
875 | self.id = upElement.get('id') | |
@@ -1111,8 +1111,8 class Project(): | |||||
1111 |
|
1111 | |||
1112 | self.id = self.projectElement.get('id') |
|
1112 | self.id = self.projectElement.get('id') | |
1113 | self.name = self.projectElement.get('name') |
|
1113 | self.name = self.projectElement.get('name') | |
1114 | self.description = self.projectElement.get('description') |
|
1114 | self.description = self.projectElement.get('description') | |
1115 |
|
1115 | |||
1116 | readUnitElementList = self.projectElement.iter(ReadUnitConf().getElementName()) |
|
1116 | readUnitElementList = self.projectElement.iter(ReadUnitConf().getElementName()) | |
1117 |
|
1117 | |||
1118 | for readUnitElement in readUnitElementList: |
|
1118 | for readUnitElement in readUnitElementList: | |
@@ -1142,8 +1142,8 class Project(): | |||||
1142 | def printattr(self): |
|
1142 | def printattr(self): | |
1143 |
|
1143 | |||
1144 | print "Project[%s]: name = %s, description = %s" %(self.id, |
|
1144 | print "Project[%s]: name = %s, description = %s" %(self.id, | |
1145 |
|
|
1145 | self.name, | |
1146 |
|
|
1146 | self.description) | |
1147 |
|
1147 | |||
1148 | for procUnitConfObj in self.procUnitConfObjDict.values(): |
|
1148 | for procUnitConfObj in self.procUnitConfObjDict.values(): | |
1149 | procUnitConfObj.printattr() |
|
1149 | procUnitConfObj.printattr() | |
@@ -1180,9 +1180,9 class Project(): | |||||
1180 | import socket |
|
1180 | import socket | |
1181 |
|
1181 | |||
1182 | err = traceback.format_exception(sys.exc_info()[0], |
|
1182 | err = traceback.format_exception(sys.exc_info()[0], | |
1183 |
|
|
1183 | sys.exc_info()[1], | |
1184 |
|
|
1184 | sys.exc_info()[2]) | |
1185 |
|
1185 | |||
1186 | print "***** Error occurred in %s *****" %(procUnitConfObj.name) |
|
1186 | print "***** Error occurred in %s *****" %(procUnitConfObj.name) | |
1187 | print "***** %s" %err[-1] |
|
1187 | print "***** %s" %err[-1] | |
1188 |
|
1188 | |||
@@ -1213,10 +1213,10 class Project(): | |||||
1213 |
|
1213 | |||
1214 | adminObj = schainpy.admin.SchainNotify() |
|
1214 | adminObj = schainpy.admin.SchainNotify() | |
1215 | adminObj.sendAlert(message=message, |
|
1215 | adminObj.sendAlert(message=message, | |
1216 | subject=subject, |
|
1216 | subject=subject, | |
1217 | subtitle=subtitle, |
|
1217 | subtitle=subtitle, | |
1218 | filename=self.filename) |
|
1218 | filename=self.filename) | |
1219 |
|
1219 | |||
1220 | def isPaused(self): |
|
1220 | def isPaused(self): | |
1221 | return 0 |
|
1221 | return 0 | |
1222 |
|
1222 |
@@ -14,8 +14,8 class ControllerThread(threading.Thread, Project): | |||||
14 | self.setDaemon(True) |
|
14 | self.setDaemon(True) | |
15 |
|
15 | |||
16 | self.lock = threading.Lock() |
|
16 | self.lock = threading.Lock() | |
17 | self.control = {'stop':False, 'pause':False} |
|
17 | self.control = { 'stop':False, 'pause':False } | |
18 |
|
18 | |||
19 | def __del__(self): |
|
19 | def __del__(self): | |
20 |
|
20 | |||
21 | self.control['stop'] = True |
|
21 | self.control['stop'] = True | |
@@ -90,7 +90,7 class ControllerThread(threading.Thread, Project): | |||||
90 | continue |
|
90 | continue | |
91 |
|
91 | |||
92 | if thisOpObj.name in plotterList: |
|
92 | if thisOpObj.name in plotterList: | |
93 |
thisOpObj.type = " |
|
93 | thisOpObj.type = "other" | |
94 |
|
94 | |||
95 | def setPlotterQueue(self, plotter_queue): |
|
95 | def setPlotterQueue(self, plotter_queue): | |
96 |
|
96 |
@@ -292,11 +292,9 class JROData(GenericData): | |||||
292 | return fmax |
|
292 | return fmax | |
293 |
|
293 | |||
294 | def getFmax(self): |
|
294 | def getFmax(self): | |
295 |
|
||||
296 | PRF = 1./(self.ippSeconds * self.nCohInt) |
|
295 | PRF = 1./(self.ippSeconds * self.nCohInt) | |
297 |
|
296 | |||
298 | fmax = PRF |
|
297 | fmax = PRF | |
299 |
|
||||
300 | return fmax |
|
298 | return fmax | |
301 |
|
299 | |||
302 | def getVmax(self): |
|
300 | def getVmax(self): |
@@ -24,8 +24,8 class SpectraPlot(Figure): | |||||
24 | self.isConfig = False |
|
24 | self.isConfig = False | |
25 | self.__nsubplots = 1 |
|
25 | self.__nsubplots = 1 | |
26 |
|
26 | |||
27 |
self.WIDTH = |
|
27 | self.WIDTH = 1000 | |
28 |
self.HEIGHT = |
|
28 | self.HEIGHT = 1000 | |
29 | self.WIDTHPROF = 120 |
|
29 | self.WIDTHPROF = 120 | |
30 | self.HEIGHTPROF = 0 |
|
30 | self.HEIGHTPROF = 0 | |
31 | self.counter_imagwr = 0 |
|
31 | self.counter_imagwr = 0 |
@@ -124,7 +124,7 class PlotManager(): | |||||
124 | subtitle = "%s:\n" %(name) |
|
124 | subtitle = "%s:\n" %(name) | |
125 | subtitle += "Hostname: %s\n" %socket.gethostbyname(socket.gethostname()) |
|
125 | subtitle += "Hostname: %s\n" %socket.gethostbyname(socket.gethostname()) | |
126 | subtitle += "Working directory: %s\n" %os.path.abspath("./") |
|
126 | subtitle += "Working directory: %s\n" %os.path.abspath("./") | |
127 | # subtitle += "Configuration file: %s\n" %self.filename |
|
127 | # subtitle += "Configuration file: %s\n" %self.filename | |
128 | subtitle += "Time: %s\n" %str(datetime.datetime.now()) |
|
128 | subtitle += "Time: %s\n" %str(datetime.datetime.now()) | |
129 |
|
129 | |||
130 | adminObj = schainpy.admin.SchainNotify() |
|
130 | adminObj = schainpy.admin.SchainNotify() |
@@ -625,7 +625,7 class JRODataReader(JRODataIO): | |||||
625 | dateList, pathList = self.findDatafiles(path, startDate, endDate, expLabel, ext, walk, include_path=True) |
|
625 | dateList, pathList = self.findDatafiles(path, startDate, endDate, expLabel, ext, walk, include_path=True) | |
626 |
|
626 | |||
627 | if dateList == []: |
|
627 | if dateList == []: | |
628 | # print "[Reading] Date range selected invalid [%s - %s]: No *%s files in %s)" %(startDate, endDate, ext, path) |
|
628 | # print "[Reading] Date range selected invalid [%s - %s]: No *%s files in %s)" %(startDate, endDate, ext, path) | |
629 | return None, None |
|
629 | return None, None | |
630 |
|
630 | |||
631 | if len(dateList) > 1: |
|
631 | if len(dateList) > 1: | |
@@ -637,8 +637,8 class JRODataReader(JRODataIO): | |||||
637 | datetimeList = [] |
|
637 | datetimeList = [] | |
638 |
|
638 | |||
639 | for thisPath in pathList: |
|
639 | for thisPath in pathList: | |
640 | # thisPath = pathList[pathDict[file]] |
|
640 | # thisPath = pathList[pathDict[file]] | |
641 |
|
641 | |||
642 | fileList = glob.glob1(thisPath, "*%s" %ext) |
|
642 | fileList = glob.glob1(thisPath, "*%s" %ext) | |
643 | fileList.sort() |
|
643 | fileList.sort() | |
644 |
|
644 | |||
@@ -767,7 +767,7 class JRODataReader(JRODataIO): | |||||
767 | idFile += 1 |
|
767 | idFile += 1 | |
768 | if not(idFile < len(self.filenameList)): |
|
768 | if not(idFile < len(self.filenameList)): | |
769 | self.flagNoMoreFiles = 1 |
|
769 | self.flagNoMoreFiles = 1 | |
770 | # print "[Reading] No more Files" |
|
770 | # print "[Reading] No more Files" | |
771 | return 0 |
|
771 | return 0 | |
772 |
|
772 | |||
773 | filename = self.filenameList[idFile] |
|
773 | filename = self.filenameList[idFile] | |
@@ -785,7 +785,7 class JRODataReader(JRODataIO): | |||||
785 | self.fileSize = fileSize |
|
785 | self.fileSize = fileSize | |
786 | self.fp = fp |
|
786 | self.fp = fp | |
787 |
|
787 | |||
788 | # print "[Reading] Setting the file: %s"%self.filename |
|
788 | # print "[Reading] Setting the file: %s"%self.filename | |
789 |
|
789 | |||
790 | return 1 |
|
790 | return 1 | |
791 |
|
791 | |||
@@ -868,14 +868,14 class JRODataReader(JRODataIO): | |||||
868 | if self.fp != None: self.fp.close() |
|
868 | if self.fp != None: self.fp.close() | |
869 | self.fp = open(fullfilename, 'rb') |
|
869 | self.fp = open(fullfilename, 'rb') | |
870 | self.flagNoMoreFiles = 0 |
|
870 | self.flagNoMoreFiles = 0 | |
871 | # print '[Reading] Setting the file: %s' % fullfilename |
|
871 | # print '[Reading] Setting the file: %s' % fullfilename | |
872 | else: |
|
872 | else: | |
873 | self.fileSize = 0 |
|
873 | self.fileSize = 0 | |
874 | self.filename = None |
|
874 | self.filename = None | |
875 | self.flagIsNewFile = 0 |
|
875 | self.flagIsNewFile = 0 | |
876 | self.fp = None |
|
876 | self.fp = None | |
877 | self.flagNoMoreFiles = 1 |
|
877 | self.flagNoMoreFiles = 1 | |
878 | # print '[Reading] No more files to read' |
|
878 | # print '[Reading] No more files to read' | |
879 |
|
879 | |||
880 | return fileOk_flag |
|
880 | return fileOk_flag | |
881 |
|
881 | |||
@@ -929,7 +929,7 class JRODataReader(JRODataIO): | |||||
929 | return 1 |
|
929 | return 1 | |
930 |
|
930 | |||
931 | if self.fileSize == self.fileSizeByHeader: |
|
931 | if self.fileSize == self.fileSizeByHeader: | |
932 | # self.flagEoF = True |
|
932 | # self.flagEoF = True | |
933 | return 0 |
|
933 | return 0 | |
934 |
|
934 | |||
935 | print "[Reading] Waiting %0.2f seconds for the next block, try %03d ..." % (self.delay, nTries+1) |
|
935 | print "[Reading] Waiting %0.2f seconds for the next block, try %03d ..." % (self.delay, nTries+1) | |
@@ -983,13 +983,13 class JRODataReader(JRODataIO): | |||||
983 | else: |
|
983 | else: | |
984 | self.fp.seek(self.fp.tell() - neededsize) |
|
984 | self.fp.seek(self.fp.tell() - neededsize) | |
985 | break |
|
985 | break | |
986 |
|
986 | |||
987 | # csize = self.fileSize - self.fp.tell() |
|
987 | # csize = self.fileSize - self.fp.tell() | |
988 | # neededsize = self.processingHeaderObj.blockSize + self.basicHeaderSize |
|
988 | # neededsize = self.processingHeaderObj.blockSize + self.basicHeaderSize | |
989 | # factor = int(csize/neededsize) |
|
989 | # factor = int(csize/neededsize) | |
990 | # if factor > 0: |
|
990 | # if factor > 0: | |
991 | # self.fp.seek(self.fp.tell() + factor*neededsize) |
|
991 | # self.fp.seek(self.fp.tell() + factor*neededsize) | |
992 |
|
992 | |||
993 | self.flagIsNewFile = 0 |
|
993 | self.flagIsNewFile = 0 | |
994 | self.__isFirstTimeOnline = 0 |
|
994 | self.__isFirstTimeOnline = 0 | |
995 |
|
995 | |||
@@ -997,10 +997,10 class JRODataReader(JRODataIO): | |||||
997 | #if self.server is None: |
|
997 | #if self.server is None: | |
998 | if self.fp == None: |
|
998 | if self.fp == None: | |
999 | return 0 |
|
999 | return 0 | |
1000 |
|
1000 | |||
1001 | # if self.online: |
|
1001 | # if self.online: | |
1002 | # self.__jumpToLastBlock() |
|
1002 | # self.__jumpToLastBlock() | |
1003 |
|
1003 | |||
1004 | if self.flagIsNewFile: |
|
1004 | if self.flagIsNewFile: | |
1005 | self.lastUTTime = self.basicHeaderObj.utc |
|
1005 | self.lastUTTime = self.basicHeaderObj.utc | |
1006 | return 1 |
|
1006 | return 1 | |
@@ -1094,8 +1094,8 class JRODataReader(JRODataIO): | |||||
1094 | self.dtype = datatype_str |
|
1094 | self.dtype = datatype_str | |
1095 | #self.ippSeconds = 2 * 1000 * self.radarControllerHeaderObj.ipp / self.c |
|
1095 | #self.ippSeconds = 2 * 1000 * self.radarControllerHeaderObj.ipp / self.c | |
1096 | self.fileSizeByHeader = self.processingHeaderObj.dataBlocksPerFile * self.processingHeaderObj.blockSize + self.firstHeaderSize + self.basicHeaderSize*(self.processingHeaderObj.dataBlocksPerFile - 1) |
|
1096 | self.fileSizeByHeader = self.processingHeaderObj.dataBlocksPerFile * self.processingHeaderObj.blockSize + self.firstHeaderSize + self.basicHeaderSize*(self.processingHeaderObj.dataBlocksPerFile - 1) | |
1097 | # self.dataOut.channelList = numpy.arange(self.systemHeaderObj.numChannels) |
|
1097 | # self.dataOut.channelList = numpy.arange(self.systemHeaderObj.numChannels) | |
1098 | # self.dataOut.channelIndexList = numpy.arange(self.systemHeaderObj.numChannels) |
|
1098 | # self.dataOut.channelIndexList = numpy.arange(self.systemHeaderObj.numChannels) | |
1099 | self.getBlockDimension() |
|
1099 | self.getBlockDimension() | |
1100 |
|
1100 | |||
1101 | def __verifyFile(self, filename, msgFlag=True): |
|
1101 | def __verifyFile(self, filename, msgFlag=True): | |
@@ -1340,47 +1340,14 class JRODataReader(JRODataIO): | |||||
1340 | return |
|
1340 | return | |
1341 |
|
1341 | |||
1342 | self.fileIndex = -1 |
|
1342 | self.fileIndex = -1 | |
1343 |
self.pathList = |
|
1343 | self.pathList = [] | |
1344 |
self.filenameList = |
|
1344 | self.filenameList = [] | |
1345 | file_name = os.path.basename(filenameList[-1]) |
|
1345 | return | |
1346 | basename, ext = os.path.splitext(file_name) |
|
1346 | ||
1347 | last_set = int(basename[-3:]) |
|
1347 | # self.getBasicHeader() | |
1348 |
|
1348 | |||
1349 | self.online = online |
|
1349 | if last_set != None: | |
1350 | self.realtime = realtime |
|
1350 | self.dataOut.last_block = last_set * self.processingHeaderObj.dataBlocksPerFile + self.basicHeaderObj.dataBlock | |
1351 | self.delay = delay |
|
|||
1352 | ext = ext.lower() |
|
|||
1353 | self.ext = ext |
|
|||
1354 | self.getByBlock = getblock |
|
|||
1355 | self.nTxs = nTxs |
|
|||
1356 | self.startTime = startTime |
|
|||
1357 | self.endTime = endTime |
|
|||
1358 |
|
||||
1359 | #Added----------------- |
|
|||
1360 | self.selBlocksize = blocksize |
|
|||
1361 | self.selBlocktime = blocktime |
|
|||
1362 |
|
||||
1363 | # Verbose----------- |
|
|||
1364 | self.verbose = verbose |
|
|||
1365 | self.warnings = warnings |
|
|||
1366 |
|
||||
1367 | if not(self.setNextFile()): |
|
|||
1368 | if (startDate!=None) and (endDate!=None): |
|
|||
1369 | print "[Reading] No files in range: %s - %s" %(datetime.datetime.combine(startDate,startTime).ctime(), datetime.datetime.combine(endDate,endTime).ctime()) |
|
|||
1370 | elif startDate != None: |
|
|||
1371 | print "[Reading] No files in range: %s" %(datetime.datetime.combine(startDate,startTime).ctime()) |
|
|||
1372 | else: |
|
|||
1373 | print "[Reading] No files" |
|
|||
1374 |
|
||||
1375 | self.fileIndex = -1 |
|
|||
1376 | self.pathList = [] |
|
|||
1377 | self.filenameList = [] |
|
|||
1378 | return |
|
|||
1379 |
|
||||
1380 | # self.getBasicHeader() |
|
|||
1381 |
|
||||
1382 | if last_set != None: |
|
|||
1383 | self.dataOut.last_block = last_set * self.processingHeaderObj.dataBlocksPerFile + self.basicHeaderObj.dataBlock |
|
|||
1384 | return |
|
1351 | return | |
1385 |
|
1352 | |||
1386 | def getBasicHeader(self): |
|
1353 | def getBasicHeader(self): |
@@ -55,6 +55,7 class DigitalRFReader(ProcessingUnit): | |||||
55 | self.__nCode = None |
|
55 | self.__nCode = None | |
56 | self.__nBaud = None |
|
56 | self.__nBaud = None | |
57 | self.__code = None |
|
57 | self.__code = None | |
|
58 | self.dtype = None | |||
58 |
|
59 | |||
59 | def close(self): |
|
60 | def close(self): | |
60 | print 'Average of writing to digital rf format is ', self.oldAverage * 1000 |
|
61 | print 'Average of writing to digital rf format is ', self.oldAverage * 1000 | |
@@ -73,13 +74,30 class DigitalRFReader(ProcessingUnit): | |||||
73 | ippSeconds = 1.0*self.__nSamples/self.__sample_rate |
|
74 | ippSeconds = 1.0*self.__nSamples/self.__sample_rate | |
74 |
|
75 | |||
75 | nProfiles = 1.0/ippSeconds # Number of profiles in one second |
|
76 | nProfiles = 1.0/ippSeconds # Number of profiles in one second | |
76 |
|
77 | |||
77 | self.dataOut.radarControllerHeaderObj = RadarControllerHeader(self.__radarControllerHeader) |
|
78 | try: | |
78 |
|
79 | self.dataOut.radarControllerHeaderObj = RadarControllerHeader(self.__radarControllerHeader) | ||
79 | self.dataOut.systemHeaderObj = SystemHeader(self.__systemHeader) |
|
80 | except: | |
80 |
|
81 | self.dataOut.radarControllerHeaderObj = RadarControllerHeader( | ||
|
82 | txA=0, | |||
|
83 | txB=0, | |||
|
84 | nWindows=1, | |||
|
85 | nHeights=self.__nSamples, | |||
|
86 | firstHeight=self.__firstHeigth, | |||
|
87 | deltaHeight=self.__deltaHeigth, | |||
|
88 | codeType=self.__codeType, | |||
|
89 | nCode=self.__nCode, nBaud=self.__nBaud, | |||
|
90 | code = self.__code) | |||
|
91 | ||||
|
92 | try: | |||
|
93 | self.dataOut.systemHeaderObj = SystemHeader(self.__systemHeader) | |||
|
94 | except: | |||
|
95 | self.dataOut.systemHeaderObj = SystemHeader(nSamples=self.__nSamples, | |||
|
96 | nProfiles=nProfiles, | |||
|
97 | nChannels=len(self.__channelList), | |||
|
98 | adcResolution=14) | |||
81 | self.dataOut.type = "Voltage" |
|
99 | self.dataOut.type = "Voltage" | |
82 |
|
100 | |||
83 | self.dataOut.data = None |
|
101 | self.dataOut.data = None | |
84 |
|
102 | |||
85 | self.dataOut.dtype = self.dtype |
|
103 | self.dataOut.dtype = self.dtype | |
@@ -112,15 +130,19 class DigitalRFReader(ProcessingUnit): | |||||
112 |
|
130 | |||
113 | self.dataOut.errorCount = 0 |
|
131 | self.dataOut.errorCount = 0 | |
114 |
|
132 | |||
115 | self.dataOut.nCohInt = self.fixed_metadata_dict['nCohInt'] |
|
133 | try: | |
|
134 | self.dataOut.nCohInt = self.fixed_metadata_dict.get('nCohInt', 1) | |||
116 |
|
135 | |||
117 | self.dataOut.flagDecodeData = self.fixed_metadata_dict['flagDecodeData'] # asumo que la data esta decodificada |
|
136 | self.dataOut.flagDecodeData = self.fixed_metadata_dict['flagDecodeData'] # asumo que la data esta decodificada | |
118 |
|
137 | |||
119 | self.dataOut.flagDeflipData = self.fixed_metadata_dict['flagDeflipData'] # asumo que la data esta sin flip |
|
138 | self.dataOut.flagDeflipData = self.fixed_metadata_dict['flagDeflipData'] # asumo que la data esta sin flip | |
120 |
|
139 | |||
121 | self.dataOut.flagShiftFFT = self.fixed_metadata_dict['flagShiftFFT'] |
|
140 | self.dataOut.flagShiftFFT = self.fixed_metadata_dict['flagShiftFFT'] | |
122 |
|
141 | |||
123 | self.dataOut.useLocalTime = self.fixed_metadata_dict['useLocalTime'] |
|
142 | self.dataOut.useLocalTime = self.fixed_metadata_dict['useLocalTime'] | |
|
143 | except: | |||
|
144 | pass | |||
|
145 | ||||
124 |
|
146 | |||
125 | self.dataOut.ippSeconds = ippSeconds |
|
147 | self.dataOut.ippSeconds = ippSeconds | |
126 |
|
148 | |||
@@ -237,35 +259,29 class DigitalRFReader(ProcessingUnit): | |||||
237 | top_properties = self.digitalReadObj.get_properties(channelNameList[channelList[0]]) |
|
259 | top_properties = self.digitalReadObj.get_properties(channelNameList[channelList[0]]) | |
238 |
|
260 | |||
239 |
|
261 | |||
240 |
|
||||
241 |
|
||||
242 |
|
||||
243 |
|
||||
244 | self.__num_subchannels = top_properties['num_subchannels'] |
|
262 | self.__num_subchannels = top_properties['num_subchannels'] | |
245 | self.__sample_rate = 1.0 * top_properties['sample_rate_numerator'] / top_properties['sample_rate_denominator'] |
|
263 | self.__sample_rate = 1.0 * top_properties['sample_rate_numerator'] / top_properties['sample_rate_denominator'] | |
246 |
|
||||
247 | # self.__samples_per_file = top_properties['samples_per_file'][0] |
|
264 | # self.__samples_per_file = top_properties['samples_per_file'][0] | |
248 | self.__deltaHeigth = 1e6*0.15/self.__sample_rate ## why 0.15? |
|
265 | self.__deltaHeigth = 1e6*0.15/self.__sample_rate ## why 0.15? | |
249 |
|
266 | |||
250 | this_metadata_file = self.digitalReadObj.get_digital_metadata(channelNameList[channelList[0]]) |
|
267 | this_metadata_file = self.digitalReadObj.get_digital_metadata(channelNameList[channelList[0]]) | |
251 | metadata_bounds = this_metadata_file.get_bounds() |
|
268 | metadata_bounds = this_metadata_file.get_bounds() | |
252 | self.fixed_metadata_dict = this_metadata_file.read(metadata_bounds[0])[metadata_bounds[0]] ## GET FIRST HEADER |
|
269 | self.fixed_metadata_dict = this_metadata_file.read(metadata_bounds[0])[metadata_bounds[0]] ## GET FIRST HEADER | |
253 | self.__processingHeader = self.fixed_metadata_dict['processingHeader'] |
|
270 | ||
254 | self.__radarControllerHeader = self.fixed_metadata_dict['radarControllerHeader'] |
|
271 | try: | |
255 |
self.__ |
|
272 | self.__processingHeader = self.fixed_metadata_dict['processingHeader'] | |
256 |
self. |
|
273 | self.__radarControllerHeader = self.fixed_metadata_dict['radarControllerHeader'] | |
|
274 | self.__systemHeader = self.fixed_metadata_dict['systemHeader'] | |||
|
275 | self.dtype = cPickle.loads(self.fixed_metadata_dict['dtype']) | |||
|
276 | except: | |||
|
277 | pass | |||
|
278 | ||||
257 |
|
279 | |||
258 | self.__frequency = None |
|
280 | self.__frequency = None | |
259 |
|
281 | |||
260 | try: |
|
282 | self.__frequency = self.fixed_metadata_dict.get('frequency', 1) | |
261 | self.__frequency = self.fixed_metadata_dict['frequency'] |
|
|||
262 | except: |
|
|||
263 | self.__frequency = None |
|
|||
264 |
|
283 | |||
265 | try: |
|
284 | self.__timezone = self.fixed_metadata_dict.get('timezone', 300) | |
266 | self.__timezone = self.fixed_metadata_dict['timezone'] * 60 |
|
|||
267 | except: |
|
|||
268 | self.__timezone = 0 |
|
|||
269 |
|
285 | |||
270 |
|
286 | |||
271 | try: |
|
287 | try: | |
@@ -284,11 +300,15 class DigitalRFReader(ProcessingUnit): | |||||
284 | nBaud = 1 |
|
300 | nBaud = 1 | |
285 | code = numpy.ones((nCode, nBaud), dtype=numpy.int) |
|
301 | code = numpy.ones((nCode, nBaud), dtype=numpy.int) | |
286 |
|
302 | |||
287 |
|
|
303 | try: | |
288 | nCode = self.__radarControllerHeader['nCode'] |
|
304 | if codeType: | |
289 |
|
|
305 | nCode = self.__radarControllerHeader['nCode'] | |
290 |
|
|
306 | nBaud = self.__radarControllerHeader['nBaud'] | |
291 |
|
307 | code = self.__radarControllerHeader['code'] | ||
|
308 | except: | |||
|
309 | pass | |||
|
310 | ||||
|
311 | ||||
292 | if not ippKm: |
|
312 | if not ippKm: | |
293 | try: |
|
313 | try: | |
294 | # seconds to km |
|
314 | # seconds to km | |
@@ -296,6 +316,7 class DigitalRFReader(ProcessingUnit): | |||||
296 | except: |
|
316 | except: | |
297 | ippKm = None |
|
317 | ippKm = None | |
298 | #################################################### |
|
318 | #################################################### | |
|
319 | self.__ippKm = ippKm | |||
299 | startUTCSecond = None |
|
320 | startUTCSecond = None | |
300 | endUTCSecond = None |
|
321 | endUTCSecond = None | |
301 |
|
322 | |||
@@ -336,7 +357,7 class DigitalRFReader(ProcessingUnit): | |||||
336 | self.profileIndex = 0 |
|
357 | self.profileIndex = 0 | |
337 | self.i= 0 |
|
358 | self.i= 0 | |
338 | self.__delay = delay |
|
359 | self.__delay = delay | |
339 | self.__ippKm = ippKm |
|
360 | ||
340 | self.__codeType = codeType |
|
361 | self.__codeType = codeType | |
341 | self.__nCode = nCode |
|
362 | self.__nCode = nCode | |
342 | self.__nBaud = nBaud |
|
363 | self.__nBaud = nBaud | |
@@ -587,6 +608,7 class DigitalRFWriter(Operation): | |||||
587 | Operation.__init__(self, **kwargs) |
|
608 | Operation.__init__(self, **kwargs) | |
588 | self.metadata_dict = {} |
|
609 | self.metadata_dict = {} | |
589 | self.dataOut = None |
|
610 | self.dataOut = None | |
|
611 | self.dtype = None | |||
590 |
|
612 | |||
591 | def setHeader(self): |
|
613 | def setHeader(self): | |
592 |
|
614 |
@@ -109,7 +109,7 class SpectraProc(ProcessingUnit): | |||||
109 |
|
109 | |||
110 | if self.dataIn.type == "Spectra": |
|
110 | if self.dataIn.type == "Spectra": | |
111 | self.dataOut.copy(self.dataIn) |
|
111 | self.dataOut.copy(self.dataIn) | |
112 | # self.__selectPairs(pairsList) |
|
112 | # self.__selectPairs(pairsList) | |
113 | return True |
|
113 | return True | |
114 |
|
114 | |||
115 | if self.dataIn.type == "Voltage": |
|
115 | if self.dataIn.type == "Voltage": | |
@@ -878,7 +878,6 class IncohInt(Operation): | |||||
878 | return self.__initime, avgdata_spc, avgdata_cspc, avgdata_dc |
|
878 | return self.__initime, avgdata_spc, avgdata_cspc, avgdata_dc | |
879 |
|
879 | |||
880 | def run(self, dataOut, n=None, timeInterval=None, overlapping=False): |
|
880 | def run(self, dataOut, n=None, timeInterval=None, overlapping=False): | |
881 |
|
||||
882 | if n==1: |
|
881 | if n==1: | |
883 | return |
|
882 | return | |
884 |
|
883 |
@@ -32,9 +32,7 class Remote(Thread): | |||||
32 | None |
|
32 | None | |
33 |
|
33 | |||
34 | Written by: |
|
34 | Written by: | |
35 |
|
35 | "Miguel Urco":mailto:miguel.urco@jro.igp.gob.pe Jun. 03, 2015 | ||
36 | "Miguel Urco":mailto:miguel.urco@jro.igp.gob.pe Jun. 03, 2015 |
|
|||
37 |
|
||||
38 | """ |
|
36 | """ | |
39 |
|
37 | |||
40 | server = None |
|
38 | server = None | |
@@ -168,13 +166,13 class Remote(Thread): | |||||
168 | return 0 |
|
166 | return 0 | |
169 |
|
167 | |||
170 | self.mutex.acquire() |
|
168 | self.mutex.acquire() | |
171 | # init = time.time() |
|
169 | # init = time.time() | |
172 | # |
|
170 | # | |
173 | # while(self.bussy): |
|
171 | # while(self.bussy): | |
174 | # sleep(0.1) |
|
172 | # sleep(0.1) | |
175 | # if time.time() - init > 2*self.period: |
|
173 | # if time.time() - init > 2*self.period: | |
176 | # return 0 |
|
174 | # return 0 | |
177 |
|
175 | |||
178 | self.fileList = fileList |
|
176 | self.fileList = fileList | |
179 | self.mutex.release() |
|
177 | self.mutex.release() | |
180 | return 1 |
|
178 | return 1 | |
@@ -197,8 +195,8 class Remote(Thread): | |||||
197 |
|
195 | |||
198 | if self.stopFlag: |
|
196 | if self.stopFlag: | |
199 | break |
|
197 | break | |
200 |
|
198 | |||
201 | # self.bussy = True |
|
199 | # self.bussy = True | |
202 | self.mutex.acquire() |
|
200 | self.mutex.acquire() | |
203 |
|
201 | |||
204 | print "[Remote Server] Opening %s" %self.__server |
|
202 | print "[Remote Server] Opening %s" %self.__server | |
@@ -400,21 +398,21 class SSHClient(Remote): | |||||
400 | def open(self, server, username, password, remotefolder, port=22): |
|
398 | def open(self, server, username, password, remotefolder, port=22): | |
401 |
|
399 | |||
402 | """ |
|
400 | """ | |
403 | This method is used to set SSH parameters and establish a connection to a remote server |
|
401 | This method is used to set SSH parameters and establish a connection to a remote server | |
404 |
|
402 | |||
405 | Inputs: |
|
403 | Inputs: | |
406 | server - remote server IP Address |
|
404 | server - remote server IP Address | |
407 |
|
405 | |||
408 | username - remote server Username |
|
406 | username - remote server Username | |
409 |
|
407 | |||
410 | password - remote server password |
|
408 | password - remote server password | |
411 |
|
409 | |||
412 | remotefolder - remote server current working directory |
|
410 | remotefolder - remote server current working directory | |
413 |
|
411 | |||
414 | Return: void |
|
412 | Return: void | |
415 |
|
413 | |||
416 | Affects: |
|
414 | Affects: | |
417 | self.status - in case of error or fail connection this parameter is set to 0 else 1 |
|
415 | self.status - in case of error or fail connection this parameter is set to 0 else 1 | |
418 |
|
416 | |||
419 | """ |
|
417 | """ | |
420 | import socket |
|
418 | import socket | |
@@ -440,7 +438,7 class SSHClient(Remote): | |||||
440 | try: |
|
438 | try: | |
441 | sshClientObj.connect(server, username=username, password=password, port=port) |
|
439 | sshClientObj.connect(server, username=username, password=password, port=port) | |
442 | except paramiko.AuthenticationException, e: |
|
440 | except paramiko.AuthenticationException, e: | |
443 | # print "SSH username or password are incorrect: %s" |
|
441 | # print "SSH username or password are incorrect: %s" | |
444 | print "[SSH Server]:", e |
|
442 | print "[SSH Server]:", e | |
445 | return 0 |
|
443 | return 0 | |
446 | except SSHException, e: |
|
444 | except SSHException, e: | |
@@ -474,7 +472,7 class SSHClient(Remote): | |||||
474 |
|
472 | |||
475 | def close(self): |
|
473 | def close(self): | |
476 | """ |
|
474 | """ | |
477 | Close connection to remote server |
|
475 | Close connection to remote server | |
478 | """ |
|
476 | """ | |
479 | if not self.status: |
|
477 | if not self.status: | |
480 | return 0 |
|
478 | return 0 | |
@@ -484,13 +482,13 class SSHClient(Remote): | |||||
484 |
|
482 | |||
485 | def __execute(self, command): |
|
483 | def __execute(self, command): | |
486 | """ |
|
484 | """ | |
487 | __execute a command on remote server |
|
485 | __execute a command on remote server | |
488 |
|
486 | |||
489 | Input: |
|
487 | Input: | |
490 | command - Exmaple 'ls -l' |
|
488 | command - Exmaple 'ls -l' | |
491 |
|
489 | |||
492 | Return: |
|
490 | Return: | |
493 | 0 in error case else 1 |
|
491 | 0 in error case else 1 | |
494 | """ |
|
492 | """ | |
495 | if not self.status: |
|
493 | if not self.status: | |
496 | return 0 |
|
494 | return 0 | |
@@ -509,13 +507,13 class SSHClient(Remote): | |||||
509 |
|
507 | |||
510 | def mkdir(self, remotefolder): |
|
508 | def mkdir(self, remotefolder): | |
511 | """ |
|
509 | """ | |
512 | mkdir is used to make a new directory in remote server |
|
510 | mkdir is used to make a new directory in remote server | |
513 |
|
511 | |||
514 | Input: |
|
512 | Input: | |
515 | remotefolder - directory name |
|
513 | remotefolder - directory name | |
516 |
|
514 | |||
517 | Return: |
|
515 | Return: | |
518 | 0 in error case else 1 |
|
516 | 0 in error case else 1 | |
519 | """ |
|
517 | """ | |
520 |
|
518 | |||
521 | command = 'mkdir %s' %remotefolder |
|
519 | command = 'mkdir %s' %remotefolder | |
@@ -530,16 +528,16 class SSHClient(Remote): | |||||
530 |
|
528 | |||
531 | def cd(self, remotefolder): |
|
529 | def cd(self, remotefolder): | |
532 | """ |
|
530 | """ | |
533 | cd is used to change remote working directory on server |
|
531 | cd is used to change remote working directory on server | |
534 |
|
532 | |||
535 | Input: |
|
533 | Input: | |
536 | remotefolder - current working directory |
|
534 | remotefolder - current working directory | |
537 |
|
535 | |||
538 | Affects: |
|
536 | Affects: | |
539 | self.remotefolder |
|
537 | self.remotefolder | |
540 |
|
538 | |||
541 | Return: |
|
539 | Return: | |
542 | 0 in case of error else 1 |
|
540 | 0 in case of error else 1 | |
543 | """ |
|
541 | """ | |
544 | if not self.status: |
|
542 | if not self.status: | |
545 | return 0 |
|
543 | return 0 | |
@@ -582,8 +580,8 class SendToServer(ProcessingUnit): | |||||
582 | ProcessingUnit.__init__(self, **kwargs) |
|
580 | ProcessingUnit.__init__(self, **kwargs) | |
583 |
|
581 | |||
584 | self.isConfig = False |
|
582 | self.isConfig = False | |
585 | self.clientObj = None |
|
583 | self.clientObj = None | |
586 |
|
584 | |||
587 | def setup(self, server, username, password, remotefolder, localfolder, ext='.png', period=60, protocol='ftp', **kwargs): |
|
585 | def setup(self, server, username, password, remotefolder, localfolder, ext='.png', period=60, protocol='ftp', **kwargs): | |
588 |
|
586 | |||
589 | self.clientObj = None |
|
587 | self.clientObj = None | |
@@ -638,18 +636,20 class SendToServer(ProcessingUnit): | |||||
638 | return fullfilenameList |
|
636 | return fullfilenameList | |
639 |
|
637 | |||
640 | def run(self, **kwargs): |
|
638 | def run(self, **kwargs): | |
641 |
|
||||
642 | if not self.isConfig: |
|
639 | if not self.isConfig: | |
643 | self.init = time.time() |
|
640 | self.init = time.time() | |
644 | self.setup(**kwargs) |
|
641 | self.setup(**kwargs) | |
645 | self.isConfig = True |
|
642 | self.isConfig = True | |
646 |
|
643 | |||
|
644 | if not self.clientObj.is_alive(): | |||
|
645 | print "[Remote Server]: Restarting connection " | |||
|
646 | self.setup(**kwargs) | |||
|
647 | ||||
647 | if time.time() - self.init >= self.period: |
|
648 | if time.time() - self.init >= self.period: | |
648 | fullfilenameList = self.findFiles() |
|
649 | fullfilenameList = self.findFiles() | |
649 |
|
650 | |||
650 | if self.clientObj.updateFileList(fullfilenameList): |
|
651 | if self.clientObj.updateFileList(fullfilenameList): | |
651 | print "[Remote Server]: Sending the next files ", str(fullfilenameList) |
|
652 | print "[Remote Server]: Sending the next files ", str(fullfilenameList) | |
652 |
|
||||
653 | self.init = time.time() |
|
653 | self.init = time.time() | |
654 |
|
654 | |||
655 | def close(self): |
|
655 | def close(self): | |
@@ -705,9 +705,9 class FTP(object): | |||||
705 | try: |
|
705 | try: | |
706 | self.ftp = ftplib.FTP(self.server) |
|
706 | self.ftp = ftplib.FTP(self.server) | |
707 | self.ftp.login(self.username,self.password) |
|
707 | self.ftp.login(self.username,self.password) | |
708 | self.ftp.cwd(self.remotefolder) |
|
708 | self.ftp.cwd(self.remotefolder) | |
709 | # print 'Connect to FTP Server: Successfully' |
|
709 | # print 'Connect to FTP Server: Successfully' | |
710 |
|
710 | |||
711 | except ftplib.all_errors: |
|
711 | except ftplib.all_errors: | |
712 | print 'Error FTP Service' |
|
712 | print 'Error FTP Service' | |
713 | self.status = 1 |
|
713 | self.status = 1 | |
@@ -738,7 +738,7 class FTP(object): | |||||
738 | name, ext = os.path.splitext(f) |
|
738 | name, ext = os.path.splitext(f) | |
739 | if ext != '': |
|
739 | if ext != '': | |
740 | self.fileList.append(f) |
|
740 | self.fileList.append(f) | |
741 | # print 'filename: %s - size: %d'%(f,self.ftp.size(f)) |
|
741 | # print 'filename: %s - size: %d'%(f,self.ftp.size(f)) | |
742 |
|
742 | |||
743 | def parmsByDefault(self): |
|
743 | def parmsByDefault(self): | |
744 | server = 'jro-app.igp.gob.pe' |
|
744 | server = 'jro-app.igp.gob.pe' |
@@ -1,1 +1,1 | |||||
1 | <Project description="Testing USRP data reader" id="191" name="test01"><ReadUnit datatype="Voltage" id="1911" inputId="0" name="VoltageReader"><Operation id="19111" name="run" priority="1" type="self"><Parameter format="str" id="191111" name="datatype" value="Voltage" /><Parameter format="str" id="191112" name="path" value="/home/nanosat/data/John" /><Parameter format="date" id="191113" name="startDate" value="2000/07/03" /><Parameter format="date" id="191114" name="endDate" value="2017/07/03" /><Parameter format="time" id="191115" name="startTime" value="00:00:00" /><Parameter format="time" id="191116" name="endTime" value="23:59:59" /><Parameter format="int" id="191118" name="blockTime" value="100" /><Parameter format="int" id="191119" name="walk" value="0" /><Parameter format="int" id="191120" name="online" value="0" /></Operation></ReadUnit><ProcUnit datatype="VoltageProc" id="1912" inputId="1911" name="VoltageProc"><Operation id="19121" name="run" priority="1" type="self" /><Operation id="19122" name="Decoder" priority="2" type="external"><Parameter format="intlist" id="191221" name="code" value="1,-1,-1,1,-1" /><Parameter format="int" id="191222" name="nCode" value="5" /><Parameter format="int" id="191223" name="nBaud" value="1" /></Operation></ProcUnit></Project> No newline at end of file |
|
1 | <Project description="Claire" id="002" name="script02"><ReadUnit datatype="VoltageReader" id="21" inputId="0" name="VoltageReader"><Operation id="211" name="run" priority="1" type="self"><Parameter format="str" id="2111" name="datatype" value="VoltageReader" /><Parameter format="str" id="2112" name="path" value="/media/nanosat/0BDE10E00BDE10E0/CLAIRE" /><Parameter format="date" id="2113" name="startDate" value="2017/07/26" /><Parameter format="date" id="2114" name="endDate" value="2017/07/26" /><Parameter format="time" id="2115" name="startTime" value="15:00:00" /><Parameter format="time" id="2116" name="endTime" value="16:00:00" /><Parameter format="int" id="2118" name="delay" value="30" /><Parameter format="int" id="2119" name="walk" value="1" /><Parameter format="int" id="2120" name="online" value="0" /></Operation><Operation id="212" name="printNumberOfBlock" priority="2" type="self" /></ReadUnit><ProcUnit datatype="VoltageProc" id="22" inputId="21" name="VoltageProc"><Operation id="221" name="run" priority="1" type="self" /></ProcUnit><ProcUnit datatype="SpectraProc" id="23" inputId="22" name="SpectraProc"><Operation id="231" name="run" priority="1" type="self"><Parameter format="int" id="2311" name="nFFTPoints" value="128" /><Parameter format="int" id="2312" name="nProfiles" value="128" /><Parameter format="pairslist" id="2313" name="pairsList" value="(0,1),(0,2),(1,2)" /></Operation><Operation id="232" name="setRadarFrequency" priority="2" type="self"><Parameter format="float" id="2321" name="frequency" value="445000000.0" /></Operation><Operation id="233" name="IncohInt" priority="3" type="other"><Parameter format="float" id="2331" name="timeInterval" value="128" /></Operation><Operation id="234" name="removeDC" priority="4" type="self"><Parameter format="int" id="2341" name="mode" value="2" /></Operation><Operation id="235" name="SpectraPlot" priority="5" type="external"><Parameter format="int" id="2351" name="id" value="235" /><Parameter format="str" id="2352" name="wintitle" value="SpectraPlot" /><Parameter format="str" id="2353" name="xaxis" value="frequency" /><Parameter format="float" id="2354" name="zmin" value="15.0" /><Parameter format="float" id="2355" name="zmax" value="50.0" /><Parameter format="int" id="2356" name="exp_code" value="21" /><Parameter format="int" id="2357" name="ftp_wei" value="0" /><Parameter format="int" id="2358" name="plot_pos" value="0" /></Operation></ProcUnit></Project> No newline at end of file |
@@ -5,54 +5,57 Created on Jul 16, 2014 | |||||
5 | @author: Miguel Urco |
|
5 | @author: Miguel Urco | |
6 | """ |
|
6 | """ | |
7 |
|
7 | |||
8 | from schainpy import __version__ |
|
|||
9 | from setuptools import setup, Extension |
|
8 | from setuptools import setup, Extension | |
|
9 | from setuptools.command.build_ext import build_ext as _build_ext | |||
|
10 | from schainpy import __version__ | |||
|
11 | ||||
|
12 | class build_ext(_build_ext): | |||
|
13 | def finalize_options(self): | |||
|
14 | _build_ext.finalize_options(self) | |||
|
15 | # Prevent numpy from thinking it is still in its setup process: | |||
|
16 | __builtins__.__NUMPY_SETUP__ = False | |||
|
17 | import numpy | |||
|
18 | self.include_dirs.append(numpy.get_include()) | |||
|
19 | ||||
10 |
|
20 | |||
11 | setup(name="schainpy", |
|
21 | setup(name="schainpy", | |
12 | version=__version__, |
|
22 | version=__version__, | |
13 | description="Python tools to read, write and process Jicamarca data", |
|
23 | description="Python tools to read, write and process Jicamarca data", | |
14 | author="Miguel Urco", |
|
24 | author="Miguel Urco", | |
15 | author_email="miguel.urco@jro.igp.gob.pe", |
|
25 | author_email="miguel.urco@jro.igp.gob.pe", | |
16 | url="http://jro.igp.gob.pe", |
|
26 | url="http://jro.igp.gob.pe", | |
17 | packages={'schainpy', |
|
27 | packages = {'schainpy', | |
18 | 'schainpy.model', |
|
28 | 'schainpy.model', | |
19 | 'schainpy.model.data', |
|
29 | 'schainpy.model.data', | |
20 | 'schainpy.model.graphics', |
|
30 | 'schainpy.model.graphics', | |
21 | 'schainpy.model.io', |
|
31 | 'schainpy.model.io', | |
22 | 'schainpy.model.proc', |
|
32 | 'schainpy.model.proc', | |
23 | 'schainpy.model.serializer', |
|
33 | 'schainpy.model.serializer', | |
24 | 'schainpy.model.utils', |
|
34 | 'schainpy.model.utils', | |
25 | 'schainpy.gui', |
|
35 | 'schainpy.gui', | |
26 | 'schainpy.gui.figures', |
|
36 | 'schainpy.gui.figures', | |
27 | 'schainpy.gui.viewcontroller', |
|
37 | 'schainpy.gui.viewcontroller', | |
28 | 'schainpy.gui.viewer', |
|
38 | 'schainpy.gui.viewer', | |
29 | 'schainpy.gui.viewer.windows'}, |
|
39 | 'schainpy.gui.viewer.windows'}, | |
30 | ext_package='schainpy', |
|
40 | ext_package='schainpy', | |
31 | py_modules=[''], |
|
41 | py_modules=[''], | |
32 | package_data={'': ['schain.conf.template'], |
|
42 | package_data={'': ['schain.conf.template'], | |
33 |
'schainpy.gui.figures': ['*.png', |
|
43 | 'schainpy.gui.figures': ['*.png','*.jpg'], | |
34 | }, |
|
44 | }, | |
35 | include_package_data=False, |
|
45 | include_package_data=False, | |
36 | entry_points={ |
|
46 | scripts =['schainpy/gui/schainGUI', | |
37 | 'console_scripts': [ |
|
47 | 'schainpy/scripts/schain'], | |
38 | 'schain = schaincli.cli:main', |
|
48 | ext_modules=[ | |
39 | ], |
|
49 | Extension("cSchain", ["schainpy/model/proc/extensions.c"] | |
40 |
|
|
50 | )], | |
41 | scripts=['schainpy/gui/schainGUI'], |
|
51 | cmdclass={'build_ext':build_ext}, | |
42 | ext_modules=[Extension("cSchain", ["schainpy/model/proc/extensions.c"], include_dirs=[numpy.get_include()])], |
|
52 | setup_requires=["numpy >= 1.11.2"], | |
43 | install_requires=[ |
|
53 | install_requires=[ | |
44 | "scipy >= 0.14.0", |
|
54 | "scipy >= 0.14.0", | |
45 | "h5py >= 2.2.1", |
|
55 | "h5py >= 2.2.1", | |
46 | "matplotlib >= 1.4.2", |
|
56 | "matplotlib >= 1.4.2", | |
47 | "pyfits >= 3.4", |
|
57 | "pyfits >= 3.4", | |
48 |
" |
|
58 | "paho-mqtt >= 1.2", | |
49 |
" |
|
59 | "zmq", | |
50 |
|
|
60 | ], | |
51 | "zmq", |
|
|||
52 | "fuzzywuzzy", |
|
|||
53 | "click", |
|
|||
54 | "colorama", |
|
|||
55 | "python-Levenshtein" |
|
|||
56 | ], |
|
|||
57 | ) |
|
61 | ) | |
58 |
|
1 | NO CONTENT: file was removed |
|
NO CONTENT: file was removed |
1 | NO CONTENT: file was removed |
|
NO CONTENT: file was removed |
1 | NO CONTENT: file was removed |
|
NO CONTENT: file was removed |
1 | NO CONTENT: file was removed |
|
NO CONTENT: file was removed |
1 | NO CONTENT: file was removed |
|
NO CONTENT: file was removed |
1 | NO CONTENT: file was removed |
|
NO CONTENT: file was removed |
1 | NO CONTENT: file was removed |
|
NO CONTENT: file was removed |
1 | NO CONTENT: file was removed |
|
NO CONTENT: file was removed |
1 | NO CONTENT: file was removed |
|
NO CONTENT: file was removed |
1 | NO CONTENT: file was removed |
|
NO CONTENT: file was removed |
1 | NO CONTENT: file was removed |
|
NO CONTENT: file was removed |
1 | NO CONTENT: file was removed |
|
NO CONTENT: file was removed |
1 | NO CONTENT: file was removed |
|
NO CONTENT: file was removed |
1 | NO CONTENT: file was removed |
|
NO CONTENT: file was removed |
1 | NO CONTENT: file was removed |
|
NO CONTENT: file was removed |
1 | NO CONTENT: file was removed |
|
NO CONTENT: file was removed |
1 | NO CONTENT: file was removed |
|
NO CONTENT: file was removed |
1 | NO CONTENT: file was removed |
|
NO CONTENT: file was removed |
1 | NO CONTENT: file was removed |
|
NO CONTENT: file was removed |
1 | NO CONTENT: file was removed |
|
NO CONTENT: file was removed |
1 | NO CONTENT: file was removed |
|
NO CONTENT: file was removed |
1 | NO CONTENT: file was removed |
|
NO CONTENT: file was removed |
1 | NO CONTENT: file was removed |
|
NO CONTENT: file was removed |
1 | NO CONTENT: file was removed |
|
NO CONTENT: file was removed |
1 | NO CONTENT: file was removed |
|
NO CONTENT: file was removed |
1 | NO CONTENT: file was removed |
|
NO CONTENT: file was removed |
1 | NO CONTENT: file was removed |
|
NO CONTENT: file was removed |
1 | NO CONTENT: file was removed |
|
NO CONTENT: file was removed |
1 | NO CONTENT: file was removed |
|
NO CONTENT: file was removed |
1 | NO CONTENT: file was removed |
|
NO CONTENT: file was removed |
1 | NO CONTENT: file was removed |
|
NO CONTENT: file was removed |
1 | NO CONTENT: file was removed |
|
NO CONTENT: file was removed |
1 | NO CONTENT: file was removed |
|
NO CONTENT: file was removed |
1 | NO CONTENT: file was removed |
|
NO CONTENT: file was removed |
1 | NO CONTENT: file was removed |
|
NO CONTENT: file was removed |
1 | NO CONTENT: file was removed |
|
NO CONTENT: file was removed |
1 | NO CONTENT: file was removed |
|
NO CONTENT: file was removed |
1 | NO CONTENT: file was removed |
|
NO CONTENT: file was removed |
1 | NO CONTENT: file was removed |
|
NO CONTENT: file was removed |
1 | NO CONTENT: file was removed |
|
NO CONTENT: file was removed |
1 | NO CONTENT: file was removed |
|
NO CONTENT: file was removed |
1 | NO CONTENT: file was removed |
|
NO CONTENT: file was removed |
1 | NO CONTENT: file was removed |
|
NO CONTENT: file was removed |
1 | NO CONTENT: file was removed |
|
NO CONTENT: file was removed |
1 | NO CONTENT: file was removed |
|
NO CONTENT: file was removed |
1 | NO CONTENT: file was removed |
|
NO CONTENT: file was removed |
1 | NO CONTENT: file was removed |
|
NO CONTENT: file was removed |
1 | NO CONTENT: file was removed |
|
NO CONTENT: file was removed |
1 | NO CONTENT: file was removed |
|
NO CONTENT: file was removed |
1 | NO CONTENT: file was removed |
|
NO CONTENT: file was removed |
1 | NO CONTENT: file was removed |
|
NO CONTENT: file was removed |
1 | NO CONTENT: file was removed |
|
NO CONTENT: file was removed |
1 | NO CONTENT: file was removed |
|
NO CONTENT: file was removed |
1 | NO CONTENT: file was removed |
|
NO CONTENT: file was removed |
1 | NO CONTENT: file was removed |
|
NO CONTENT: file was removed |
1 | NO CONTENT: file was removed |
|
NO CONTENT: file was removed |
1 | NO CONTENT: file was removed |
|
NO CONTENT: file was removed |
General Comments 0
You need to be logged in to leave comments.
Login now