schain Commit - r1334:96199ee477a0 · Jicamarca Repository

Fix xmin & xmax in plots, fix SendToFTP

jespinoza -

r1334:96199ee477a0

parent child

Context file:

r1334:96199ee477a0

README.md +1 -1

             # Signal Chain
             Signal Chain is a radar data processing library wich includes modules to read,
             and write different files formats, besides modules to process and visualize the
             data.
             ## Dependencies
             - GCC (gcc or gfortran)
             - Python.h (python-dev or python-devel)
             - Python-TK (python-tk)
             - HDF5 libraries (libhdf5-dev)
             ## Installation
             To get started the easiest way to install it is through [PyPI](https://pypi.org/project/schainpy/) with pip:
             ```bash
             pip install schainpy
             ```
             ### From source
             First, ensure that you have the above-listed dependencies installed, then clone
             the repository and install as normal python package:
             ```bash
             git clone https://github.com/JRO-Peru/schain.git
             cd schain
             git checkout `branch-name` (optional)
             sudo pip install ./
             ```
             ### Using Docker
             Download Dockerfile from the repository, and create a docker image:
             ```bash
             docker build -t schain .
             ```
             You can run a container using an xml file or a schain script also you need to
             mount a volume for the data input and for the output files/plots:
             ```bash
             docker run -it --rm --volume /path/to/host/data:/data schain xml /data/test.xml
             docker run -it --rm --volume /path/to/host/data:/data --entrypoint /urs/local/bin/python schain /data/test.py
             ```
             ## CLI (command line interface)
             Signal Chain provides the following commands:
             - schainGUI: Open the GUI
             - schain: Signal chain command line
             ## Example
             Here you can find an script to read Spectra data (.pdata), remove dc and plot
             self-spectra & RTI:
             ```python
             #!/usr/bin/python
             from schainpy.controller import Project
             prj = Project()
             read_unit = prj.addReadUnit(
                 datatype='Spectra',
                 path='/path/to/pdata/',
                 startDate='2014/01/31',
                 endDate='2014/03/31',
                 startTime='00:00:00',
                 endTime='23:59:59',
                 online=0,
                 walk=0
                 )
             proc_unit = prj.addProcUnit(
                 datatype='Spectra',
                 inputId=read_unit.getId()
                 )
             op = proc_unit.addOperation(name='selectChannels')
             op.addParameter(name='channelList', value='0,1')
             op = proc_unit.addOperation(name='selectHeights')
             op.addParameter(name='minHei', value='80')
             op.addParameter(name='maxHei', value='200')
             op = proc_unit.addOperation(name='removeDC')
             op = proc_unit.addOperation(name='SpectraPlot')
             op.addParameter(name='wintitle', value='Spectra', format='str')
-            op = procUnitConfObj1.addOperation(name='RTIPlot')
+            op = proc_unit.addOperation(name='RTIPlot')
             op.addParameter(name='wintitle', value='RTI', format='str')
             prj.start()
             ```

schainpy/model/graphics/jroplot_base.py +64 -115

+            # Copyright (c) 2012-2020 Jicamarca Radio Observatory
+            # All rights reserved.
+            #
+            # Distributed under the terms of the BSD 3-clause license.
+            """Base class to create plot operations
+            """
             import os
             import sys
             import zmq
             import time
             import numpy
             import datetime
-            try:
+            from multiprocessing import Queue
-                from queue import Queue
-            except:
-                from Queue import Queue
             from functools import wraps
             from threading import Thread
             import matplotlib
             if 'BACKEND' in os.environ:
                 matplotlib.use(os.environ['BACKEND'])
             elif 'linux' in sys.platform:
                 matplotlib.use("TkAgg")
             elif 'darwin' in sys.platform:
                 matplotlib.use('WxAgg')
             else:
                 from schainpy.utils import log
                 log.warning('Using default Backend="Agg"', 'INFO')
                 matplotlib.use('Agg')
             import matplotlib.pyplot as plt
             from matplotlib.patches import Polygon
             from mpl_toolkits.axes_grid1 import make_axes_locatable
             from matplotlib.ticker import FuncFormatter, LinearLocator, MultipleLocator
             from schainpy.model.data.jrodata import PlotterData
             from schainpy.model.proc.jroproc_base import ProcessingUnit, Operation, MPDecorator
             from schainpy.utils import log
             jet_values = matplotlib.pyplot.get_cmap('jet', 100)(numpy.arange(100))[10:90]
             blu_values = matplotlib.pyplot.get_cmap(
                 'seismic_r', 20)(numpy.arange(20))[10:15]
             ncmap = matplotlib.colors.LinearSegmentedColormap.from_list(
                 'jro', numpy.vstack((blu_values, jet_values)))
             matplotlib.pyplot.register_cmap(cmap=ncmap)
             CMAPS = [plt.get_cmap(s) for s in ('jro', 'jet', 'viridis',
                                                'plasma', 'inferno', 'Greys', 'seismic', 'bwr', 'coolwarm')]
             EARTH_RADIUS = 6.3710e3
             def ll2xy(lat1, lon1, lat2, lon2):
                 p = 0.017453292519943295
                 a = 0.5 - numpy.cos((lat2 - lat1) * p)/2 + numpy.cos(lat1 * p) * \
                     numpy.cos(lat2 * p) * (1 - numpy.cos((lon2 - lon1) * p)) / 2
                 r = 12742 * numpy.arcsin(numpy.sqrt(a))
                 theta = numpy.arctan2(numpy.sin((lon2-lon1)*p)*numpy.cos(lat2*p), numpy.cos(lat1*p)
                                       * numpy.sin(lat2*p)-numpy.sin(lat1*p)*numpy.cos(lat2*p)*numpy.cos((lon2-lon1)*p))
                 theta = -theta + numpy.pi/2
                 return r*numpy.cos(theta), r*numpy.sin(theta)
             def km2deg(km):
                 '''
                 Convert distance in km to degrees
                 '''
                 return numpy.rad2deg(km/EARTH_RADIUS)
             def figpause(interval):
                 backend = plt.rcParams['backend']
                 if backend in matplotlib.rcsetup.interactive_bk:
                     figManager = matplotlib._pylab_helpers.Gcf.get_active()
                     if figManager is not None:
                         canvas = figManager.canvas
                         if canvas.figure.stale:
                             canvas.draw()
                         try:
                             canvas.start_event_loop(interval)
                         except:
                             pass
                         return
             def popup(message):
                 '''
                 '''
                 fig = plt.figure(figsize=(12, 8), facecolor='r')
                 text = '\n'.join([s.strip() for s in message.split(':')])
                 fig.text(0.01, 0.5, text, ha='left', va='center',
                          size='20', weight='heavy', color='w')
                 fig.show()
                 figpause(1000)
             class Throttle(object):
                 '''
                 Decorator that prevents a function from being called more than once every
                 time period.
                 To create a function that cannot be called more than once a minute, but
                 will sleep until it can be called:
                 @Throttle(minutes=1)
                 def foo():
                   pass
                 for i in range(10):
                   foo()
                   print "This function has run %s times." % i
                 '''
                 def __init__(self, seconds=0, minutes=0, hours=0):
                     self.throttle_period = datetime.timedelta(
                         seconds=seconds, minutes=minutes, hours=hours
                     )
                     self.time_of_last_call = datetime.datetime.min
                 def __call__(self, fn):
                     @wraps(fn)
                     def wrapper(*args, **kwargs):
                         coerce = kwargs.pop('coerce', None)
                         if coerce:
                             self.time_of_last_call = datetime.datetime.now()
                             return fn(*args, **kwargs)
                         else:
                             now = datetime.datetime.now()
                             time_since_last_call = now - self.time_of_last_call
                             time_left = self.throttle_period - time_since_last_call
                             if time_left > datetime.timedelta(seconds=0):
                                 return
                         self.time_of_last_call = datetime.datetime.now()
                         return fn(*args, **kwargs)
                     return wrapper
             def apply_throttle(value):
                 @Throttle(seconds=value)
                 def fnThrottled(fn):
                     fn()
                 return fnThrottled
             @MPDecorator
             class Plot(Operation):
-                '''
+                """Base class for Schain plotting operations
-                Base class for Schain plotting operations
-                '''
+                This class should never be use directtly you must subclass a new operation,
+                children classes must be defined as follow:
+                ExamplePlot(Plot):
+                    CODE = 'code'
+                    colormap = 'jet'
+                    plot_type = 'pcolor' # options are ('pcolor', 'pcolorbuffer', 'scatter', 'scatterbuffer')
+                    def setup(self):
+                        pass
+                    def plot(self):
+                        pass
+                """
                 CODE = 'Figure'
                 colormap = 'jet'
                 bgcolor = 'white'
                 buffering = True
                 __missing = 1E30
                 __attrs__ = ['show', 'save', 'ymin', 'ymax', 'zmin', 'zmax', 'title',
                              'showprofile']
                 def __init__(self):
                     Operation.__init__(self)
                     self.isConfig = False
                     self.isPlotConfig = False
-                    self.save_counter = 1
+                    self.save_time = 0
                     self.sender_time = 0
                     self.data = None
                     self.firsttime = True
                     self.sender_queue = Queue(maxsize=60)
                     self.plots_adjust = {'left': 0.125, 'right': 0.9, 'bottom': 0.15, 'top': 0.9, 'wspace': 0.2, 'hspace': 0.2}
                 def __fmtTime(self, x, pos):
                     '''
                     '''
                     return '{}'.format(self.getDateTime(x).strftime('%H:%M'))
                 def __setup(self, **kwargs):
                     '''
                     Initialize variables
                     '''
                     self.figures = []
                     self.axes = []
                     self.cb_axes = []
                     self.localtime = kwargs.pop('localtime', True)
                     self.show = kwargs.get('show', True)
                     self.save = kwargs.get('save', False)
-                    self.save_period = kwargs.get('save_period', 1)
+                    self.save_period = kwargs.get('save_period', 60)
                     self.colormap = kwargs.get('colormap', self.colormap)
                     self.colormap_coh = kwargs.get('colormap_coh', 'jet')
                     self.colormap_phase = kwargs.get('colormap_phase', 'RdBu_r')
                     self.colormaps = kwargs.get('colormaps', None)
                     self.bgcolor = kwargs.get('bgcolor', self.bgcolor)
                     self.showprofile = kwargs.get('showprofile', False)
                     self.title = kwargs.get('wintitle', self.CODE.upper())
                     self.cb_label = kwargs.get('cb_label', None)
                     self.cb_labels = kwargs.get('cb_labels', None)
                     self.labels = kwargs.get('labels', None)
                     self.xaxis = kwargs.get('xaxis', 'frequency')
                     self.zmin = kwargs.get('zmin', None)
                     self.zmax = kwargs.get('zmax', None)
                     self.zlimits = kwargs.get('zlimits', None)
                     self.xmin = kwargs.get('xmin', None)
                     self.xmax = kwargs.get('xmax', None)
                     self.xrange = kwargs.get('xrange', 12)
                     self.xscale = kwargs.get('xscale', None)
                     self.ymin = kwargs.get('ymin', None)
                     self.ymax = kwargs.get('ymax', None)
                     self.yscale = kwargs.get('yscale', None)
                     self.xlabel = kwargs.get('xlabel', None)
                     self.attr_time = kwargs.get('attr_time', 'utctime')
                     self.decimation = kwargs.get('decimation', None)
                     self.showSNR = kwargs.get('showSNR', False)
                     self.oneFigure = kwargs.get('oneFigure', True)
                     self.width = kwargs.get('width', None)
                     self.height = kwargs.get('height', None)
                     self.colorbar = kwargs.get('colorbar', True)
                     self.factors = kwargs.get('factors', [1, 1, 1, 1, 1, 1, 1, 1])
                     self.channels = kwargs.get('channels', None)
                     self.titles = kwargs.get('titles', [])
                     self.polar = False
                     self.type = kwargs.get('type', 'iq')
                     self.grid = kwargs.get('grid', False)
                     self.pause = kwargs.get('pause', False)
-                    self.save_code = kwargs.get('save_code', None)
+                    self.save_code = kwargs.get('save_code', self.CODE)
                     self.throttle = kwargs.get('throttle', 0)
                     self.exp_code = kwargs.get('exp_code', None)
                     self.plot_server = kwargs.get('plot_server', False)
                     self.sender_period = kwargs.get('sender_period', 60)
                     self.tag = kwargs.get('tag', '')
                     self.height_index = kwargs.get('height_index', None)
                     self.__throttle_plot = apply_throttle(self.throttle)
                     self.data = PlotterData(
                         self.CODE, self.throttle, self.exp_code, self.localtime, self.buffering, snr=self.showSNR)
                     if self.plot_server:
                         if not self.plot_server.startswith('tcp://'):
                             self.plot_server = 'tcp://{}'.format(self.plot_server)
                         log.success(
                             'Sending to server: {}'.format(self.plot_server),
                             self.name
                         )
-                    if 'plot_name' in kwargs:
-                        self.plot_name = kwargs['plot_name']
                 def __setup_plot(self):
                     '''
                     Common setup for all figures, here figures and axes are created
                     '''
                     self.setup()
                     self.time_label = 'LT' if self.localtime else 'UTC'
                     if self.width is None:
                         self.width = 8
                     self.figures = []
                     self.axes = []
                     self.cb_axes = []
                     self.pf_axes = []
                     self.cmaps = []
                     size = '15%' if self.ncols == 1 else '30%'
                     pad = '4%' if self.ncols == 1 else '8%'
                     if self.oneFigure:
                         if self.height is None:
                             self.height = 1.4 * self.nrows + 1
                         fig = plt.figure(figsize=(self.width, self.height),
                                          edgecolor='k',
                                          facecolor='w')
                         self.figures.append(fig)
                         for n in range(self.nplots):
                             ax = fig.add_subplot(self.nrows, self.ncols,
                                                  n + 1, polar=self.polar)
                             ax.tick_params(labelsize=8)
                             ax.firsttime = True
                             ax.index = 0
                             ax.press = None
                             self.axes.append(ax)
                             if self.showprofile:
                                 cax = self.__add_axes(ax, size=size, pad=pad)
                                 cax.tick_params(labelsize=8)
                                 self.pf_axes.append(cax)
                     else:
                         if self.height is None:
                             self.height = 3
                         for n in range(self.nplots):
                             fig = plt.figure(figsize=(self.width, self.height),
                                              edgecolor='k',
                                              facecolor='w')
                             ax = fig.add_subplot(1, 1, 1, polar=self.polar)
                             ax.tick_params(labelsize=8)
                             ax.firsttime = True
                             ax.index = 0
                             ax.press = None
                             self.figures.append(fig)
                             self.axes.append(ax)
                             if self.showprofile:
                                 cax = self.__add_axes(ax, size=size, pad=pad)
                                 cax.tick_params(labelsize=8)
                                 self.pf_axes.append(cax)
                     for n in range(self.nrows):
                         if self.colormaps is not None:
                             cmap = plt.get_cmap(self.colormaps[n])
                         else:
                             cmap = plt.get_cmap(self.colormap)
                         cmap.set_bad(self.bgcolor, 1.)
                         self.cmaps.append(cmap)
                 def __add_axes(self, ax, size='30%', pad='8%'):
                     '''
                     Add new axes to the given figure
                     '''
                     divider = make_axes_locatable(ax)
                     nax = divider.new_horizontal(size=size, pad=pad)
                     ax.figure.add_axes(nax)
                     return nax
                 def fill_gaps(self, x_buffer, y_buffer, z_buffer):
                     '''
                     Create a masked array for missing data
                     '''
                     if x_buffer.shape[0] < 2:
                         return x_buffer, y_buffer, z_buffer
                     deltas = x_buffer[1:] - x_buffer[0:-1]
                     x_median = numpy.median(deltas)
                     index = numpy.where(deltas > 5 * x_median)
                     if len(index[0]) != 0:
                         z_buffer[::, index[0], ::] = self.__missing
                         z_buffer = numpy.ma.masked_inside(z_buffer,
 .99 * self.__missing,
 .01 * self.__missing)
                     return x_buffer, y_buffer, z_buffer
                 def decimate(self):
                     # dx = int(len(self.x)/self.__MAXNUMX) + 1
                     dy = int(len(self.y) / self.decimation) + 1
                     # x = self.x[::dx]
                     x = self.x
                     y = self.y[::dy]
                     z = self.z[::, ::, ::dy]
                     return x, y, z
                 def format(self):
                     '''
                     Set min and max values, labels, ticks and titles
                     '''
-                    if self.xmin is None:
+                    for n, ax in enumerate(self.axes):
-                        xmin = self.data.min_time
+                        if ax.firsttime:
-                    else:
+                            if self.xaxis != 'time':
-                        if self.xaxis is 'time':
-                            dt = self.getDateTime(self.data.min_time)
-                            xmin = (dt.replace(hour=int(self.xmin), minute=0, second=0) -
-                                    datetime.datetime(1970, 1, 1)).total_seconds()
-                            if self.data.localtime:
-                                xmin += time.timezone
-                        else:
                                 xmin = self.xmin
-                    if self.xmax is None:
-                        xmax = xmin + self.xrange * 60 * 60
-                    else:
-                        if self.xaxis is 'time':
-                            dt = self.getDateTime(self.data.max_time)
-                            xmax = self.xmax - 1
-                            xmax = (dt.replace(hour=int(xmax), minute=59, second=59) -
-                                    datetime.datetime(1970, 1, 1) + datetime.timedelta(seconds=1)).total_seconds()
-                            if self.data.localtime:
-                                xmax += time.timezone
-                        else:
                                 xmax = self.xmax
+                            else:
+                                xmin = self.tmin
+                                xmax = self.tmin + self.xrange*60*60
+                                ax.xaxis.set_major_formatter(FuncFormatter(self.__fmtTime))
+                                ax.xaxis.set_major_locator(LinearLocator(9))
                             ymin = self.ymin if self.ymin else numpy.nanmin(self.y)
                             ymax = self.ymax if self.ymax else numpy.nanmax(self.y)
-                    for n, ax in enumerate(self.axes):
-                        if ax.firsttime:
-                            dig = int(numpy.log10(ymax))
-                            if dig == 0:
-                                digD = len(str(ymax)) - 2
-                                ydec = ymax*(10**digD)
-                                dig = int(numpy.log10(ydec))
-                                ystep = ((ydec + (10**(dig)))//10**(dig))*(10**(dig))
-                                ystep = ystep/5
-                                ystep = ystep/(10**digD)
-                            else:
-                                ystep = ((ymax + (10**(dig)))//10**(dig))*(10**(dig))
-                                ystep = ystep/5
-                            if self.xaxis is not 'time':
-                                dig = int(numpy.log10(xmax))
-                                if dig <= 0:
-                                    digD = len(str(xmax)) - 2
-                                    xdec = xmax*(10**digD)
-                                    dig = int(numpy.log10(xdec))
-                                    xstep = ((xdec + (10**(dig)))//10**(dig))*(10**(dig))
-                                    xstep = xstep*0.5
-                                    xstep = xstep/(10**digD)
-                                else:
-                                    xstep = ((xmax + (10**(dig)))//10**(dig))*(10**(dig))
-                                    xstep = xstep/5
                             ax.set_facecolor(self.bgcolor)
-                            ax.yaxis.set_major_locator(MultipleLocator(ystep))
                             if self.xscale:
                                 ax.xaxis.set_major_formatter(FuncFormatter(
                                     lambda x, pos: '{0:g}'.format(x*self.xscale)))
-                            if self.xscale:
+                            if self.yscale:
                                 ax.yaxis.set_major_formatter(FuncFormatter(
                                     lambda x, pos: '{0:g}'.format(x*self.yscale)))
-                            if self.xaxis is 'time':
-                                ax.xaxis.set_major_formatter(FuncFormatter(self.__fmtTime))
-                                ax.xaxis.set_major_locator(LinearLocator(9))
-                            else:
-                                ax.xaxis.set_major_locator(MultipleLocator(xstep))
                             if self.xlabel is not None:
                                 ax.set_xlabel(self.xlabel)
+                            if self.ylabel is not None:
                                 ax.set_ylabel(self.ylabel)
-                            ax.firsttime = False
                             if self.showprofile:
                                 self.pf_axes[n].set_ylim(ymin, ymax)
                                 self.pf_axes[n].set_xlim(self.zmin, self.zmax)
                                 self.pf_axes[n].set_xlabel('dB')
                                 self.pf_axes[n].grid(b=True, axis='x')
                                 [tick.set_visible(False)
                                  for tick in self.pf_axes[n].get_yticklabels()]
                             if self.colorbar:
                                 ax.cbar = plt.colorbar(
                                     ax.plt, ax=ax, fraction=0.05, pad=0.02, aspect=10)
                                 ax.cbar.ax.tick_params(labelsize=8)
                                 ax.cbar.ax.press = None
                                 if self.cb_label:
                                     ax.cbar.set_label(self.cb_label, size=8)
                                 elif self.cb_labels:
                                     ax.cbar.set_label(self.cb_labels[n], size=8)
                             else:
                                 ax.cbar = None
+                            ax.set_xlim(xmin, xmax)
+                            ax.set_ylim(ymin, ymax)
+                            ax.firsttime = False
                             if self.grid:
                                 ax.grid(True)
                         if not self.polar:
-                            ax.set_xlim(xmin, xmax)
-                            ax.set_ylim(ymin, ymax)
                             ax.set_title('{} {} {}'.format(
                                 self.titles[n],
                                 self.getDateTime(self.data.max_time).strftime(
                                     '%Y-%m-%d %H:%M:%S'),
                                 self.time_label),
                                 size=8)
                         else:
                             ax.set_title('{}'.format(self.titles[n]), size=8)
                             ax.set_ylim(0, 90)
                             ax.set_yticks(numpy.arange(0, 90, 20))
                             ax.yaxis.labelpad = 40
                     if self.firsttime:
                         for n, fig in enumerate(self.figures):
                             fig.subplots_adjust(**self.plots_adjust)
                         self.firsttime = False
                 def clear_figures(self):
                     '''
                     Reset axes for redraw plots
                     '''
                     for ax in self.axes+self.pf_axes+self.cb_axes:
                         ax.clear()
                         ax.firsttime = True
                         if hasattr(ax, 'cbar') and ax.cbar:
                             ax.cbar.remove()
                 def __plot(self):
                     '''
                     Main function to plot, format and save figures
                     '''
                     self.plot()
                     self.format()
                     for n, fig in enumerate(self.figures):
                         if self.nrows == 0 or self.nplots == 0:
                             log.warning('No data', self.name)
                             fig.text(0.5, 0.5, 'No Data', fontsize='large', ha='center')
                             fig.canvas.manager.set_window_title(self.CODE)
                             continue
                         fig.canvas.manager.set_window_title('{} - {}'.format(self.title,
                                                                              self.getDateTime(self.data.max_time).strftime('%Y/%m/%d')))
                         fig.canvas.draw()
                         if self.show:
                             fig.show()
                             figpause(0.01)
                         if self.save:
                             self.save_figure(n)
                     if self.plot_server:
                         self.send_to_server()
                 def save_figure(self, n):
                     '''
                     '''
-                    if self.save_counter < self.save_period:
+                    if (self.data.tm - self.save_time) < self.sender_period:
-                        self.save_counter += 1
                         return
-                    self.save_counter = 1
+                    self.save_time = self.data.tm
                     fig = self.figures[n]
-                    if self.save_code:
-                        if isinstance(self.save_code, str):
-                            labels = [self.save_code for x in self.figures]
-                        else:
-                            labels = self.save_code
-                    else:
-                        labels = [self.CODE for x in self.figures]
                     figname = os.path.join(
                         self.save,
-                        labels[n],
+                        self.save_code,
                         '{}_{}.png'.format(
-                            labels[n],
+                            self.save_code,
                             self.getDateTime(self.data.max_time).strftime(
                                 '%Y%m%d_%H%M%S'
                                 ),
                             )
                         )
                     log.log('Saving figure: {}'.format(figname), self.name)
                     if not os.path.isdir(os.path.dirname(figname)):
                         os.makedirs(os.path.dirname(figname))
                     fig.savefig(figname)
                     if self.throttle == 0:
                         figname = os.path.join(
                             self.save,
                             '{}_{}.png'.format(
-                                labels[n],
+                                self.save_code,
                                 self.getDateTime(self.data.min_time).strftime(
                                     '%Y%m%d'
                                     ),
                                 )
                             )
                         fig.savefig(figname)
                 def send_to_server(self):
                     '''
                     '''
                     interval = self.data.tm - self.sender_time
                     if interval < self.sender_period:
                         return
                     self.sender_time = self.data.tm
                     attrs = ['titles', 'zmin', 'zmax', 'tag', 'ymin', 'ymax']
                     for attr in attrs:
                         value = getattr(self, attr)
                         if value:
                             if isinstance(value, (numpy.float32, numpy.float64)):
                                 value = round(float(value), 2)
                             self.data.meta[attr] = value
                     if self.colormap == 'jet':
                         self.data.meta['colormap'] = 'Jet'
                     elif 'RdBu' in self.colormap:
                         self.data.meta['colormap'] = 'RdBu'
                     else:
                         self.data.meta['colormap'] = 'Viridis'
                     self.data.meta['interval'] = int(interval)
-                    # msg = self.data.jsonify(self.data.tm, self.plot_name, self.plot_type)
                     try:
                         self.sender_queue.put(self.data.tm, block=False)
                     except:
                         tm = self.sender_queue.get()
                         self.sender_queue.put(self.data.tm)
                     while True:
                         if self.sender_queue.empty():
                             break
                         tm = self.sender_queue.get()
                         try:
-                            msg = self.data.jsonify(tm, self.plot_name, self.plot_type)
+                            msg = self.data.jsonify(tm, self.save_code, self.plot_type)
                         except:
                             continue
                         self.socket.send_string(msg)
                         socks = dict(self.poll.poll(5000))
                         if socks.get(self.socket) == zmq.POLLIN:
                             reply = self.socket.recv_string()
                             if reply == 'ok':
                                 log.log("Response from server ok", self.name)
                                 time.sleep(0.2)
                                 continue
                             else:
                                 log.warning(
                                     "Malformed reply from server: {}".format(reply), self.name)
                         else:
                             log.warning(
                                 "No response from server, retrying...", self.name)
                             self.sender_queue.put(self.data.tm)
                         self.socket.setsockopt(zmq.LINGER, 0)
                         self.socket.close()
                         self.poll.unregister(self.socket)
                         time.sleep(0.1)
                         self.socket = self.context.socket(zmq.REQ)
                         self.socket.connect(self.plot_server)
                         self.poll.register(self.socket, zmq.POLLIN)
                         break
                 def setup(self):
                     '''
                     This method should be implemented in the child class, the following
                     attributes should be set:
                     self.nrows: number of rows
                     self.ncols: number of cols
                     self.nplots: number of plots (channels or pairs)
                     self.ylabel: label for Y axes
                     self.titles: list of axes title
                     '''
                     raise NotImplementedError
                 def plot(self):
                     '''
                     Must be defined in the child class
                     '''
                     raise NotImplementedError
                 def run(self, dataOut, **kwargs):
                     '''
                     Main plotting routine
                     '''
                     if self.isConfig is False:
                         self.__setup(**kwargs)
-                        t = getattr(dataOut, self.attr_time)
                         if self.localtime:
                             self.getDateTime = datetime.datetime.fromtimestamp
                         else:
                             self.getDateTime = datetime.datetime.utcfromtimestamp
-                        if self.xmin is None:
-                            self.tmin = t
-                            if 'buffer' in self.plot_type:
-                                self.xmin = self.getDateTime(t).hour
-                        else:
-                            self.tmin = (
-                                self.getDateTime(t).replace(
-                                    hour=int(self.xmin),
-                                    minute=0,
-                                    second=0) - self.getDateTime(0)).total_seconds()
                         self.data.setup()
                         self.isConfig = True
                         if self.plot_server:
                             self.context = zmq.Context()
                             self.socket = self.context.socket(zmq.REQ)
                             self.socket.connect(self.plot_server)
                             self.poll = zmq.Poller()
                             self.poll.register(self.socket, zmq.POLLIN)
                     tm = getattr(dataOut, self.attr_time)
-                    if self.data and (tm - self.tmin) >= self.xrange*60*60:
+                    if self.data and 'time' in self.xaxis and (tm - self.tmin) >= self.xrange*60*60:
                         self.save_counter = self.save_period
                         self.__plot()
-                        if 'time' in self.xaxis:
-                            self.xmin += self.xrange
-                            if self.xmin >= 24:
-                                self.xmin -= 24
                         self.tmin += self.xrange*60*60
                         self.data.setup()
                         self.clear_figures()
                     self.data.update(dataOut, tm)
                     if self.isPlotConfig is False:
                         self.__setup_plot()
                         self.isPlotConfig = True
+                        if self.xaxis == 'time':
+                            dt = self.getDateTime(tm)
+                            if self.xmin is None:
+                                self.tmin = tm
+                                self.xmin = dt.hour
+                            minutes = (self.xmin-int(self.xmin)) * 60
+                            seconds = (minutes - int(minutes)) * 60
+                            self.tmin = (dt.replace(hour=int(self.xmin), minute=int(minutes), second=int(seconds)) -
+                                    datetime.datetime(1970, 1, 1)).total_seconds()
+                            if self.localtime:
+                                self.tmin += time.timezone
+                            if self.xmin is not None and self.xmax is not None:
+                                self.xrange = self.xmax - self.xmin
                     if self.throttle == 0:
                         self.__plot()
                     else:
                         self.__throttle_plot(self.__plot)#, coerce=coerce)
                 def close(self):
                     if self.data and not self.data.flagNoData:
                         self.save_counter = self.save_period
                         self.__plot()
                     if self.data and not self.data.flagNoData and self.pause:
                         figpause(10)

schainpy/model/graphics/jroplot_parameters.py 0 -7

             import os
             import datetime
             import numpy
             from schainpy.model.graphics.jroplot_base import Plot, plt
             from schainpy.model.graphics.jroplot_spectra import SpectraPlot, RTIPlot, CoherencePlot
             from schainpy.utils import log
             EARTH_RADIUS = 6.3710e3
             def ll2xy(lat1, lon1, lat2, lon2):
                 p = 0.017453292519943295
                 a = 0.5 - numpy.cos((lat2 - lat1) * p)/2 + numpy.cos(lat1 * p) * \
                     numpy.cos(lat2 * p) * (1 - numpy.cos((lon2 - lon1) * p)) / 2
                 r = 12742 * numpy.arcsin(numpy.sqrt(a))
                 theta = numpy.arctan2(numpy.sin((lon2-lon1)*p)*numpy.cos(lat2*p), numpy.cos(lat1*p)
                                       * numpy.sin(lat2*p)-numpy.sin(lat1*p)*numpy.cos(lat2*p)*numpy.cos((lon2-lon1)*p))
                 theta = -theta + numpy.pi/2
                 return r*numpy.cos(theta), r*numpy.sin(theta)
             def km2deg(km):
                 '''
                 Convert distance in km to degrees
                 '''
                 return numpy.rad2deg(km/EARTH_RADIUS)
             class SpectralMomentsPlot(SpectraPlot):
                 '''
                 Plot for Spectral Moments
                 '''
                 CODE = 'spc_moments'
                 colormap = 'jet'
-                plot_name = 'SpectralMoments'
                 plot_type = 'pcolor'
             class SnrPlot(RTIPlot):
                 '''
                 Plot for SNR Data
                 '''
                 CODE = 'snr'
                 colormap = 'jet'
-                plot_name = 'SNR'
             class DopplerPlot(RTIPlot):
                 '''
                 Plot for DOPPLER Data (1st moment)
                 '''
                 CODE = 'dop'
                 colormap = 'jet'
-                plot_name = 'DopplerShift'
             class PowerPlot(RTIPlot):
                 '''
                 Plot for Power Data (0 moment)
                 '''
                 CODE = 'pow'
                 colormap = 'jet'
-                plot_name = 'TotalPower'
             class SpectralWidthPlot(RTIPlot):
                 '''
                 Plot for Spectral Width Data (2nd moment)
                 '''
                 CODE = 'width'
                 colormap = 'jet'
-                plot_name = 'SpectralWidth'
             class SkyMapPlot(Plot):
                 '''
                 Plot for meteors detection data
                 '''
                 CODE = 'param'
                 def setup(self):
                     self.ncols = 1
                     self.nrows = 1
                     self.width = 7.2
                     self.height = 7.2
                     self.nplots = 1
                     self.xlabel = 'Zonal Zenith Angle (deg)'
                     self.ylabel = 'Meridional Zenith Angle (deg)'
                     self.polar = True
                     self.ymin = -180
                     self.ymax = 180
                     self.colorbar = False
                 def plot(self):
                     arrayParameters = numpy.concatenate(self.data['param'])
                     error = arrayParameters[:, -1]
                     indValid = numpy.where(error == 0)[0]
                     finalMeteor = arrayParameters[indValid, :]
                     finalAzimuth = finalMeteor[:, 3]
                     finalZenith = finalMeteor[:, 4]
                     x = finalAzimuth * numpy.pi / 180
                     y = finalZenith
                     ax = self.axes[0]
                     if ax.firsttime:
                         ax.plot = ax.plot(x, y, 'bo', markersize=5)[0]
                     else:
                         ax.plot.set_data(x, y)
                     dt1 = self.getDateTime(self.data.min_time).strftime('%y/%m/%d %H:%M:%S')
                     dt2 = self.getDateTime(self.data.max_time).strftime('%y/%m/%d %H:%M:%S')
                     title = 'Meteor Detection Sky Map\n %s - %s \n Number of events: %5.0f\n' % (dt1,
                                                                                                  dt2,
                                                                                                  len(x))
                     self.titles[0] = title
             class ParametersPlot(RTIPlot):
                 '''
                 Plot for data_param object
                 '''
                 CODE = 'param'
                 colormap = 'seismic'
-                plot_name = 'Parameters'
                 def setup(self):
                     self.xaxis = 'time'
                     self.ncols = 1
                     self.nrows = self.data.shape(self.CODE)[0]
                     self.nplots = self.nrows
                     self.plots_adjust.update({'hspace':0.8, 'left': 0.1, 'bottom': 0.08, 'right':0.95, 'top': 0.95})
                     if not self.xlabel:
                         self.xlabel = 'Time'
                     if self.showSNR:
                         self.nrows += 1
                         self.nplots += 1
                     self.ylabel = 'Height [km]'
                     if not self.titles:
                         self.titles = self.data.parameters \
                             if self.data.parameters else ['Param {}'.format(x) for x in range(self.nrows)]
                         if self.showSNR:
                             self.titles.append('SNR')
                 def plot(self):
                     self.data.normalize_heights()
                     self.x = self.data.times
                     self.y = self.data.heights
                     if self.showSNR:
                         self.z = numpy.concatenate(
                             (self.data[self.CODE], self.data['snr'])
                         )
                     else:
                         self.z = self.data[self.CODE]
                     self.z = numpy.ma.masked_invalid(self.z)
                     if self.decimation is None:
                         x, y, z = self.fill_gaps(self.x, self.y, self.z)
                     else:
                         x, y, z = self.fill_gaps(*self.decimate())
                     for n, ax in enumerate(self.axes):
                         self.zmax = self.zmax if self.zmax is not None else numpy.max(
                             self.z[n])
                         self.zmin = self.zmin if self.zmin is not None else numpy.min(
                             self.z[n])
                         if ax.firsttime:
                             if self.zlimits is not None:
                                 self.zmin, self.zmax = self.zlimits[n]
                             ax.plt = ax.pcolormesh(x, y, z[n].T * self.factors[n],
                                                    vmin=self.zmin,
                                                    vmax=self.zmax,
                                                    cmap=self.cmaps[n]
                                                    )
                         else:
                             if self.zlimits is not None:
                                 self.zmin, self.zmax = self.zlimits[n]
                             ax.collections.remove(ax.collections[0])
                             ax.plt = ax.pcolormesh(x, y, z[n].T * self.factors[n],
                                                    vmin=self.zmin,
                                                    vmax=self.zmax,
                                                    cmap=self.cmaps[n]
                                                    )
             class OutputPlot(ParametersPlot):
                 '''
                 Plot data_output object
                 '''
                 CODE = 'output'
                 colormap = 'seismic'
-                plot_name = 'Output'
             class PolarMapPlot(Plot):
                 '''
                 Plot for weather radar
                 '''
                 CODE = 'param'
                 colormap = 'seismic'
                 def setup(self):
                     self.ncols = 1
                     self.nrows = 1
                     self.width = 9
                     self.height = 8
                     self.mode = self.data.meta['mode']
                     if self.channels is not None:
                         self.nplots = len(self.channels)
                         self.nrows = len(self.channels)
                     else:
                         self.nplots = self.data.shape(self.CODE)[0]
                         self.nrows = self.nplots
                         self.channels = list(range(self.nplots))
                     if self.mode == 'E':
                         self.xlabel = 'Longitude'
                         self.ylabel = 'Latitude'
                     else:
                         self.xlabel = 'Range (km)'
                         self.ylabel = 'Height (km)'
                     self.bgcolor = 'white'
                     self.cb_labels = self.data.meta['units']
                     self.lat = self.data.meta['latitude']
                     self.lon = self.data.meta['longitude']
                     self.xmin, self.xmax = float(
                         km2deg(self.xmin) + self.lon), float(km2deg(self.xmax) + self.lon)
                     self.ymin, self.ymax = float(
                         km2deg(self.ymin) + self.lat), float(km2deg(self.ymax) + self.lat)
                     #  self.polar = True
                 def plot(self):
                     for n, ax in enumerate(self.axes):
                         data = self.data['param'][self.channels[n]]
                         zeniths = numpy.linspace(
 , self.data.meta['max_range'], data.shape[1])
                         if self.mode == 'E':
                             azimuths = -numpy.radians(self.data.heights)+numpy.pi/2
                             r, theta = numpy.meshgrid(zeniths, azimuths)
                             x, y = r*numpy.cos(theta)*numpy.cos(numpy.radians(self.data.meta['elevation'])), r*numpy.sin(
                                 theta)*numpy.cos(numpy.radians(self.data.meta['elevation']))
                             x = km2deg(x) + self.lon
                             y = km2deg(y) + self.lat
                         else:
                             azimuths = numpy.radians(self.data.heights)
                             r, theta = numpy.meshgrid(zeniths, azimuths)
                             x, y = r*numpy.cos(theta), r*numpy.sin(theta)
                         self.y = zeniths
                         if ax.firsttime:
                             if self.zlimits is not None:
                                 self.zmin, self.zmax = self.zlimits[n]
                             ax.plt = ax.pcolormesh(  # r, theta, numpy.ma.array(data, mask=numpy.isnan(data)),
                                 x, y, numpy.ma.array(data, mask=numpy.isnan(data)),
                                 vmin=self.zmin,
                                 vmax=self.zmax,
                                 cmap=self.cmaps[n])
                         else:
                             if self.zlimits is not None:
                                 self.zmin, self.zmax = self.zlimits[n]
                             ax.collections.remove(ax.collections[0])
                             ax.plt = ax.pcolormesh(  # r, theta, numpy.ma.array(data, mask=numpy.isnan(data)),
                                 x, y, numpy.ma.array(data, mask=numpy.isnan(data)),
                                 vmin=self.zmin,
                                 vmax=self.zmax,
                                 cmap=self.cmaps[n])
                         if self.mode == 'A':
                             continue
                         # plot district names
                         f = open('/data/workspace/schain_scripts/distrito.csv')
                         for line in f:
                             label, lon, lat = [s.strip() for s in line.split(',') if s]
                             lat = float(lat)
                             lon = float(lon)
                             # ax.plot(lon, lat, '.b', ms=2)
                             ax.text(lon, lat, label.decode('utf8'), ha='center',
                                     va='bottom', size='8', color='black')
                         # plot limites
                         limites = []
                         tmp = []
                         for line in open('/data/workspace/schain_scripts/lima.csv'):
                             if '#' in line:
                                 if tmp:
                                     limites.append(tmp)
                                 tmp = []
                                 continue
                             values = line.strip().split(',')
                             tmp.append((float(values[0]), float(values[1])))
                         for points in limites:
                             ax.add_patch(
                                 Polygon(points, ec='k', fc='none', ls='--', lw=0.5))
                         # plot Cuencas
                         for cuenca in ('rimac', 'lurin', 'mala', 'chillon', 'chilca', 'chancay-huaral'):
                             f = open('/data/workspace/schain_scripts/{}.csv'.format(cuenca))
                             values = [line.strip().split(',') for line in f]
                             points = [(float(s[0]), float(s[1])) for s in values]
                             ax.add_patch(Polygon(points, ec='b', fc='none'))
                         # plot grid
                         for r in (15, 30, 45, 60):
                             ax.add_artist(plt.Circle((self.lon, self.lat),
                                                      km2deg(r), color='0.6', fill=False, lw=0.2))
                             ax.text(
                                 self.lon + (km2deg(r))*numpy.cos(60*numpy.pi/180),
                                 self.lat + (km2deg(r))*numpy.sin(60*numpy.pi/180),
                                 '{}km'.format(r),
                                 ha='center', va='bottom', size='8', color='0.6', weight='heavy')
                     if self.mode == 'E':
                         title = 'El={}$^\circ$'.format(self.data.meta['elevation'])
                         label = 'E{:02d}'.format(int(self.data.meta['elevation']))
                     else:
                         title = 'Az={}$^\circ$'.format(self.data.meta['azimuth'])
                         label = 'A{:02d}'.format(int(self.data.meta['azimuth']))
                     self.save_labels = ['{}-{}'.format(lbl, label) for lbl in self.labels]
                     self.titles = ['{} {}'.format(
                         self.data.parameters[x], title) for x in self.channels]

schainpy/model/graphics/jroplot_spectra.py +4 -12

             '''
             Created on Jul 9, 2014
             Modified on May 10, 2020
             @author: Juan C. Espinoza
             '''
             import os
             import datetime
             import numpy
             from schainpy.model.graphics.jroplot_base import Plot, plt
             class SpectraPlot(Plot):
                 '''
                 Plot for Spectra data
                 '''
                 CODE = 'spc'
                 colormap = 'jet'
-                plot_name = 'Spectra'
                 plot_type = 'pcolor'
                 def setup(self):
                     self.nplots = len(self.data.channels)
                     self.ncols = int(numpy.sqrt(self.nplots) + 0.9)
                     self.nrows = int((1.0 * self.nplots / self.ncols) + 0.9)
-                    self.height = 3 * self.nrows
+                    self.height = 2.6 * self.nrows
                     self.cb_label = 'dB'
                     if self.showprofile:
                         self.width = 4 * self.ncols
                     else:
                         self.width = 3.5 * self.ncols
                     self.plots_adjust.update({'wspace': 0.4, 'hspace':0.4, 'left': 0.1, 'right': 0.9, 'bottom': 0.08})
                     self.ylabel = 'Range [km]'
                 def plot(self):
                     if self.xaxis == "frequency":
                         x = self.data.xrange[0]
                         self.xlabel = "Frequency (kHz)"
                     elif self.xaxis == "time":
                         x = self.data.xrange[1]
                         self.xlabel = "Time (ms)"
                     else:
                         x = self.data.xrange[2]
                         self.xlabel = "Velocity (m/s)"
                     if self.CODE == 'spc_moments':
                         x = self.data.xrange[2]
                         self.xlabel = "Velocity (m/s)"
                     self.titles = []
                     y = self.data.heights
                     self.y = y
                     z = self.data['spc']
                     for n, ax in enumerate(self.axes):
                         noise = self.data['noise'][n][-1]
                         if self.CODE == 'spc_moments':
                             mean = self.data['moments'][n, :, 1, :][-1]
                         if ax.firsttime:
                             self.xmax = self.xmax if self.xmax else numpy.nanmax(x)
                             self.xmin = self.xmin if self.xmin else -self.xmax
                             self.zmin = self.zmin if self.zmin else numpy.nanmin(z)
                             self.zmax = self.zmax if self.zmax else numpy.nanmax(z)
                             ax.plt = ax.pcolormesh(x, y, z[n].T,
                                                    vmin=self.zmin,
                                                    vmax=self.zmax,
                                                    cmap=plt.get_cmap(self.colormap)
                                                    )
                             if self.showprofile:
                                 ax.plt_profile = self.pf_axes[n].plot(
                                     self.data['rti'][n][-1], y)[0]
                                 ax.plt_noise = self.pf_axes[n].plot(numpy.repeat(noise, len(y)), y,
                                                                     color="k", linestyle="dashed", lw=1)[0]
                             if self.CODE == 'spc_moments':
                                 ax.plt_mean = ax.plot(mean, y, color='k')[0]
                         else:
                             ax.plt.set_array(z[n].T.ravel())
                             if self.showprofile:
                                 ax.plt_profile.set_data(self.data['rti'][n][-1], y)
                                 ax.plt_noise.set_data(numpy.repeat(noise, len(y)), y)
                             if self.CODE == 'spc_moments':
                                 ax.plt_mean.set_data(mean, y)
                         self.titles.append('CH {}: {:3.2f}dB'.format(n, noise))
             class CrossSpectraPlot(Plot):
                 CODE = 'cspc'
                 colormap = 'jet'
-                plot_name = 'CrossSpectra'
                 plot_type = 'pcolor'
                 zmin_coh = None
                 zmax_coh = None
                 zmin_phase = None
                 zmax_phase = None
                 def setup(self):
                     self.ncols = 4
                     self.nrows = len(self.data.pairs)
                     self.nplots = self.nrows * 4
-                    self.width = 3.4 * self.ncols
+                    self.width = 3.1 * self.ncols
-                    self.height = 3 * self.nrows
+                    self.height = 2.6 * self.nrows
                     self.ylabel = 'Range [km]'
                     self.showprofile = False
-                    self.plots_adjust.update({'bottom': 0.08})
+                    self.plots_adjust.update({'left': 0.08, 'right': 0.92, 'wspace': 0.5, 'hspace':0.4, 'top':0.95, 'bottom': 0.08})
                 def plot(self):
                     if self.xaxis == "frequency":
                         x = self.data.xrange[0]
                         self.xlabel = "Frequency (kHz)"
                     elif self.xaxis == "time":
                         x = self.data.xrange[1]
                         self.xlabel = "Time (ms)"
                     else:
                         x = self.data.xrange[2]
                         self.xlabel = "Velocity (m/s)"
                     self.titles = []
                     y = self.data.heights
                     self.y = y
                     nspc = self.data['spc']
                     spc = self.data['cspc'][0]
                     cspc = self.data['cspc'][1]
                     for n in range(self.nrows):
                         noise = self.data['noise'][:,-1]
                         pair = self.data.pairs[n]
                         ax = self.axes[4 * n]
                         if ax.firsttime:
                             self.xmax = self.xmax if self.xmax else numpy.nanmax(x)
                             self.xmin = self.xmin if self.xmin else -self.xmax
                             self.zmin = self.zmin if self.zmin else numpy.nanmin(nspc)
                             self.zmax = self.zmax if self.zmax else numpy.nanmax(nspc)
                             ax.plt = ax.pcolormesh(x , y , nspc[pair[0]].T,
                                                    vmin=self.zmin,
                                                    vmax=self.zmax,
                                                    cmap=plt.get_cmap(self.colormap)
                                                    )
                         else:
                             ax.plt.set_array(nspc[pair[0]].T.ravel())
                         self.titles.append('CH {}: {:3.2f}dB'.format(pair[0], noise[pair[0]]))
                         ax = self.axes[4 * n + 1]
                         if ax.firsttime:
                             ax.plt = ax.pcolormesh(x , y, nspc[pair[1]].T,
                                                    vmin=self.zmin,
                                                    vmax=self.zmax,
                                                    cmap=plt.get_cmap(self.colormap)
                                                    )
                         else:
                             ax.plt.set_array(nspc[pair[1]].T.ravel())
                         self.titles.append('CH {}: {:3.2f}dB'.format(pair[1], noise[pair[1]]))
                         out = cspc[n] / numpy.sqrt(spc[pair[0]] * spc[pair[1]])
                         coh = numpy.abs(out)
                         phase = numpy.arctan2(out.imag, out.real) * 180 / numpy.pi
                         ax = self.axes[4 * n + 2]
                         if ax.firsttime:
                             ax.plt = ax.pcolormesh(x, y, coh.T,
                                                    vmin=0,
                                                    vmax=1,
                                                    cmap=plt.get_cmap(self.colormap_coh)
                                                    )
                         else:
                             ax.plt.set_array(coh.T.ravel())
                         self.titles.append(
                             'Coherence Ch{} * Ch{}'.format(pair[0], pair[1]))
                         ax = self.axes[4 * n + 3]
                         if ax.firsttime:
                             ax.plt = ax.pcolormesh(x, y, phase.T,
                                                    vmin=-180,
                                                    vmax=180,
                                                    cmap=plt.get_cmap(self.colormap_phase)
                                                    )
                         else:
                             ax.plt.set_array(phase.T.ravel())
                         self.titles.append('Phase CH{} * CH{}'.format(pair[0], pair[1]))
             class RTIPlot(Plot):
                 '''
                 Plot for RTI data
                 '''
                 CODE = 'rti'
                 colormap = 'jet'
-                plot_name = 'RTI'
                 plot_type = 'pcolorbuffer'
                 def setup(self):
                     self.xaxis = 'time'
                     self.ncols = 1
                     self.nrows = len(self.data.channels)
                     self.nplots = len(self.data.channels)
                     self.ylabel = 'Range [km]'
                     self.xlabel = 'Time'
                     self.cb_label = 'dB'
                     self.plots_adjust.update({'hspace':0.8, 'left': 0.1, 'bottom': 0.08, 'right':0.95})
                     self.titles = ['{} Channel {}'.format(
                         self.CODE.upper(), x) for x in range(self.nrows)]
                 def plot(self):
                     self.x = self.data.times
                     self.y = self.data.heights
                     self.z = self.data[self.CODE]
                     self.z = numpy.ma.masked_invalid(self.z)
                     if self.decimation is None:
                         x, y, z = self.fill_gaps(self.x, self.y, self.z)
                     else:
                         x, y, z = self.fill_gaps(*self.decimate())
                     for n, ax in enumerate(self.axes):
                         self.zmin = self.zmin if self.zmin else numpy.min(self.z)
                         self.zmax = self.zmax if self.zmax else numpy.max(self.z)
                         if ax.firsttime:
                             ax.plt = ax.pcolormesh(x, y, z[n].T,
                                                    vmin=self.zmin,
                                                    vmax=self.zmax,
                                                    cmap=plt.get_cmap(self.colormap)
                                                    )
                             if self.showprofile:
                                 ax.plot_profile = self.pf_axes[n].plot(
                                     self.data['rti'][n][-1], self.y)[0]
                                 ax.plot_noise = self.pf_axes[n].plot(numpy.repeat(self.data['noise'][n][-1], len(self.y)), self.y,
                                                                      color="k", linestyle="dashed", lw=1)[0]
                         else:
                             ax.collections.remove(ax.collections[0])
                             ax.plt = ax.pcolormesh(x, y, z[n].T,
                                                    vmin=self.zmin,
                                                    vmax=self.zmax,
                                                    cmap=plt.get_cmap(self.colormap)
                                                    )
                             if self.showprofile:
                                 ax.plot_profile.set_data(self.data['rti'][n][-1], self.y)
                                 ax.plot_noise.set_data(numpy.repeat(
                                     self.data['noise'][n][-1], len(self.y)), self.y)
             class CoherencePlot(RTIPlot):
                 '''
                 Plot for Coherence data
                 '''
                 CODE = 'coh'
-                plot_name = 'Coherence'
                 def setup(self):
                     self.xaxis = 'time'
                     self.ncols = 1
                     self.nrows = len(self.data.pairs)
                     self.nplots = len(self.data.pairs)
                     self.ylabel = 'Range [km]'
                     self.xlabel = 'Time'
                     self.plots_adjust.update({'hspace':0.6, 'left': 0.1, 'bottom': 0.1,'right':0.95})
                     if self.CODE == 'coh':
                         self.cb_label = ''
                         self.titles = [
                             'Coherence Map Ch{} * Ch{}'.format(x[0], x[1]) for x in self.data.pairs]
                     else:
                         self.cb_label = 'Degrees'
                         self.titles = [
                             'Phase Map Ch{} * Ch{}'.format(x[0], x[1]) for x in self.data.pairs]
             class PhasePlot(CoherencePlot):
                 '''
                 Plot for Phase map data
                 '''
                 CODE = 'phase'
                 colormap = 'seismic'
-                plot_name = 'Phase'
             class NoisePlot(Plot):
                 '''
                 Plot for noise
                 '''
                 CODE = 'noise'
-                plot_name = 'Noise'
                 plot_type = 'scatterbuffer'
                 def setup(self):
                     self.xaxis = 'time'
                     self.ncols = 1
                     self.nrows = 1
                     self.nplots = 1
                     self.ylabel = 'Intensity [dB]'
                     self.xlabel = 'Time'
                     self.titles = ['Noise']
                     self.colorbar = False
                 def plot(self):
                     x = self.data.times
                     xmin = self.data.min_time
                     xmax = xmin + self.xrange * 60 * 60
                     Y = self.data[self.CODE]
                     if self.axes[0].firsttime:
                         for ch in self.data.channels:
                             y = Y[ch]
                             self.axes[0].plot(x, y, lw=1, label='Ch{}'.format(ch))
                         plt.legend()
                     else:
                         for ch in self.data.channels:
                             y = Y[ch]
                             self.axes[0].lines[ch].set_data(x, y)
                     self.ymin = numpy.nanmin(Y) - 5
                     self.ymax = numpy.nanmax(Y) + 5
             class PowerProfilePlot(Plot):
                 CODE = 'spcprofile'
-                plot_name = 'Power Profile'
                 plot_type = 'scatter'
                 buffering = False
                 def setup(self):
                     self.ncols = 1
                     self.nrows = 1
                     self.nplots = 1
                     self.height = 4
                     self.width = 3
                     self.ylabel = 'Range [km]'
                     self.xlabel = 'Intensity [dB]'
                     self.titles = ['Power Profile']
                     self.colorbar = False
                 def plot(self):
                     y = self.data.heights
                     self.y = y
                     x = self.data['spcprofile']
                     if self.xmin is None: self.xmin = numpy.nanmin(x)*0.9
                     if self.xmax is None: self.xmax = numpy.nanmax(x)*1.1
                     if self.axes[0].firsttime:
                         for ch in self.data.channels:
                             self.axes[0].plot(x[ch], y, lw=1, label='Ch{}'.format(ch))
                         plt.legend()
                     else:
                         for ch in self.data.channels:
                             self.axes[0].lines[ch].set_data(x[ch], y)
             class SpectraCutPlot(Plot):
                 CODE = 'spc_cut'
-                plot_name = 'Spectra Cut'
                 plot_type = 'scatter'
                 buffering = False
                 def setup(self):
                     self.nplots = len(self.data.channels)
                     self.ncols = int(numpy.sqrt(self.nplots) + 0.9)
                     self.nrows = int((1.0 * self.nplots / self.ncols) + 0.9)
                     self.width = 3.4 * self.ncols + 1.5
                     self.height = 3 * self.nrows
                     self.ylabel = 'Power [dB]'
                     self.colorbar = False
                     self.plots_adjust.update({'left':0.1, 'hspace':0.3, 'right': 0.75, 'bottom':0.08})
                 def plot(self):
                     if self.xaxis == "frequency":
                         x = self.data.xrange[0][1:]
                         self.xlabel = "Frequency (kHz)"
                     elif self.xaxis == "time":
                         x = self.data.xrange[1]
                         self.xlabel = "Time (ms)"
                     else:
                         x = self.data.xrange[2]
                         self.xlabel = "Velocity (m/s)"
                     self.titles = []
                     y = self.data.heights
                     #self.y = y
                     z = self.data['spc_cut']
                     if self.height_index:
                         index = numpy.array(self.height_index)
                     else:
                         index = numpy.arange(0, len(y), int((len(y))/9))
                     for n, ax in enumerate(self.axes):
                         if ax.firsttime:
                             self.xmax = self.xmax if self.xmax else numpy.nanmax(x)
                             self.xmin = self.xmin if self.xmin else -self.xmax
                             self.ymin = self.ymin if self.ymin else numpy.nanmin(z)
                             self.ymax = self.ymax if self.ymax else numpy.nanmax(z)
                             ax.plt = ax.plot(x, z[n, :, index].T)
                             labels = ['Range = {:2.1f}km'.format(y[i]) for i in index]
                             self.figures[0].legend(ax.plt, labels, loc='center right')
                         else:
                             for i, line in enumerate(ax.plt):
                                 line.set_data(x, z[n, :, i])
                         self.titles.append('CH {}'.format(n))
             class BeaconPhase(Plot):
                 __isConfig = None
                 __nsubplots = None
                 PREFIX = 'beacon_phase'
                 def __init__(self):
                     Plot.__init__(self)
                     self.timerange = 24*60*60
                     self.isConfig = False
                     self.__nsubplots = 1
                     self.counter_imagwr = 0
                     self.WIDTH = 800
                     self.HEIGHT = 400
                     self.WIDTHPROF = 120
                     self.HEIGHTPROF = 0
                     self.xdata = None
                     self.ydata = None
                     self.PLOT_CODE = BEACON_CODE
                     self.FTP_WEI = None
                     self.EXP_CODE = None
                     self.SUB_EXP_CODE = None
                     self.PLOT_POS = None
                     self.filename_phase = None
                     self.figfile = None
                     self.xmin = None
                     self.xmax = None
                 def getSubplots(self):
                     ncol = 1
                     nrow = 1
                     return nrow, ncol
                 def setup(self, id, nplots, wintitle, showprofile=True, show=True):
                     self.__showprofile = showprofile
                     self.nplots = nplots
                     ncolspan = 7
                     colspan = 6
                     self.__nsubplots = 2
                     self.createFigure(id = id,
                                       wintitle = wintitle,
                                       widthplot = self.WIDTH+self.WIDTHPROF,
                                       heightplot = self.HEIGHT+self.HEIGHTPROF,
                                       show=show)
                     nrow, ncol = self.getSubplots()
                     self.addAxes(nrow, ncol*ncolspan, 0, 0, colspan, 1)
                 def save_phase(self, filename_phase):
                     f = open(filename_phase,'w+')
                     f.write('\n\n')
                     f.write('JICAMARCA RADIO OBSERVATORY - Beacon Phase \n')
                     f.write('DD MM YYYY  HH MM SS   pair(2,0) pair(2,1) pair(2,3) pair(2,4)\n\n' )
                     f.close()
                 def save_data(self, filename_phase, data, data_datetime):
                     f=open(filename_phase,'a')
                     timetuple_data = data_datetime.timetuple()
                     day = str(timetuple_data.tm_mday)
                     month = str(timetuple_data.tm_mon)
                     year = str(timetuple_data.tm_year)
                     hour = str(timetuple_data.tm_hour)
                     minute = str(timetuple_data.tm_min)
                     second = str(timetuple_data.tm_sec)
                     f.write(day+' '+month+' '+year+'  '+hour+' '+minute+' '+second+'   '+str(data[0])+'   '+str(data[1])+'   '+str(data[2])+'   '+str(data[3])+'\n')
                     f.close()
                 def plot(self):
                     log.warning('TODO: Not yet implemented...')
                 def run(self, dataOut, id, wintitle="", pairsList=None, showprofile='True',
                         xmin=None, xmax=None, ymin=None, ymax=None, hmin=None, hmax=None,
                         timerange=None,
                         save=False, figpath='./', figfile=None, show=True, ftp=False, wr_period=1,
                         server=None, folder=None, username=None, password=None,
                         ftp_wei=0, exp_code=0, sub_exp_code=0, plot_pos=0):
                     if dataOut.flagNoData:
                         return dataOut
                     if not isTimeInHourRange(dataOut.datatime, xmin, xmax):
                         return
                     if pairsList == None:
                         pairsIndexList = dataOut.pairsIndexList[:10]
                     else:
                         pairsIndexList = []
                         for pair in pairsList:
                             if pair not in dataOut.pairsList:
                                 raise ValueError("Pair %s is not in dataOut.pairsList" %(pair))
                             pairsIndexList.append(dataOut.pairsList.index(pair))
                     if pairsIndexList == []:
                         return
              #         if len(pairsIndexList) > 4:
              #             pairsIndexList = pairsIndexList[0:4]
                     hmin_index = None
                     hmax_index = None
                     if hmin != None and hmax != None:
                         indexes = numpy.arange(dataOut.nHeights)
                         hmin_list = indexes[dataOut.heightList >= hmin]
                         hmax_list = indexes[dataOut.heightList <= hmax]
                         if hmin_list.any():
                             hmin_index = hmin_list[0]
                         if hmax_list.any():
                             hmax_index = hmax_list[-1]+1
                     x = dataOut.getTimeRange()
                     #y = dataOut.getHeiRange()
                     thisDatetime = dataOut.datatime
                     title = wintitle + " Signal Phase" # : %s" %(thisDatetime.strftime("%d-%b-%Y"))
                     xlabel = "Local Time"
                     ylabel = "Phase (degrees)"
                     update_figfile = False
                     nplots = len(pairsIndexList)
                     #phase = numpy.zeros((len(pairsIndexList),len(dataOut.beacon_heiIndexList)))
                     phase_beacon = numpy.zeros(len(pairsIndexList))
                     for i in range(nplots):
                         pair = dataOut.pairsList[pairsIndexList[i]]
                         ccf = numpy.average(dataOut.data_cspc[pairsIndexList[i], :, hmin_index:hmax_index], axis=0)
                         powa = numpy.average(dataOut.data_spc[pair[0], :, hmin_index:hmax_index], axis=0)
                         powb = numpy.average(dataOut.data_spc[pair[1], :, hmin_index:hmax_index], axis=0)
                         avgcoherenceComplex = ccf/numpy.sqrt(powa*powb)
                         phase = numpy.arctan2(avgcoherenceComplex.imag, avgcoherenceComplex.real)*180/numpy.pi
                         if dataOut.beacon_heiIndexList:
                             phase_beacon[i] = numpy.average(phase[dataOut.beacon_heiIndexList])
                         else:
                             phase_beacon[i] = numpy.average(phase)
                     if not self.isConfig:
                         nplots = len(pairsIndexList)
                         self.setup(id=id,
                                    nplots=nplots,
                                    wintitle=wintitle,
                                    showprofile=showprofile,
                                    show=show)
                         if timerange != None:
                             self.timerange = timerange
                         self.xmin, self.xmax = self.getTimeLim(x, xmin, xmax, timerange)
                         if ymin == None: ymin = 0
                         if ymax == None: ymax = 360
                         self.FTP_WEI = ftp_wei
                         self.EXP_CODE = exp_code
                         self.SUB_EXP_CODE = sub_exp_code
                         self.PLOT_POS = plot_pos
                         self.name = thisDatetime.strftime("%Y%m%d_%H%M%S")
                         self.isConfig = True
                         self.figfile = figfile
                         self.xdata = numpy.array([])
                         self.ydata = numpy.array([])
                         update_figfile = True
                         #open file beacon phase
                         path = '%s%03d' %(self.PREFIX, self.id)
                         beacon_file = os.path.join(path,'%s.txt'%self.name)
                         self.filename_phase = os.path.join(figpath,beacon_file)
                         #self.save_phase(self.filename_phase)
                     #store data beacon phase
                     #self.save_data(self.filename_phase, phase_beacon, thisDatetime)
                     self.setWinTitle(title)
                     title = "Phase Plot %s" %(thisDatetime.strftime("%Y/%m/%d %H:%M:%S"))
                     legendlabels = ["Pair (%d,%d)"%(pair[0], pair[1]) for pair in dataOut.pairsList]
                     axes = self.axesList[0]
                     self.xdata = numpy.hstack((self.xdata, x[0:1]))
                     if len(self.ydata)==0:
                         self.ydata = phase_beacon.reshape(-1,1)
                     else:
                         self.ydata = numpy.hstack((self.ydata, phase_beacon.reshape(-1,1)))
                     axes.pmultilineyaxis(x=self.xdata, y=self.ydata,
                                 xmin=self.xmin, xmax=self.xmax, ymin=ymin, ymax=ymax,
                                 xlabel=xlabel, ylabel=ylabel, title=title, legendlabels=legendlabels, marker='x', markersize=8, linestyle="solid",
                                 XAxisAsTime=True, grid='both'
                                 )
                     self.draw()
                     if dataOut.ltctime >= self.xmax:
                         self.counter_imagwr = wr_period
                         self.isConfig = False
                         update_figfile = True
                     self.save(figpath=figpath,
                               figfile=figfile,
                               save=save,
                               ftp=ftp,
                               wr_period=wr_period,
                               thisDatetime=thisDatetime,
                               update_figfile=update_figfile)
                     return dataOut

schainpy/model/graphics/jroplot_voltage.py 0 -5

             '''
             Created on Jul 9, 2014
             @author: roj-idl71
             '''
             import os
             import datetime
             import numpy
             from schainpy.model.graphics.jroplot_base import Plot, plt
             class ScopePlot(Plot):
                 '''
                    Plot for Scope
                 '''
                 CODE = 'scope'
-                plot_name = 'Scope'
                 plot_type = 'scatter'
                 def setup(self):
                     self.xaxis = 'Range (Km)'
                     self.ncols = 1
                     self.nrows = 1
                     self.nplots = 1
                     self.ylabel = 'Intensity [dB]'
                     self.titles = ['Scope']
                     self.colorbar = False
                     self.width = 6
                     self.height = 4
                 def plot_iq(self, x, y, channelIndexList, thisDatetime, wintitle):
                     yreal = y[channelIndexList,:].real
                     yimag = y[channelIndexList,:].imag
                     title = wintitle + " Scope: %s" %(thisDatetime.strftime("%d-%b-%Y"))
                     self.xlabel = "Range (Km)"
                     self.ylabel = "Intensity - IQ"
                     self.y = yreal
                     self.x = x
                     self.xmin = min(x)
                     self.xmax = max(x)
                     self.titles[0] = title
                     for i,ax in enumerate(self.axes):
                         title = "Channel %d" %(i)
                         if ax.firsttime:
                             ax.plt_r = ax.plot(x, yreal[i,:], color='b')[0]
                             ax.plt_i = ax.plot(x, yimag[i,:], color='r')[0]
                         else:
                             ax.plt_r.set_data(x, yreal[i,:])
                             ax.plt_i.set_data(x, yimag[i,:])
                 def plot_power(self, x, y, channelIndexList, thisDatetime, wintitle):
                     y = y[channelIndexList,:] * numpy.conjugate(y[channelIndexList,:])
                     yreal = y.real
                     yreal = 10*numpy.log10(yreal)
                     self.y = yreal
                     title = wintitle + " Scope: %s" %(thisDatetime.strftime("%d-%b-%Y"))
                     self.xlabel = "Range (Km)"
                     self.ylabel = "Intensity"
                     self.xmin = min(x)
                     self.xmax = max(x)
                     self.titles[0] = title
                     for i,ax in enumerate(self.axes):
                         title = "Channel %d" %(i)
                         ychannel = yreal[i,:]
                         if ax.firsttime:
                             ax.plt_r = ax.plot(x, ychannel)[0]
                         else:
                             #pass
                             ax.plt_r.set_data(x, ychannel)
                 def plot_weatherpower(self, x, y, channelIndexList, thisDatetime, wintitle):
                     y      = y[channelIndexList,:]
                     yreal  = y.real
                     yreal  = 10*numpy.log10(yreal)
                     self.y = yreal
                     title  = wintitle + " Scope: %s" %(thisDatetime.strftime("%d-%b-%Y %H:%M:%S"))
                     self.xlabel = "Range (Km)"
                     self.ylabel = "Intensity"
                     self.xmin   = min(x)
                     self.xmax   = max(x)
                     self.titles[0] =title
                     for i,ax in enumerate(self.axes):
                         title    = "Channel %d" %(i)
                         ychannel = yreal[i,:]
                         if ax.firsttime:
                             ax.plt_r = ax.plot(x, ychannel)[0]
                         else:
                             #pass
                             ax.plt_r.set_data(x, ychannel)
                 def plot_weathervelocity(self, x, y, channelIndexList, thisDatetime, wintitle):
                     x = x[channelIndexList,:]
                     yreal  = y
                     self.y = yreal
                     title = wintitle + " Scope: %s" %(thisDatetime.strftime("%d-%b-%Y %H:%M:%S"))
                     self.xlabel = "Velocity (m/s)"
                     self.ylabel = "Range (Km)"
                     self.xmin   = numpy.min(x)
                     self.xmax   = numpy.max(x)
                     self.titles[0] =title
                     for i,ax in enumerate(self.axes):
                         title    = "Channel %d" %(i)
                         xchannel    = x[i,:]
                         if ax.firsttime:
                             ax.plt_r = ax.plot(xchannel, yreal)[0]
                         else:
                             #pass
                             ax.plt_r.set_data(xchannel, yreal)
                 def plot_weatherspecwidth(self, x, y, channelIndexList, thisDatetime, wintitle):
                     x = x[channelIndexList,:]
                     yreal  = y
                     self.y = yreal
                     title = wintitle + " Scope: %s" %(thisDatetime.strftime("%d-%b-%Y %H:%M:%S"))
                     self.xlabel = "width "
                     self.ylabel = "Range (Km)"
                     self.xmin   = numpy.min(x)
                     self.xmax   = numpy.max(x)
                     self.titles[0] =title
                     for i,ax in enumerate(self.axes):
                         title    = "Channel %d" %(i)
                         xchannel    = x[i,:]
                         if ax.firsttime:
                             ax.plt_r = ax.plot(xchannel, yreal)[0]
                         else:
                             #pass
                             ax.plt_r.set_data(xchannel, yreal)
                 def plot(self):
                     if self.channels:
                         channels = self.channels
                     else:
                         channels = self.data.channels
                     thisDatetime = datetime.datetime.utcfromtimestamp(self.data.times[-1])
                     if self.CODE == "pp_power":
                         scope = self.data['pp_power']
                     elif self.CODE == "pp_signal":
                         scope = self.data["pp_signal"]
                     elif self.CODE == "pp_velocity":
                         scope = self.data["pp_velocity"]
                     elif self.CODE == "pp_specwidth":
                         scope = self.data["pp_specwidth"]
                     else:
                         scope =self.data["scope"]
                     if self.data.flagDataAsBlock:
                         for i in range(self.data.nProfiles):
                             wintitle1 = " [Profile = %d] " %i
                             if self.CODE =="scope":
                                 if self.type == "power":
                                     self.plot_power(self.data.heights,
                                                     scope[:,i,:],
                                                     channels,
                                                     thisDatetime,
                                                     wintitle1
                                                     )
                                 if self.type == "iq":
                                     self.plot_iq(self.data.heights,
                                                  scope[:,i,:],
                                                  channels,
                                                  thisDatetime,
                                                  wintitle1
                                                 )
                             if self.CODE=="pp_power":
                                 self.plot_weatherpower(self.data.heights,
                                            scope[:,i,:],
                                            channels,
                                            thisDatetime,
                                            wintitle
                                            )
                             if self.CODE=="pp_signal":
                                 self.plot_weatherpower(self.data.heights,
                                            scope[:,i,:],
                                            channels,
                                            thisDatetime,
                                            wintitle
                                            )
                             if self.CODE=="pp_velocity":
                                 self.plot_weathervelocity(scope[:,i,:],
                                            self.data.heights,
                                            channels,
                                            thisDatetime,
                                            wintitle
                                            )
                             if self.CODE=="pp_spcwidth":
                                 self.plot_weatherspecwidth(scope[:,i,:],
                                            self.data.heights,
                                            channels,
                                            thisDatetime,
                                            wintitle
                                            )
                     else:
                         wintitle = " [Profile = %d] " %self.data.profileIndex
                         if self.CODE== "scope":
                             if self.type == "power":
                                  self.plot_power(self.data.heights,
                                             scope,
                                             channels,
                                             thisDatetime,
                                             wintitle
                                             )
                             if self.type == "iq":
                                 self.plot_iq(self.data.heights,
                                             scope,
                                             channels,
                                             thisDatetime,
                                             wintitle
                                             )
                         if self.CODE=="pp_power":
                             self.plot_weatherpower(self.data.heights,
                                                 scope,
                                                 channels,
                                                 thisDatetime,
                                                 wintitle
                                                    )
                         if self.CODE=="pp_signal":
                             self.plot_weatherpower(self.data.heights,
                                                 scope,
                                                 channels,
                                                 thisDatetime,
                                                 wintitle
                                                    )
                         if self.CODE=="pp_velocity":
                             self.plot_weathervelocity(scope,
                                                    self.data.heights,
                                                    channels,
                                                    thisDatetime,
                                                    wintitle
                                                    )
                         if self.CODE=="pp_specwidth":
                             self.plot_weatherspecwidth(scope,
                                                    self.data.heights,
                                                    channels,
                                                    thisDatetime,
                                                    wintitle
                                                    )
             class PulsepairPowerPlot(ScopePlot):
                 '''
                 Plot for  P= S+N
                 '''
                 CODE = 'pp_power'
-                plot_name = 'PulsepairPower'
                 plot_type = 'scatter'
                 buffering = False
             class PulsepairVelocityPlot(ScopePlot):
                 '''
                 Plot for VELOCITY
                 '''
                 CODE = 'pp_velocity'
-                plot_name = 'PulsepairVelocity'
                 plot_type = 'scatter'
                 buffering = False
             class PulsepairSpecwidthPlot(ScopePlot):
                 '''
                 Plot for WIDTH
                 '''
                 CODE = 'pp_specwidth'
-                plot_name = 'PulsepairSpecwidth'
                 plot_type = 'scatter'
                 buffering = False
             class PulsepairSignalPlot(ScopePlot):
                 '''
                 Plot for S
                 '''
                 CODE = 'pp_signal'
-                plot_name = 'PulsepairSignal'
                 plot_type = 'scatter'
                 buffering = False

schainpy/model/utils/jroutils_publish.py +60 -35

-            '''
+            # Copyright (c) 2012-2020 Jicamarca Radio Observatory
-            @author: Juan C. Espinoza
+            # All rights reserved.
-            '''
+            # Distributed under the terms of the BSD 3-clause license.
+            """Utilities for publish/send data, files & plots over different protocols
+            """
             import os
             import glob
             import time
             import json
             import numpy
             import zmq
             import datetime
             import ftplib
             from functools import wraps
             from threading import Thread
             from multiprocessing import Process
             from schainpy.model.proc.jroproc_base import Operation, ProcessingUnit, MPDecorator
             from schainpy.model.data.jrodata import JROData
             from schainpy.utils import log
-            MAXNUMX = 500
-            MAXNUMY = 500
             PLOT_CODES = {
                 'rti': 0,            # Range time intensity (RTI).
                 'spc': 1,            # Spectra (and Cross-spectra) information.
                 'cspc': 2,           # Cross-Correlation information.
                 'coh': 3,            # Coherence map.
                 'base': 4,           # Base lines graphic.
                 'row': 5,            # Row Spectra.
                 'total': 6,          # Total Power.
                 'drift': 7,          # Drifts graphics.
                 'height': 8,         # Height profile.
                 'phase': 9,          # Signal Phase.
                 'power': 16,
                 'noise': 17,
                 'beacon': 18,
                 'wind': 22,
                 'skymap': 23,
                 'Unknown': 24,
                 'V-E': 25,          # PIP Velocity.
                 'Z-E': 26,          # PIP Reflectivity.
                 'V-A': 27,          # RHI Velocity.
                 'Z-A': 28,          # RHI Reflectivity.
             }
             def get_plot_code(s):
                 label = s.split('_')[0]
                 codes = [key for key in PLOT_CODES if key in label]
                 if codes:
                     return PLOT_CODES[codes[0]]
                 else:
                     return 24
-            def decimate(z, MAXNUMY):
-                dy = int(len(z[0])/MAXNUMY) + 1
-                return z[::, ::dy]
             class PublishData(Operation):
                 '''
                 Operation to send data over zmq.
                 '''
                 __attrs__ = ['host', 'port', 'delay', 'verbose']
                 def setup(self, server='zmq.pipe', delay=0, verbose=True, **kwargs):
                     self.counter = 0
                     self.delay = kwargs.get('delay', 0)
                     self.cnt = 0
                     self.verbose = verbose
                     context = zmq.Context()
                     self.zmq_socket = context.socket(zmq.PUSH)
                     server = kwargs.get('server', 'zmq.pipe')
                     if 'tcp://' in server:
                         address = server
                     else:
                         address = 'ipc:///tmp/%s' % server
                     self.zmq_socket.connect(address)
                     time.sleep(1)
                 def publish_data(self):
                     self.dataOut.finished = False
                     if self.verbose:
                         log.log(
                             'Sending {} - {}'.format(self.dataOut.type, self.dataOut.datatime),
                             self.name
                         )
                     self.zmq_socket.send_pyobj(self.dataOut)
                 def run(self, dataOut, **kwargs):
                     self.dataOut = dataOut
                     if not self.isConfig:
                         self.setup(**kwargs)
                         self.isConfig = True
                     self.publish_data()
                     time.sleep(self.delay)
                 def close(self):
                     self.dataOut.finished = True
                     self.zmq_socket.send_pyobj(self.dataOut)
                     time.sleep(0.1)
                     self.zmq_socket.close()
             class ReceiverData(ProcessingUnit):
                 __attrs__ = ['server']
                 def __init__(self, **kwargs):
                     ProcessingUnit.__init__(self, **kwargs)
                     self.isConfig = False
                     server = kwargs.get('server', 'zmq.pipe')
                     if 'tcp://' in server:
                         address = server
                     else:
                         address = 'ipc:///tmp/%s' % server
                     self.address = address
                     self.dataOut = JROData()
                 def setup(self):
                     self.context = zmq.Context()
                     self.receiver = self.context.socket(zmq.PULL)
                     self.receiver.bind(self.address)
                     time.sleep(0.5)
                     log.success('ReceiverData from {}'.format(self.address))
                 def run(self):
                     if not self.isConfig:
                         self.setup()
                         self.isConfig = True
                     self.dataOut = self.receiver.recv_pyobj()
                     log.log('{} - {}'.format(self.dataOut.type,
                                              self.dataOut.datatime.ctime(),),
                             'Receiving')
             @MPDecorator
             class SendToFTP(Operation):
+                """Operation for send files over FTP
+                This operation is used to send files over FTP, you can send different files
+                from different folders by adding as many `pattern` as you wish.
+                Parameters:
+                -----------
+                server : str
+                    FTP server address.
+                username : str
+                    FTP username
+                password : str
+                    FTP password
+                timeout : int
+                    timeout to restart the connection
+                patternX : list
+                    detail of files to be send must have the following order: local, remote
+                    ext, period, exp_code, sub_exp_code
+                Example:
+                --------
+                ftp = proc_unit.addOperation(name='SendToFTP', optype='external')
+                ftp.addParameter(name='server', value='jro-app.igp.gob.pe')
+                ftp.addParameter(name='username', value='wmaster')
+                ftp.addParameter(name='password', value='mst2010vhf')
+                ftp.addParameter(
+                    name='pattern1',
+                    value='/local/path/rti,/remote/path,png,300,11,0'
+                    )
+                ftp.addParameter(
+                    name='pattern2',
+                    value='/local/path/spc,/remote/path,png,300,11,0'
+                    )
+                ftp.addParameter(
+                    name='pattern3',
+                    value='/local/path/param,/remote/path,hdf5,300,,'
+                    )
-                '''
+                """
-                Operation to send data over FTP.
-                patternX = 'local, remote, ext, period, exp_code, sub_exp_code'
-                '''
                 __attrs__ = ['server', 'username', 'password', 'timeout', 'patternX']
                 def __init__(self):
                     '''
                     '''
                     Operation.__init__(self)
                     self.ftp = None
                     self.ready = False
                 def setup(self, server, username, password, timeout, **kwargs):
                     '''
                     '''
                     self.server = server
                     self.username = username
                     self.password = password
                     self.timeout = timeout
                     self.patterns = []
                     self.times = []
                     self.latest = []
                     for arg, value in kwargs.items():
                         if 'pattern' in arg:
                             self.patterns.append(value)
-                            self.times.append(time.time())
+                            self.times.append(0)
                             self.latest.append('')
                 def connect(self):
                     '''
                     '''
                     log.log('Connecting to ftp://{}'.format(self.server), self.name)
                     try:
                         self.ftp = ftplib.FTP(self.server, timeout=self.timeout)
                     except ftplib.all_errors:
                         log.error('Server connection fail: {}'.format(self.server), self.name)
                         if self.ftp is not None:
                             self.ftp.close()
                         self.ftp = None
                         self.ready = False
                         return
                     try:
                         self.ftp.login(self.username, self.password)
                     except ftplib.all_errors:
                         log.error('The given username y/o password are incorrect', self.name)
                         if self.ftp is not None:
                             self.ftp.close()
                         self.ftp = None
                         self.ready = False
                         return
                     log.success('Connection success', self.name)
                     self.ready = True
                     return
                 def check(self):
                     try:
                         self.ftp.voidcmd("NOOP")
                     except:
                         log.warning('Connection lost... trying to reconnect', self.name)
                         if self.ftp is not None:
                             self.ftp.close()
                         self.ftp = None
                         self.connect()
                 def find_files(self, path, ext):
-                    files = glob.glob1(path, '*{}'.format(ext))
+                    files = glob.glob1(path.strip(), '*{}'.format(ext.strip()))
                     files.sort()
                     if files:
                         return files[-1]
                     return None
                 def getftpname(self, filename, exp_code, sub_exp_code):
                     thisDatetime = datetime.datetime.strptime(filename.split('_')[1], '%Y%m%d')
                     YEAR_STR = '%4.4d' % thisDatetime.timetuple().tm_year
                     DOY_STR = '%3.3d' % thisDatetime.timetuple().tm_yday
                     exp_code = '%3.3d' % exp_code
                     sub_exp_code = '%2.2d' % sub_exp_code
                     plot_code = '%2.2d' % get_plot_code(filename)
                     name = YEAR_STR + DOY_STR + '00' + exp_code + sub_exp_code + plot_code + '00.png'
                     return name
                 def upload(self, src, dst):
                     log.log('Uploading {} -> {} '.format(
                         src.split('/')[-1], dst.split('/')[-1]),
                         self.name,
                         nl=False
                         )
                     fp = open(src, 'rb')
                     command = 'STOR {}'.format(dst)
                     try:
                         self.ftp.storbinary(command, fp, blocksize=1024)
                     except Exception as e:
                         log.error('{}'.format(e), self.name)
-                        if self.ftp is not None:
-                            self.ftp.close()
-                        self.ftp = None
                         return 0
                     try:
                         self.ftp.sendcmd('SITE CHMOD 755 {}'.format(dst))
                     except Exception as e:
                         log.error('{}'.format(e), self.name)
-                        if self.ftp is not None:
-                            self.ftp.close()
-                        self.ftp = None
                         return 0
                     fp.close()
                     log.success('OK', tag='')
                     return 1
                 def send_files(self):
                     for x, pattern in enumerate(self.patterns):
                         local, remote, ext, period, exp_code, sub_exp_code = pattern
-                        if time.time()-self.times[x] >= int(period):
+                        if (self.dataOut.utctime - self.times[x]) < int(period):
+                            continue
                         srcname = self.find_files(local, ext)
-                            src = os.path.join(local, srcname)
-                            if os.path.getmtime(src) < time.time() - 30*60:
+                        if srcname is None:
-                                log.warning('Skipping old file {}'.format(srcname))
                             continue
-                            if srcname is None or srcname == self.latest[x]:
+                        if srcname == self.latest[x]:
                             log.warning('File alreday uploaded {}'.format(srcname))
                             continue
-                            if 'png' in ext:
+                        if exp_code.strip():
                             dstname = self.getftpname(srcname, int(exp_code), int(sub_exp_code))
                         else:
                             dstname = srcname
-                            dst = os.path.join(remote, dstname)
+                        src = os.path.join(local, srcname)
+                        dst = os.path.join(remote.strip(), dstname)
                         if self.upload(src, dst):
-                                self.times[x] = time.time()
+                            self.times[x] = self.dataOut.utctime
                             self.latest[x] = srcname
-                            else:
-                                self.ready = False
-                                break
                 def run(self, dataOut, server, username, password, timeout=10, **kwargs):
                     if not self.isConfig:
                         self.setup(
                             server=server,
                             username=username,
                             password=password,
                             timeout=timeout,
                             **kwargs
                             )
                         self.isConfig = True
-                    if not self.ready:
                         self.connect()
-                    if self.ftp is not None:
+                    self.dataOut = dataOut
                     self.check()
                     self.send_files()
                 def close(self):
                     if self.ftp is not None:
                         self.ftp.close()

General Comments 0

Write
Preview

You need to be logged in to leave comments. Login now

No TODOs yet

	Site-wide shortcuts
/	Use quick search box
g h	Goto home page
g g	Goto my private gists page
g G	Goto my public gists page
g 0-9	Goto bookmarked items from 0-9
n r	New repository page
n g	New gist page

	Repositories
g s	Goto summary page
g c	Goto changelog page
g f	Goto files page
g F	Goto files page with file search activated
g p	Goto pull requests page
g o	Goto repository settings
g O	Goto repository access permissions settings
t s	Toggle sidebar on some pages