##// END OF EJS Templates
Python 2to3, Spectra (all operations) working
Python 2to3, Spectra (all operations) working

File last commit:

r1167:1f521b07c958
r1167:1f521b07c958
Show More
jroutils_ftp.py
1007 lines | 26.6 KiB | text/x-python | PythonLexer
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 '''
@author: Daniel Suarez
'''
import os
import glob
import ftplib
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573
try:
import paramiko
import scp
except:
George Yong
Python 2to3, Spectra (all operations) working
r1167 print("You should install paramiko and scp libraries \nif you want to use SSH protocol to upload files to the server")
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573
import time
Miguel Valdez
SendToServer:...
r633 import threading
Thread = threading.Thread
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
SendToServer:...
r633 # try:
# from gevent import sleep
# except:
from time import sleep
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
Merge with branch schain_julia_drifts from rev. 803 to 995....
r568 from schainpy.model.proc.jroproc_base import ProcessingUnit, Operation
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488
Miguel Valdez
SendToServer:...
r633 class Remote(Thread):
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 """
Remote is a parent class used to define the behaviour of FTP and SSH class. These clases are
used to upload or download files remotely.
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 Non-standard Python modules used:
None
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 Written by:
José Chávez
setup actualizado para no necesitar numpy previamente
r1026 "Miguel Urco":mailto:miguel.urco@jro.igp.gob.pe Jun. 03, 2015
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 """
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 server = None
username = None
password = None
remotefolder = None
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 period = 60
fileList = []
bussy = False
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 def __init__(self, server, username, password, remotefolder, period=60):
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
SendToServer:...
r633 Thread.__init__(self)
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
Bugs fixed: Error selecting FTP parameters in Signal Chain GUI
r606 self.setDaemon(True)
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 self.status = 0
Miguel Valdez
jroutils_ftp.py: Remote Server is opened and closed each time a file is sent
r653 self.__server = server
self.__username = username
self.__password = password
self.__remotefolder = remotefolder
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 self.period = period
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 self.fileList = []
self.bussy = False
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 self.stopFlag = False
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
George Yong
Python 2to3, Spectra (all operations) working
r1167 print("[Remote Server] Opening server: %s" %self.__server)
Miguel Valdez
jroutils_ftp.py: Remote Server is opened and closed each time a file is sent
r653 if self.open(self.__server, self.__username, self.__password, self.__remotefolder):
George Yong
Python 2to3, Spectra (all operations) working
r1167 print("[Remote Server] %s server was opened successfully" %self.__server)
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
jroutils_ftp.py: Remote Server is opened and closed each time a file is sent
r653 self.close()
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
SendToServer:...
r633 self.mutex = threading.Lock()
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 def stop(self):
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 self.stopFlag = True
Miguel Valdez
SendToServer:...
r633 self.join(10)
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 def open(self):
"""
Connect to server and create a connection class (FTP or SSH) to remote server.
"""
George Yong
Python 2to3, Spectra (all operations) working
r1167 raise NotImplementedError("Implement this method in child class")
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 def close(self):
"""
Close connection to server
"""
George Yong
Python 2to3, Spectra (all operations) working
r1167 raise NotImplementedError("Implement this method in child class")
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 def mkdir(self, remotefolder):
"""
Create a folder remotely
"""
George Yong
Python 2to3, Spectra (all operations) working
r1167 raise NotImplementedError("Implement this method in child class")
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 def cd(self, remotefolder):
"""
Change working directory in remote server
"""
George Yong
Python 2to3, Spectra (all operations) working
r1167 raise NotImplementedError("Implement this method in child class")
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 def download(self, filename, localfolder=None):
"""
Download a file from server to local host
"""
George Yong
Python 2to3, Spectra (all operations) working
r1167 raise NotImplementedError("Implement this method in child class")
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 def sendFile(self, fullfilename):
"""
sendFile method is used to upload a local file to the current directory in remote server
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 Inputs:
fullfilename - full path name of local file to store in remote directory
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 Returns:
0 in error case else 1
"""
George Yong
Python 2to3, Spectra (all operations) working
r1167 raise NotImplementedError("Implement this method in child class")
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
def upload(self, fullfilename, remotefolder=None):
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 """
upload method is used to upload a local file to remote directory. This method changes
working directory before sending a file.
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 Inputs:
fullfilename - full path name of local file to store in remote directory
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
remotefolder - remote directory
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 Returns:
0 in error case else 1
"""
George Yong
Python 2to3, Spectra (all operations) working
r1167 print("[Remote Server] Uploading %s to %s:%s" %(fullfilename, self.server, self.remotefolder))
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 if not self.status:
return 0
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 if remotefolder == None:
remotefolder = self.remotefolder
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 if not self.cd(remotefolder):
return 0
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 if not self.sendFile(fullfilename):
George Yong
Python 2to3, Spectra (all operations) working
r1167 print("[Remote Server] Error uploading file %s" %fullfilename)
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 return 0
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
George Yong
Python 2to3, Spectra (all operations) working
r1167 print("[Remote Server] upload finished successfully")
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 return 1
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 def delete(self, filename):
"""
Remove a file from remote server
"""
pass
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 def updateFileList(self, fileList):
"""
Remove a file from remote server
"""
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 if fileList == self.fileList:
Miguel Valdez
Bugs fixed: Error selecting FTP parameters in Signal Chain GUI
r606 return 0
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
SendToServer:...
r633 self.mutex.acquire()
José Chávez
setup actualizado para no necesitar numpy previamente
r1026 # init = time.time()
#
# while(self.bussy):
# sleep(0.1)
# if time.time() - init > 2*self.period:
# return 0
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 self.fileList = fileList
Miguel Valdez
SendToServer:...
r633 self.mutex.release()
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 return 1
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 def run(self):
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
Signal Chain GUI updated:...
r580 if not self.status:
George Yong
Python 2to3, Spectra (all operations) working
r1167 print("Finishing FTP service")
Miguel Valdez
Signal Chain GUI updated:...
r580 return
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 if not self.cd(self.remotefolder):
George Yong
Python 2to3, Spectra (all operations) working
r1167 raise ValueError("Could not access to the new remote directory: %s" %self.remotefolder)
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 while True:
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
SendToServer:...
r633 for i in range(self.period):
if self.stopFlag:
break
sleep(1)
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
SendToServer:...
r633 if self.stopFlag:
break
José Chávez
merge con schain_mp
r1034 # self.bussy = True
Miguel Valdez
SendToServer:...
r633 self.mutex.acquire()
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
George Yong
Python 2to3, Spectra (all operations) working
r1167 print("[Remote Server] Opening %s" %self.__server)
Miguel Valdez
jroutils_ftp.py: Remote Server is opened and closed each time a file is sent
r653 if not self.open(self.__server, self.__username, self.__password, self.__remotefolder):
Miguel Valdez
RemoteServer: Closing semaphore
r661 self.mutex.release()
Miguel Valdez
r662 continue
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 for thisFile in self.fileList:
Miguel Valdez
Bug fixed: FTP Server thread stopped when an error sending file occurred
r646 self.upload(thisFile, self.remotefolder)
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
George Yong
Python 2to3, Spectra (all operations) working
r1167 print("[Remote Server] Closing %s" %self.__server)
Miguel Valdez
jroutils_ftp.py: Remote Server is opened and closed each time a file is sent
r653 self.close()
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
SendToServer:...
r633 self.mutex.release()
José Chávez
merge con schain_mp
r1034 # self.bussy = False
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
George Yong
Python 2to3, Spectra (all operations) working
r1167 print("[Remote Server] Thread stopped successfully")
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 class FTPClient(Remote):
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 __ftpClientObj = None
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 def __init__(self, server, username, password, remotefolder, period=60):
"""
"""
Remote.__init__(self, server, username, password, remotefolder, period)
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 def open(self, server, username, password, remotefolder):
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 """
This method is used to set FTP parameters and establish a connection to remote server
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 Inputs:
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897 server - remote server IP Address
username - remote server Username
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 password - remote server password
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 remotefolder - remote server current working directory
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
jroutils_ftp.py: Remote Server is opened and closed each time a file is sent
r653 Return:
Boolean - Returns 1 if a connection has been established, 0 otherwise
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Affects:
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 self.status - in case of error or fail connection this parameter is set to 0 else 1
"""
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 if server == None:
George Yong
Python 2to3, Spectra (all operations) working
r1167 raise ValueError("FTP server should be defined")
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 if username == None:
George Yong
Python 2to3, Spectra (all operations) working
r1167 raise ValueError("FTP username should be defined")
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 if password == None:
George Yong
Python 2to3, Spectra (all operations) working
r1167 raise ValueError("FTP password should be defined")
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 if remotefolder == None:
George Yong
Python 2to3, Spectra (all operations) working
r1167 raise ValueError("FTP remote folder should be defined")
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 try:
ftpClientObj = ftplib.FTP(server)
George Yong
Python 2to3, Spectra (all operations) working
r1167 except ftplib.all_errors as e:
print("[FTP Server]: FTP server connection fail: %s" %server)
print("[FTP Server]:", e)
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 self.status = 0
return 0
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 try:
ftpClientObj.login(username, password)
except ftplib.all_errors:
George Yong
Python 2to3, Spectra (all operations) working
r1167 print("[FTP Server]: FTP username or password are incorrect")
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 self.status = 0
return 0
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 if remotefolder == None:
remotefolder = ftpClientObj.pwd()
else:
try:
ftpClientObj.cwd(remotefolder)
except ftplib.all_errors:
George Yong
Python 2to3, Spectra (all operations) working
r1167 print("[FTP Server]: FTP remote folder is invalid: %s" %remotefolder)
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 remotefolder = ftpClientObj.pwd()
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 self.server = server
self.username = username
self.password = password
self.remotefolder = remotefolder
self.__ftpClientObj = ftpClientObj
self.status = 1
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 return 1
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 def close(self):
"""
Close connection to remote server
"""
if not self.status:
return 0
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 self.__ftpClientObj.close()
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 def mkdir(self, remotefolder):
"""
mkdir is used to make a new directory in remote server
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 Input:
remotefolder - directory name
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 Return:
0 in error case else 1
"""
if not self.status:
return 0
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 try:
self.__ftpClientObj.mkd(dirname)
except ftplib.all_errors:
George Yong
Python 2to3, Spectra (all operations) working
r1167 print("[FTP Server]: Error creating remote folder: %s" %remotefolder)
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 return 0
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 return 1
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 def cd(self, remotefolder):
"""
cd is used to change remote working directory on server
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 Input:
remotefolder - current working directory
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 Affects:
self.remotefolder
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Return:
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 0 in case of error else 1
"""
if not self.status:
return 0
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 if remotefolder == self.remotefolder:
return 1
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
try:
self.__ftpClientObj.cwd(remotefolder)
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 except ftplib.all_errors:
George Yong
Python 2to3, Spectra (all operations) working
r1167 print('[FTP Server]: Error changing to %s' %remotefolder)
print('[FTP Server]: Trying to create remote folder')
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 if not self.mkdir(remotefolder):
George Yong
Python 2to3, Spectra (all operations) working
r1167 print('[FTP Server]: Remote folder could not be created')
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 return 0
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
try:
self.__ftpClientObj.cwd(remotefolder)
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 except ftplib.all_errors:
return 0
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 self.remotefolder = remotefolder
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 return 1
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 def sendFile(self, fullfilename):
if not self.status:
return 0
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
jroutils_ftp.py: Remote Server is opened and closed each time a file is sent
r653 fp = open(fullfilename, 'rb')
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
Bugs fixed: Error selecting FTP parameters in Signal Chain GUI
r606 filename = os.path.basename(fullfilename)
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 command = "STOR %s" %filename
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 try:
Miguel Valdez
jroutils_ftp.py: Remote Server is opened and closed each time a file is sent
r653 self.__ftpClientObj.storbinary(command, fp)
George Yong
Python 2to3, Spectra (all operations) working
r1167 except ftplib.all_errors as e:
print("[FTP Server]:", e)
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 return 0
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 try:
self.__ftpClientObj.sendcmd('SITE CHMOD 755 ' + filename)
George Yong
Python 2to3, Spectra (all operations) working
r1167 except ftplib.all_errors as e:
print("[FTP Server]:", e)
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
jroutils_ftp.py: Remote Server is opened and closed each time a file is sent
r653 fp.close()
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 return 1
class SSHClient(Remote):
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 __sshClientObj = None
__scpClientObj = None
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 def __init__(self, server, username, password, remotefolder, period=60):
"""
"""
Remote.__init__(self, server, username, password, remotefolder, period)
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 def open(self, server, username, password, remotefolder, port=22):
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 """
José Chávez
setup actualizado para no necesitar numpy previamente
r1026 This method is used to set SSH parameters and establish a connection to a remote server
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573
José Chávez
setup actualizado para no necesitar numpy previamente
r1026 Inputs:
server - remote server IP Address
username - remote server Username
password - remote server password
remotefolder - remote server current working directory
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573
José Chávez
setup actualizado para no necesitar numpy previamente
r1026 Return: void
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573
José Chávez
setup actualizado para no necesitar numpy previamente
r1026 Affects:
self.status - in case of error or fail connection this parameter is set to 0 else 1
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573
"""
Miguel Valdez
jroutils_ftp.py: Remote Server is opened and closed each time a file is sent
r653 import socket
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 if server == None:
George Yong
Python 2to3, Spectra (all operations) working
r1167 raise ValueError("SSH server should be defined")
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 if username == None:
George Yong
Python 2to3, Spectra (all operations) working
r1167 raise ValueError("SSH username should be defined")
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 if password == None:
George Yong
Python 2to3, Spectra (all operations) working
r1167 raise ValueError("SSH password should be defined")
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 if remotefolder == None:
George Yong
Python 2to3, Spectra (all operations) working
r1167 raise ValueError("SSH remote folder should be defined")
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
jroutils_ftp.py: Remote Server is opened and closed each time a file is sent
r653 sshClientObj = paramiko.SSHClient()
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 sshClientObj.load_system_host_keys()
sshClientObj.set_missing_host_key_policy(paramiko.WarningPolicy())
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
jroutils_ftp.py: Remote Server is opened and closed each time a file is sent
r653 self.status = 0
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 try:
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897 sshClientObj.connect(server, username=username, password=password, port=port)
George Yong
Python 2to3, Spectra (all operations) working
r1167 except paramiko.AuthenticationException as e:
José Chávez
setup actualizado para no necesitar numpy previamente
r1026 # print "SSH username or password are incorrect: %s"
George Yong
Python 2to3, Spectra (all operations) working
r1167 print("[SSH Server]:", e)
Miguel Valdez
jroutils_ftp.py: Remote Server is opened and closed each time a file is sent
r653 return 0
George Yong
Python 2to3, Spectra (all operations) working
r1167 except SSHException as e:
print("[SSH Server]:", e)
Miguel Valdez
jroutils_ftp.py: Remote Server is opened and closed each time a file is sent
r653 return 0
except socket.error:
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 self.status = 0
George Yong
Python 2to3, Spectra (all operations) working
r1167 print("[SSH Server]:", e)
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 return 0
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
jroutils_ftp.py: Remote Server is opened and closed each time a file is sent
r653 self.status = 1
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 scpClientObj = scp.SCPClient(sshClientObj.get_transport(), socket_timeout=30)
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 if remotefolder == None:
remotefolder = self.pwd()
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 self.server = server
self.username = username
self.password = password
self.__sshClientObj = sshClientObj
self.__scpClientObj = scpClientObj
self.status = 1
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
jroutils_ftp.py: SSHClient changes file permissions on remote server after upload a file
r576 if not self.cd(remotefolder):
George Yong
Python 2to3, Spectra (all operations) working
r1167 raise ValueError("[SSH Server]: Could not access to remote folder: %s" %remotefolder)
Miguel Valdez
jroutils_ftp.py: SSHClient changes file permissions on remote server after upload a file
r576 return 0
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
jroutils_ftp.py: SSHClient changes file permissions on remote server after upload a file
r576 self.remotefolder = remotefolder
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 return 1
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 def close(self):
"""
José Chávez
setup actualizado para no necesitar numpy previamente
r1026 Close connection to remote server
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 """
if not self.status:
return 0
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
jroutils_ftp.py: Remote Server is opened and closed each time a file is sent
r653 self.__scpClientObj.close()
self.__sshClientObj.close()
Miguel Valdez
jroutils_ftp.py: SSHClient changes file permissions on remote server after upload a file
r576
def __execute(self, command):
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 """
José Chávez
setup actualizado para no necesitar numpy previamente
r1026 __execute a command on remote server
Input:
command - Exmaple 'ls -l'
Return:
0 in error case else 1
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 """
if not self.status:
return 0
Miguel Valdez
jroutils_ftp.py: SSHClient changes file permissions on remote server after upload a file
r576
stdin, stdout, stderr = self.__sshClientObj.exec_command(command)
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
jroutils_ftp.py: SSHClient changes file permissions on remote server after upload a file
r576 result = stderr.readlines()
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 if len(result) > 1:
return 0
Miguel Valdez
jroutils_ftp.py: SSHClient changes file permissions on remote server after upload a file
r576
result = stdout.readlines()
if len(result) > 1:
return result[0][:-1]
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 return 1
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
jroutils_ftp.py: SSHClient changes file permissions on remote server after upload a file
r576 def mkdir(self, remotefolder):
"""
José Chávez
setup actualizado para no necesitar numpy previamente
r1026 mkdir is used to make a new directory in remote server
Input:
remotefolder - directory name
Return:
0 in error case else 1
Miguel Valdez
jroutils_ftp.py: SSHClient changes file permissions on remote server after upload a file
r576 """
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
jroutils_ftp.py: SSHClient changes file permissions on remote server after upload a file
r576 command = 'mkdir %s' %remotefolder
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
jroutils_ftp.py: SSHClient changes file permissions on remote server after upload a file
r576 return self.__execute(command)
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 def pwd(self):
Miguel Valdez
jroutils_ftp.py: SSHClient changes file permissions on remote server after upload a file
r576 command = 'pwd'
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
jroutils_ftp.py: SSHClient changes file permissions on remote server after upload a file
r576 return self.__execute(command)
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 def cd(self, remotefolder):
"""
José Chávez
setup actualizado para no necesitar numpy previamente
r1026 cd is used to change remote working directory on server
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573
José Chávez
setup actualizado para no necesitar numpy previamente
r1026 Input:
remotefolder - current working directory
Affects:
self.remotefolder
Return:
0 in case of error else 1
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 """
if not self.status:
return 0
Miguel Valdez
jroutils_ftp.py: SSHClient changes file permissions on remote server after upload a file
r576
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 if remotefolder == self.remotefolder:
return 1
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
jroutils_ftp.py: SSHClient changes file permissions on remote server after upload a file
r576 chk_command = "cd %s; pwd" %remotefolder
mkdir_command = "mkdir %s" %remotefolder
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
jroutils_ftp.py: SSHClient changes file permissions on remote server after upload a file
r576 if not self.__execute(chk_command):
if not self.__execute(mkdir_command):
self.remotefolder = None
return 0
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 self.remotefolder = remotefolder
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 return 1
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 def sendFile(self, fullfilename):
if not self.status:
return 0
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 try:
self.__scpClientObj.put(fullfilename, remote_path=self.remotefolder)
George Yong
Python 2to3, Spectra (all operations) working
r1167 except scp.ScpError as e:
print("[SSH Server]", str(e))
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 return 0
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
jroutils_ftp.py: SSHClient changes file permissions on remote server after upload a file
r576 remotefile = os.path.join(self.remotefolder, os.path.split(fullfilename)[-1])
command = 'chmod 775 %s' %remotefile
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
jroutils_ftp.py: SSHClient changes file permissions on remote server after upload a file
r576 return self.__execute(command)
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 class SendToServer(ProcessingUnit):
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
def __init__(self, **kwargs):
ProcessingUnit.__init__(self, **kwargs)
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 self.isConfig = False
José Chávez
setup actualizado para no necesitar numpy previamente
r1026 self.clientObj = None
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573
Miguel Valdez
Bug fixed in SignalChain GUI:...
r581 def setup(self, server, username, password, remotefolder, localfolder, ext='.png', period=60, protocol='ftp', **kwargs):
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 self.clientObj = None
self.localfolder = localfolder
self.ext = ext
self.period = period
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 if str.lower(protocol) == 'ftp':
self.clientObj = FTPClient(server, username, password, remotefolder, period)
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 if str.lower(protocol) == 'ssh':
self.clientObj = SSHClient(server, username, password, remotefolder, period)
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 if not self.clientObj:
George Yong
Python 2to3, Spectra (all operations) working
r1167 raise ValueError("%s has been chosen as remote access protocol but it is not valid" %protocol)
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 self.clientObj.start()
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 def findFiles(self):
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
Bugs fixed: Error selecting FTP parameters in Signal Chain GUI
r606 if not type(self.localfolder) == list:
folderList = [self.localfolder]
else:
folderList = self.localfolder
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
SendToServer:...
r633 #Remove duplicate items
folderList = list(set(folderList))
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
Bugs fixed: Error selecting FTP parameters in Signal Chain GUI
r606 fullfilenameList = []
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
Bugs fixed: Error selecting FTP parameters in Signal Chain GUI
r606 for thisFolder in folderList:
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
George Yong
Python 2to3, Spectra (all operations) working
r1167 print("[Remote Server]: Searching files on %s" %thisFolder)
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
Bugs fixed: Error selecting FTP parameters in Signal Chain GUI
r606 filenameList = glob.glob1(thisFolder, '*%s' %self.ext)
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
Bugs fixed: Error selecting FTP parameters in Signal Chain GUI
r606 if len(filenameList) < 1:
José Chávez
cambios para online offline
r1019
Miguel Valdez
Bugs fixed: Error selecting FTP parameters in Signal Chain GUI
r606 continue
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
Bugs fixed: Error selecting FTP parameters in Signal Chain GUI
r606 for thisFile in filenameList:
fullfilename = os.path.join(thisFolder, thisFile)
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
SendToServer:...
r633 if fullfilename in fullfilenameList:
continue
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
SendToServer:...
r633 #Only files modified in the last 30 minutes are considered
if os.path.getmtime(fullfilename) < time.time() - 30*60:
continue
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
Bugs fixed: Error selecting FTP parameters in Signal Chain GUI
r606 fullfilenameList.append(fullfilename)
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 return fullfilenameList
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 def run(self, **kwargs):
if not self.isConfig:
self.init = time.time()
self.setup(**kwargs)
self.isConfig = True
José Chávez
this fixes bug #1074 and bug #1101
r1011 if not self.clientObj.is_alive():
George Yong
Python 2to3, Spectra (all operations) working
r1167 print("[Remote Server]: Restarting connection ")
José Chávez
this fixes bug #1074 and bug #1101
r1011 self.setup(**kwargs)
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 if time.time() - self.init >= self.period:
fullfilenameList = self.findFiles()
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
Bugs fixed: Error selecting FTP parameters in Signal Chain GUI
r606 if self.clientObj.updateFileList(fullfilenameList):
George Yong
Python 2to3, Spectra (all operations) working
r1167 print("[Remote Server]: Sending the next files ", str(fullfilenameList))
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 self.init = time.time()
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 def close(self):
George Yong
Python 2to3, Spectra (all operations) working
r1167 print("[Remote Server] Stopping thread")
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 self.clientObj.stop()
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Daniel Valdez
This update consider connect and send FTP using multiprocessing with timeout
r517 class FTP(object):
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 """
Ftp is a public class used to define custom File Transfer Protocol from "ftplib" python module
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 Non-standard Python modules used: None
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 Written by "Daniel Suarez":mailto:daniel.suarez@jro.igp.gob.pe Oct. 26, 2010
"""
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
def __init__(self,server = None, username=None, password=None, remotefolder=None):
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 """
This method is used to setting parameters for FTP and establishing connection to remote server
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 Inputs:
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897 server - remote server IP Address
username - remote server Username
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 password - remote server password
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 remotefolder - remote server current working directory
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 Return: void
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Affects:
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 self.status - in Error Case or Connection Failed this parameter is set to 1 else 0
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 self.folderList - sub-folder list of remote folder
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 self.fileList - file list of remote folder
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 """
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 if ((server == None) and (username==None) and (password==None) and (remotefolder==None)):
server, username, password, remotefolder = self.parmsByDefault()
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 self.server = server
self.username = username
self.password = password
self.remotefolder = remotefolder
self.file = None
self.ftp = None
self.status = 0
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 try:
self.ftp = ftplib.FTP(self.server)
self.ftp.login(self.username,self.password)
self.ftp.cwd(self.remotefolder)
José Chávez
merge con schain_mp
r1034 # print 'Connect to FTP Server: Successfully'
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488
except ftplib.all_errors:
George Yong
Python 2to3, Spectra (all operations) working
r1167 print('Error FTP Service')
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 self.status = 1
return
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 self.dirList = []
try:
self.dirList = self.ftp.nlst()
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
George Yong
Python 2to3, Spectra (all operations) working
r1167 except ftplib.error_perm as resp:
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 if str(resp) == "550 No files found":
George Yong
Python 2to3, Spectra (all operations) working
r1167 print("no files in this directory")
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 self.status = 1
return
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 except ftplib.all_errors:
George Yong
Python 2to3, Spectra (all operations) working
r1167 print('Error Displaying Dir-Files')
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 self.status = 1
return
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 self.fileList = []
self.folderList = []
#only for test
for f in self.dirList:
name, ext = os.path.splitext(f)
if ext != '':
self.fileList.append(f)
José Chávez
setup actualizado para no necesitar numpy previamente
r1026 # print 'filename: %s - size: %d'%(f,self.ftp.size(f))
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488
def parmsByDefault(self):
server = 'jro-app.igp.gob.pe'
username = 'wmaster'
password = 'mst2010vhf'
remotefolder = '/home/wmaster/graficos'
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 return server, username, password, remotefolder
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 def mkd(self,dirname):
"""
mkd is used to make directory in remote server
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 Input:
dirname - directory name
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 Return:
1 in error case else 0
"""
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897 try:
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 self.ftp.mkd(dirname)
except:
George Yong
Python 2to3, Spectra (all operations) working
r1167 print('Error creating remote folder:%s'%dirname)
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 return 1
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 return 0
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 def delete(self,filename):
"""
delete is used to delete file in current working directory of remote server
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 Input:
filename - filename to delete in remote folder
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 Return:
1 in error case else 0
"""
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 try:
self.ftp.delete(filename)
except:
George Yong
Python 2to3, Spectra (all operations) working
r1167 print('Error deleting remote file:%s'%filename)
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 return 1
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 return 0
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 def download(self,filename,localfolder):
"""
download is used to downloading file from remote folder into local folder
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 Inputs:
filename - filename to donwload
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 localfolder - directory local to store filename
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 Returns:
self.status - 1 in error case else 0
"""
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 self.status = 0
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 if not(filename in self.fileList):
George Yong
Python 2to3, Spectra (all operations) working
r1167 print('filename:%s not exists'%filename)
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 self.status = 1
return self.status
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 newfilename = os.path.join(localfolder,filename)
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
self.file = open(newfilename, 'wb')
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 try:
George Yong
Python 2to3, Spectra (all operations) working
r1167 print('Download: ' + filename)
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 self.ftp.retrbinary('RETR ' + filename, self.__handleDownload)
George Yong
Python 2to3, Spectra (all operations) working
r1167 print('Download Complete')
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 except ftplib.all_errors:
George Yong
Python 2to3, Spectra (all operations) working
r1167 print('Error Downloading ' + filename)
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 self.status = 1
return self.status
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 self.file.close()
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
return self.status
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488
def __handleDownload(self,block):
"""
__handleDownload is used to handle writing file
"""
self.file.write(block)
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 def upload(self,filename,remotefolder=None):
"""
upload is used to uploading local file to remote directory
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 Inputs:
filename - full path name of local file to store in remote directory
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
remotefolder - remote directory
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 Returns:
self.status - 1 in error case else 0
"""
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 if remotefolder == None:
remotefolder = self.remotefolder
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 self.status = 0
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 try:
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897 self.ftp.cwd(remotefolder)
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 self.file = open(filename, 'rb')
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 (head, tail) = os.path.split(filename)
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 command = "STOR " + tail
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
George Yong
Python 2to3, Spectra (all operations) working
r1167 print('Uploading: ' + tail)
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897 self.ftp.storbinary(command, self.file)
George Yong
Python 2to3, Spectra (all operations) working
r1167 print('Upload Completed')
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 except ftplib.all_errors:
George Yong
Python 2to3, Spectra (all operations) working
r1167 print('Error Uploading ' + tail)
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 self.status = 1
return self.status
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 self.file.close()
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 #back to initial directory in __init__()
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897 self.ftp.cwd(self.remotefolder)
return self.status
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 def dir(self,remotefolder):
"""
dir is used to change working directory of remote server and get folder and file list
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 Input:
remotefolder - current working directory
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 Affects:
self.fileList - file list of working directory
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Return:
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 infoList - list with filenames and size of file in bytes
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 self.folderList - folder list
"""
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 self.remotefolder = remotefolder
George Yong
Python 2to3, Spectra (all operations) working
r1167 print('Change to ' + self.remotefolder)
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897 try:
self.ftp.cwd(remotefolder)
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 except ftplib.all_errors:
George Yong
Python 2to3, Spectra (all operations) working
r1167 print('Error Change to ' + self.remotefolder)
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 infoList = None
self.folderList = None
return infoList,self.folderList
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 self.dirList = []
try:
self.dirList = self.ftp.nlst()
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
George Yong
Python 2to3, Spectra (all operations) working
r1167 except ftplib.error_perm as resp:
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897 if str(resp) == "550 No files found":
George Yong
Python 2to3, Spectra (all operations) working
r1167 print("no files in this directory")
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 infoList = None
self.folderList = None
return infoList,self.folderList
except ftplib.all_errors:
George Yong
Python 2to3, Spectra (all operations) working
r1167 print('Error Displaying Dir-Files')
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 infoList = None
self.folderList = None
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897 return infoList,self.folderList
infoList = []
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 self.fileList = []
self.folderList = []
for f in self.dirList:
name,ext = os.path.splitext(f)
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897 if ext != '':
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 self.fileList.append(f)
value = (f,self.ftp.size(f))
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897 infoList.append(value)
if ext == '':
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 self.folderList.append(f)
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 return infoList,self.folderList
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 def close(self):
"""
close is used to close and end FTP connection
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 Inputs: None
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 Return: void
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 """
self.ftp.close()
class SendByFTP(Operation):
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
def __init__(self, **kwargs):
Operation.__init__(self, **kwargs)
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 self.status = 1
Daniel Valdez
setting a period to send by FTP
r489 self.counter = 0
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 def error_print(self, ValueError):
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
George Yong
Python 2to3, Spectra (all operations) working
r1167 print(ValueError, 'Error FTP')
print("don't worry the program is running...")
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Daniel Valdez
This update consider connect and send FTP using multiprocessing with timeout
r517 def worker_ftp(self, server, username, password, remotefolder, filenameList):
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 self.ftpClientObj = FTP(server, username, password, remotefolder)
Daniel Valdez
This update consider connect and send FTP using multiprocessing with timeout
r517 for filename in filenameList:
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 self.ftpClientObj.upload(filename)
self.ftpClientObj.close()
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Daniel Valdez
This update consider connect and send FTP using multiprocessing with timeout
r517 def ftp_thread(self, server, username, password, remotefolder):
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 if not(self.status):
return
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
SendToServer:...
r633 import multiprocessing
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Daniel Valdez
This update consider connect and send FTP using multiprocessing with timeout
r517 p = multiprocessing.Process(target=self.worker_ftp, args=(server, username, password, remotefolder, self.filenameList,))
p.start()
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Daniel Valdez
This update consider connect and send FTP using multiprocessing with timeout
r517 p.join(3)
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Daniel Valdez
This update consider connect and send FTP using multiprocessing with timeout
r517 if p.is_alive():
p.terminate()
p.join()
George Yong
Python 2to3, Spectra (all operations) working
r1167 print('killing ftp process...')
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 self.status = 0
return
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Daniel Valdez
This update consider connect and send FTP using multiprocessing with timeout
r517 self.status = 1
return
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 def filterByExt(self, ext, localfolder):
fnameList = glob.glob1(localfolder,ext)
self.filenameList = [os.path.join(localfolder,x) for x in fnameList]
if len(self.filenameList) == 0:
self.status = 0
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Daniel Valdez
setting a period to send by FTP
r489 def run(self, dataOut, ext, localfolder, remotefolder, server, username, password, period=1):
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
self.counter += 1
Daniel Valdez
setting a period to send by FTP
r489 if self.counter >= period:
self.filterByExt(ext, localfolder)
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Daniel Valdez
This update consider connect and send FTP using multiprocessing with timeout
r517 self.ftp_thread(server, username, password, remotefolder)
Daniel Valdez
setting a period to send by FTP
r489 self.counter = 0
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
George Yong
Python 2to3, Spectra (all operations) working
r1167 self.status = 1