##// END OF EJS Templates
Hot fix
Hot fix

File last commit:

r1037:2fe9a199ea60
r1166:ce0987c48654
Show More
jroutils_ftp.py
1008 lines | 26.5 KiB | text/x-python | PythonLexer
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 '''
@author: Daniel Suarez
'''
import os
import glob
import ftplib
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573
try:
import paramiko
import scp
except:
Miguel Valdez
minor changes and comments added
r641 print "You should install paramiko and scp libraries \nif you want to use SSH protocol to upload files to the server"
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573
import time
Miguel Valdez
SendToServer:...
r633 import threading
Thread = threading.Thread
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
SendToServer:...
r633 # try:
# from gevent import sleep
# except:
from time import sleep
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
Merge with branch schain_julia_drifts from rev. 803 to 995....
r568 from schainpy.model.proc.jroproc_base import ProcessingUnit, Operation
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488
Miguel Valdez
SendToServer:...
r633 class Remote(Thread):
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 """
Remote is a parent class used to define the behaviour of FTP and SSH class. These clases are
used to upload or download files remotely.
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 Non-standard Python modules used:
None
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 Written by:
José Chávez
setup actualizado para no necesitar numpy previamente
r1026 "Miguel Urco":mailto:miguel.urco@jro.igp.gob.pe Jun. 03, 2015
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 """
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 server = None
username = None
password = None
remotefolder = None
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 period = 60
fileList = []
bussy = False
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 def __init__(self, server, username, password, remotefolder, period=60):
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
SendToServer:...
r633 Thread.__init__(self)
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
Bugs fixed: Error selecting FTP parameters in Signal Chain GUI
r606 self.setDaemon(True)
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 self.status = 0
Miguel Valdez
jroutils_ftp.py: Remote Server is opened and closed each time a file is sent
r653 self.__server = server
self.__username = username
self.__password = password
self.__remotefolder = remotefolder
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 self.period = period
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 self.fileList = []
self.bussy = False
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 self.stopFlag = False
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
jroutils_ftp.py: Remote Server is opened and closed each time a file is sent
r653 print "[Remote Server] Opening server: %s" %self.__server
if self.open(self.__server, self.__username, self.__password, self.__remotefolder):
print "[Remote Server] %s server was opened successfully" %self.__server
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
jroutils_ftp.py: Remote Server is opened and closed each time a file is sent
r653 self.close()
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
SendToServer:...
r633 self.mutex = threading.Lock()
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 def stop(self):
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 self.stopFlag = True
Miguel Valdez
SendToServer:...
r633 self.join(10)
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 def open(self):
"""
Connect to server and create a connection class (FTP or SSH) to remote server.
"""
raise NotImplementedError, "Implement this method in child class"
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 def close(self):
"""
Close connection to server
"""
raise NotImplementedError, "Implement this method in child class"
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 def mkdir(self, remotefolder):
"""
Create a folder remotely
"""
raise NotImplementedError, "Implement this method in child class"
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 def cd(self, remotefolder):
"""
Change working directory in remote server
"""
raise NotImplementedError, "Implement this method in child class"
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 def download(self, filename, localfolder=None):
"""
Download a file from server to local host
"""
raise NotImplementedError, "Implement this method in child class"
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 def sendFile(self, fullfilename):
"""
sendFile method is used to upload a local file to the current directory in remote server
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 Inputs:
fullfilename - full path name of local file to store in remote directory
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 Returns:
0 in error case else 1
"""
raise NotImplementedError, "Implement this method in child class"
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
def upload(self, fullfilename, remotefolder=None):
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 """
upload method is used to upload a local file to remote directory. This method changes
working directory before sending a file.
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 Inputs:
fullfilename - full path name of local file to store in remote directory
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
remotefolder - remote directory
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 Returns:
0 in error case else 1
"""
print "[Remote Server] Uploading %s to %s:%s" %(fullfilename, self.server, self.remotefolder)
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 if not self.status:
return 0
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 if remotefolder == None:
remotefolder = self.remotefolder
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 if not self.cd(remotefolder):
return 0
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 if not self.sendFile(fullfilename):
print "[Remote Server] Error uploading file %s" %fullfilename
return 0
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 print "[Remote Server] upload finished successfully"
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 return 1
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 def delete(self, filename):
"""
Remove a file from remote server
"""
pass
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 def updateFileList(self, fileList):
"""
Remove a file from remote server
"""
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 if fileList == self.fileList:
Miguel Valdez
Bugs fixed: Error selecting FTP parameters in Signal Chain GUI
r606 return 0
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
SendToServer:...
r633 self.mutex.acquire()
José Chávez
setup actualizado para no necesitar numpy previamente
r1026 # init = time.time()
#
# while(self.bussy):
# sleep(0.1)
# if time.time() - init > 2*self.period:
# return 0
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 self.fileList = fileList
Miguel Valdez
SendToServer:...
r633 self.mutex.release()
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 return 1
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 def run(self):
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
Signal Chain GUI updated:...
r580 if not self.status:
print "Finishing FTP service"
return
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 if not self.cd(self.remotefolder):
Miguel Valdez
jroutils_ftp.py: SSHClient changes file permissions on remote server after upload a file
r576 raise ValueError, "Could not access to the new remote directory: %s" %self.remotefolder
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 while True:
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
SendToServer:...
r633 for i in range(self.period):
if self.stopFlag:
break
sleep(1)
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
SendToServer:...
r633 if self.stopFlag:
break
José Chávez
merge con schain_mp
r1034 # self.bussy = True
Miguel Valdez
SendToServer:...
r633 self.mutex.acquire()
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
jroutils_ftp.py: Remote Server is opened and closed each time a file is sent
r653 print "[Remote Server] Opening %s" %self.__server
if not self.open(self.__server, self.__username, self.__password, self.__remotefolder):
Miguel Valdez
RemoteServer: Closing semaphore
r661 self.mutex.release()
Miguel Valdez
r662 continue
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 for thisFile in self.fileList:
Miguel Valdez
Bug fixed: FTP Server thread stopped when an error sending file occurred
r646 self.upload(thisFile, self.remotefolder)
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
jroutils_ftp.py: Remote Server is opened and closed each time a file is sent
r653 print "[Remote Server] Closing %s" %self.__server
self.close()
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
SendToServer:...
r633 self.mutex.release()
José Chávez
merge con schain_mp
r1034 # self.bussy = False
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 print "[Remote Server] Thread stopped successfully"
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 class FTPClient(Remote):
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 __ftpClientObj = None
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 def __init__(self, server, username, password, remotefolder, period=60):
"""
"""
Remote.__init__(self, server, username, password, remotefolder, period)
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 def open(self, server, username, password, remotefolder):
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 """
This method is used to set FTP parameters and establish a connection to remote server
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 Inputs:
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897 server - remote server IP Address
username - remote server Username
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 password - remote server password
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 remotefolder - remote server current working directory
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
jroutils_ftp.py: Remote Server is opened and closed each time a file is sent
r653 Return:
Boolean - Returns 1 if a connection has been established, 0 otherwise
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Affects:
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 self.status - in case of error or fail connection this parameter is set to 0 else 1
"""
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 if server == None:
raise ValueError, "FTP server should be defined"
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 if username == None:
raise ValueError, "FTP username should be defined"
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 if password == None:
raise ValueError, "FTP password should be defined"
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 if remotefolder == None:
raise ValueError, "FTP remote folder should be defined"
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 try:
ftpClientObj = ftplib.FTP(server)
Miguel Valdez
jroutils_ftp.py: Remote Server is opened and closed each time a file is sent
r653 except ftplib.all_errors, e:
print "[FTP Server]: FTP server connection fail: %s" %server
print "[FTP Server]:", e
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 self.status = 0
return 0
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 try:
ftpClientObj.login(username, password)
except ftplib.all_errors:
Miguel Valdez
jroutils_ftp.py: Remote Server is opened and closed each time a file is sent
r653 print "[FTP Server]: FTP username or password are incorrect"
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 self.status = 0
return 0
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 if remotefolder == None:
remotefolder = ftpClientObj.pwd()
else:
try:
ftpClientObj.cwd(remotefolder)
except ftplib.all_errors:
Miguel Valdez
jroutils_ftp.py: Remote Server is opened and closed each time a file is sent
r653 print "[FTP Server]: FTP remote folder is invalid: %s" %remotefolder
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 remotefolder = ftpClientObj.pwd()
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 self.server = server
self.username = username
self.password = password
self.remotefolder = remotefolder
self.__ftpClientObj = ftpClientObj
self.status = 1
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 return 1
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 def close(self):
"""
Close connection to remote server
"""
if not self.status:
return 0
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 self.__ftpClientObj.close()
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 def mkdir(self, remotefolder):
"""
mkdir is used to make a new directory in remote server
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 Input:
remotefolder - directory name
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 Return:
0 in error case else 1
"""
if not self.status:
return 0
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 try:
self.__ftpClientObj.mkd(dirname)
except ftplib.all_errors:
Miguel Valdez
jroutils_ftp.py: Remote Server is opened and closed each time a file is sent
r653 print "[FTP Server]: Error creating remote folder: %s" %remotefolder
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 return 0
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 return 1
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 def cd(self, remotefolder):
"""
cd is used to change remote working directory on server
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 Input:
remotefolder - current working directory
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 Affects:
self.remotefolder
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Return:
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 0 in case of error else 1
"""
if not self.status:
return 0
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 if remotefolder == self.remotefolder:
return 1
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
try:
self.__ftpClientObj.cwd(remotefolder)
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 except ftplib.all_errors:
Miguel Valdez
jroutils_ftp.py: Remote Server is opened and closed each time a file is sent
r653 print '[FTP Server]: Error changing to %s' %remotefolder
print '[FTP Server]: Trying to create remote folder'
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 if not self.mkdir(remotefolder):
Miguel Valdez
jroutils_ftp.py: Remote Server is opened and closed each time a file is sent
r653 print '[FTP Server]: Remote folder could not be created'
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 return 0
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
try:
self.__ftpClientObj.cwd(remotefolder)
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 except ftplib.all_errors:
return 0
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 self.remotefolder = remotefolder
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 return 1
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 def sendFile(self, fullfilename):
if not self.status:
return 0
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
jroutils_ftp.py: Remote Server is opened and closed each time a file is sent
r653 fp = open(fullfilename, 'rb')
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
Bugs fixed: Error selecting FTP parameters in Signal Chain GUI
r606 filename = os.path.basename(fullfilename)
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 command = "STOR %s" %filename
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 try:
Miguel Valdez
jroutils_ftp.py: Remote Server is opened and closed each time a file is sent
r653 self.__ftpClientObj.storbinary(command, fp)
except ftplib.all_errors, e:
print "[FTP Server]:", e
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 return 0
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 try:
self.__ftpClientObj.sendcmd('SITE CHMOD 755 ' + filename)
except ftplib.all_errors, e:
Miguel Valdez
jroutils_ftp.py: Remote Server is opened and closed each time a file is sent
r653 print "[FTP Server]:", e
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
jroutils_ftp.py: Remote Server is opened and closed each time a file is sent
r653 fp.close()
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 return 1
class SSHClient(Remote):
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 __sshClientObj = None
__scpClientObj = None
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 def __init__(self, server, username, password, remotefolder, period=60):
"""
"""
Remote.__init__(self, server, username, password, remotefolder, period)
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 def open(self, server, username, password, remotefolder, port=22):
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 """
José Chávez
setup actualizado para no necesitar numpy previamente
r1026 This method is used to set SSH parameters and establish a connection to a remote server
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573
José Chávez
setup actualizado para no necesitar numpy previamente
r1026 Inputs:
server - remote server IP Address
username - remote server Username
password - remote server password
remotefolder - remote server current working directory
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573
José Chávez
setup actualizado para no necesitar numpy previamente
r1026 Return: void
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573
José Chávez
setup actualizado para no necesitar numpy previamente
r1026 Affects:
self.status - in case of error or fail connection this parameter is set to 0 else 1
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573
"""
Miguel Valdez
jroutils_ftp.py: Remote Server is opened and closed each time a file is sent
r653 import socket
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 if server == None:
raise ValueError, "SSH server should be defined"
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 if username == None:
raise ValueError, "SSH username should be defined"
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 if password == None:
raise ValueError, "SSH password should be defined"
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 if remotefolder == None:
raise ValueError, "SSH remote folder should be defined"
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
jroutils_ftp.py: Remote Server is opened and closed each time a file is sent
r653 sshClientObj = paramiko.SSHClient()
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 sshClientObj.load_system_host_keys()
sshClientObj.set_missing_host_key_policy(paramiko.WarningPolicy())
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
jroutils_ftp.py: Remote Server is opened and closed each time a file is sent
r653 self.status = 0
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 try:
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897 sshClientObj.connect(server, username=username, password=password, port=port)
Miguel Valdez
jroutils_ftp.py: Remote Server is opened and closed each time a file is sent
r653 except paramiko.AuthenticationException, e:
José Chávez
setup actualizado para no necesitar numpy previamente
r1026 # print "SSH username or password are incorrect: %s"
Miguel Valdez
jroutils_ftp.py: Remote Server is opened and closed each time a file is sent
r653 print "[SSH Server]:", e
return 0
except SSHException, e:
print "[SSH Server]:", e
return 0
except socket.error:
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 self.status = 0
Miguel Valdez
jroutils_ftp.py: Remote Server is opened and closed each time a file is sent
r653 print "[SSH Server]:", e
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 return 0
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
jroutils_ftp.py: Remote Server is opened and closed each time a file is sent
r653 self.status = 1
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 scpClientObj = scp.SCPClient(sshClientObj.get_transport(), socket_timeout=30)
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 if remotefolder == None:
remotefolder = self.pwd()
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 self.server = server
self.username = username
self.password = password
self.__sshClientObj = sshClientObj
self.__scpClientObj = scpClientObj
self.status = 1
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
jroutils_ftp.py: SSHClient changes file permissions on remote server after upload a file
r576 if not self.cd(remotefolder):
Miguel Valdez
jroutils_ftp.py: Remote Server is opened and closed each time a file is sent
r653 raise ValueError, "[SSH Server]: Could not access to remote folder: %s" %remotefolder
Miguel Valdez
jroutils_ftp.py: SSHClient changes file permissions on remote server after upload a file
r576 return 0
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
jroutils_ftp.py: SSHClient changes file permissions on remote server after upload a file
r576 self.remotefolder = remotefolder
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 return 1
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 def close(self):
"""
José Chávez
setup actualizado para no necesitar numpy previamente
r1026 Close connection to remote server
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 """
if not self.status:
return 0
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
jroutils_ftp.py: Remote Server is opened and closed each time a file is sent
r653 self.__scpClientObj.close()
self.__sshClientObj.close()
Miguel Valdez
jroutils_ftp.py: SSHClient changes file permissions on remote server after upload a file
r576
def __execute(self, command):
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 """
José Chávez
setup actualizado para no necesitar numpy previamente
r1026 __execute a command on remote server
Input:
command - Exmaple 'ls -l'
Return:
0 in error case else 1
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 """
if not self.status:
return 0
Miguel Valdez
jroutils_ftp.py: SSHClient changes file permissions on remote server after upload a file
r576
stdin, stdout, stderr = self.__sshClientObj.exec_command(command)
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
jroutils_ftp.py: SSHClient changes file permissions on remote server after upload a file
r576 result = stderr.readlines()
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 if len(result) > 1:
return 0
Miguel Valdez
jroutils_ftp.py: SSHClient changes file permissions on remote server after upload a file
r576
result = stdout.readlines()
if len(result) > 1:
return result[0][:-1]
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 return 1
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
jroutils_ftp.py: SSHClient changes file permissions on remote server after upload a file
r576 def mkdir(self, remotefolder):
"""
José Chávez
setup actualizado para no necesitar numpy previamente
r1026 mkdir is used to make a new directory in remote server
Input:
remotefolder - directory name
Return:
0 in error case else 1
Miguel Valdez
jroutils_ftp.py: SSHClient changes file permissions on remote server after upload a file
r576 """
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
jroutils_ftp.py: SSHClient changes file permissions on remote server after upload a file
r576 command = 'mkdir %s' %remotefolder
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
jroutils_ftp.py: SSHClient changes file permissions on remote server after upload a file
r576 return self.__execute(command)
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 def pwd(self):
Miguel Valdez
jroutils_ftp.py: SSHClient changes file permissions on remote server after upload a file
r576 command = 'pwd'
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
jroutils_ftp.py: SSHClient changes file permissions on remote server after upload a file
r576 return self.__execute(command)
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 def cd(self, remotefolder):
"""
José Chávez
setup actualizado para no necesitar numpy previamente
r1026 cd is used to change remote working directory on server
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573
José Chávez
setup actualizado para no necesitar numpy previamente
r1026 Input:
remotefolder - current working directory
Affects:
self.remotefolder
Return:
0 in case of error else 1
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 """
if not self.status:
return 0
Miguel Valdez
jroutils_ftp.py: SSHClient changes file permissions on remote server after upload a file
r576
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 if remotefolder == self.remotefolder:
return 1
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
jroutils_ftp.py: SSHClient changes file permissions on remote server after upload a file
r576 chk_command = "cd %s; pwd" %remotefolder
mkdir_command = "mkdir %s" %remotefolder
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
jroutils_ftp.py: SSHClient changes file permissions on remote server after upload a file
r576 if not self.__execute(chk_command):
if not self.__execute(mkdir_command):
self.remotefolder = None
return 0
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 self.remotefolder = remotefolder
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 return 1
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 def sendFile(self, fullfilename):
if not self.status:
return 0
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 try:
self.__scpClientObj.put(fullfilename, remote_path=self.remotefolder)
Miguel Valdez
jroutils_ftp.py: Remote Server is opened and closed each time a file is sent
r653 except scp.ScpError, e:
print "[SSH Server]", str(e)
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 return 0
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
jroutils_ftp.py: SSHClient changes file permissions on remote server after upload a file
r576 remotefile = os.path.join(self.remotefolder, os.path.split(fullfilename)[-1])
command = 'chmod 775 %s' %remotefile
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
jroutils_ftp.py: SSHClient changes file permissions on remote server after upload a file
r576 return self.__execute(command)
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 class SendToServer(ProcessingUnit):
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
def __init__(self, **kwargs):
ProcessingUnit.__init__(self, **kwargs)
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 self.isConfig = False
José Chávez
setup actualizado para no necesitar numpy previamente
r1026 self.clientObj = None
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573
Miguel Valdez
Bug fixed in SignalChain GUI:...
r581 def setup(self, server, username, password, remotefolder, localfolder, ext='.png', period=60, protocol='ftp', **kwargs):
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 self.clientObj = None
self.localfolder = localfolder
self.ext = ext
self.period = period
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 if str.lower(protocol) == 'ftp':
self.clientObj = FTPClient(server, username, password, remotefolder, period)
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 if str.lower(protocol) == 'ssh':
self.clientObj = SSHClient(server, username, password, remotefolder, period)
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 if not self.clientObj:
raise ValueError, "%s has been chosen as remote access protocol but it is not valid" %protocol
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 self.clientObj.start()
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 def findFiles(self):
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
Bugs fixed: Error selecting FTP parameters in Signal Chain GUI
r606 if not type(self.localfolder) == list:
folderList = [self.localfolder]
else:
folderList = self.localfolder
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
SendToServer:...
r633 #Remove duplicate items
folderList = list(set(folderList))
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
Bugs fixed: Error selecting FTP parameters in Signal Chain GUI
r606 fullfilenameList = []
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
Bugs fixed: Error selecting FTP parameters in Signal Chain GUI
r606 for thisFolder in folderList:
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
SendToServer:...
r633 print "[Remote Server]: Searching files on %s" %thisFolder
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
Bugs fixed: Error selecting FTP parameters in Signal Chain GUI
r606 filenameList = glob.glob1(thisFolder, '*%s' %self.ext)
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
Bugs fixed: Error selecting FTP parameters in Signal Chain GUI
r606 if len(filenameList) < 1:
José Chávez
cambios para online offline
r1019
Miguel Valdez
Bugs fixed: Error selecting FTP parameters in Signal Chain GUI
r606 continue
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
Bugs fixed: Error selecting FTP parameters in Signal Chain GUI
r606 for thisFile in filenameList:
fullfilename = os.path.join(thisFolder, thisFile)
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
SendToServer:...
r633 if fullfilename in fullfilenameList:
continue
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
SendToServer:...
r633 #Only files modified in the last 30 minutes are considered
if os.path.getmtime(fullfilename) < time.time() - 30*60:
continue
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
Bugs fixed: Error selecting FTP parameters in Signal Chain GUI
r606 fullfilenameList.append(fullfilename)
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 return fullfilenameList
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 def run(self, **kwargs):
if not self.isConfig:
self.init = time.time()
self.setup(**kwargs)
self.isConfig = True
José Chávez
this fixes bug #1074 and bug #1101
r1011 if not self.clientObj.is_alive():
print "[Remote Server]: Restarting connection "
self.setup(**kwargs)
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 if time.time() - self.init >= self.period:
fullfilenameList = self.findFiles()
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
Bugs fixed: Error selecting FTP parameters in Signal Chain GUI
r606 if self.clientObj.updateFileList(fullfilenameList):
print "[Remote Server]: Sending the next files ", str(fullfilenameList)
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 self.init = time.time()
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 def close(self):
print "[Remote Server] Stopping thread"
self.clientObj.stop()
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Daniel Valdez
This update consider connect and send FTP using multiprocessing with timeout
r517 class FTP(object):
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 """
Ftp is a public class used to define custom File Transfer Protocol from "ftplib" python module
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 Non-standard Python modules used: None
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 Written by "Daniel Suarez":mailto:daniel.suarez@jro.igp.gob.pe Oct. 26, 2010
"""
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
def __init__(self,server = None, username=None, password=None, remotefolder=None):
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 """
This method is used to setting parameters for FTP and establishing connection to remote server
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 Inputs:
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897 server - remote server IP Address
username - remote server Username
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 password - remote server password
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 remotefolder - remote server current working directory
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 Return: void
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Affects:
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 self.status - in Error Case or Connection Failed this parameter is set to 1 else 0
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 self.folderList - sub-folder list of remote folder
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 self.fileList - file list of remote folder
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 """
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 if ((server == None) and (username==None) and (password==None) and (remotefolder==None)):
server, username, password, remotefolder = self.parmsByDefault()
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 self.server = server
self.username = username
self.password = password
self.remotefolder = remotefolder
self.file = None
self.ftp = None
self.status = 0
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 try:
self.ftp = ftplib.FTP(self.server)
self.ftp.login(self.username,self.password)
self.ftp.cwd(self.remotefolder)
José Chávez
merge con schain_mp
r1034 # print 'Connect to FTP Server: Successfully'
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488
except ftplib.all_errors:
print 'Error FTP Service'
self.status = 1
return
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 self.dirList = []
try:
self.dirList = self.ftp.nlst()
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 except ftplib.error_perm, resp:
if str(resp) == "550 No files found":
print "no files in this directory"
self.status = 1
return
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 except ftplib.all_errors:
print 'Error Displaying Dir-Files'
self.status = 1
return
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 self.fileList = []
self.folderList = []
#only for test
for f in self.dirList:
name, ext = os.path.splitext(f)
if ext != '':
self.fileList.append(f)
José Chávez
setup actualizado para no necesitar numpy previamente
r1026 # print 'filename: %s - size: %d'%(f,self.ftp.size(f))
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488
def parmsByDefault(self):
server = 'jro-app.igp.gob.pe'
username = 'wmaster'
password = 'mst2010vhf'
remotefolder = '/home/wmaster/graficos'
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 return server, username, password, remotefolder
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 def mkd(self,dirname):
"""
mkd is used to make directory in remote server
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 Input:
dirname - directory name
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 Return:
1 in error case else 0
"""
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897 try:
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 self.ftp.mkd(dirname)
except:
print 'Error creating remote folder:%s'%dirname
return 1
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 return 0
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 def delete(self,filename):
"""
delete is used to delete file in current working directory of remote server
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 Input:
filename - filename to delete in remote folder
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 Return:
1 in error case else 0
"""
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 try:
self.ftp.delete(filename)
except:
print 'Error deleting remote file:%s'%filename
return 1
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 return 0
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 def download(self,filename,localfolder):
"""
download is used to downloading file from remote folder into local folder
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 Inputs:
filename - filename to donwload
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 localfolder - directory local to store filename
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 Returns:
self.status - 1 in error case else 0
"""
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 self.status = 0
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 if not(filename in self.fileList):
print 'filename:%s not exists'%filename
self.status = 1
return self.status
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 newfilename = os.path.join(localfolder,filename)
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
self.file = open(newfilename, 'wb')
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 try:
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897 print 'Download: ' + filename
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 self.ftp.retrbinary('RETR ' + filename, self.__handleDownload)
print 'Download Complete'
except ftplib.all_errors:
print 'Error Downloading ' + filename
self.status = 1
return self.status
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 self.file.close()
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
return self.status
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488
def __handleDownload(self,block):
"""
__handleDownload is used to handle writing file
"""
self.file.write(block)
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 def upload(self,filename,remotefolder=None):
"""
upload is used to uploading local file to remote directory
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 Inputs:
filename - full path name of local file to store in remote directory
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
remotefolder - remote directory
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 Returns:
self.status - 1 in error case else 0
"""
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 if remotefolder == None:
remotefolder = self.remotefolder
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 self.status = 0
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 try:
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897 self.ftp.cwd(remotefolder)
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 self.file = open(filename, 'rb')
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 (head, tail) = os.path.split(filename)
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 command = "STOR " + tail
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 print 'Uploading: ' + tail
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897 self.ftp.storbinary(command, self.file)
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 print 'Upload Completed'
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 except ftplib.all_errors:
print 'Error Uploading ' + tail
self.status = 1
return self.status
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 self.file.close()
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 #back to initial directory in __init__()
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897 self.ftp.cwd(self.remotefolder)
return self.status
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 def dir(self,remotefolder):
"""
dir is used to change working directory of remote server and get folder and file list
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 Input:
remotefolder - current working directory
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 Affects:
self.fileList - file list of working directory
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Return:
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 infoList - list with filenames and size of file in bytes
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 self.folderList - folder list
"""
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 self.remotefolder = remotefolder
print 'Change to ' + self.remotefolder
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897 try:
self.ftp.cwd(remotefolder)
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 except ftplib.all_errors:
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897 print 'Error Change to ' + self.remotefolder
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 infoList = None
self.folderList = None
return infoList,self.folderList
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 self.dirList = []
try:
self.dirList = self.ftp.nlst()
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 except ftplib.error_perm, resp:
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897 if str(resp) == "550 No files found":
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 print "no files in this directory"
infoList = None
self.folderList = None
return infoList,self.folderList
except ftplib.all_errors:
print 'Error Displaying Dir-Files'
infoList = None
self.folderList = None
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897 return infoList,self.folderList
infoList = []
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 self.fileList = []
self.folderList = []
for f in self.dirList:
name,ext = os.path.splitext(f)
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897 if ext != '':
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 self.fileList.append(f)
value = (f,self.ftp.size(f))
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897 infoList.append(value)
if ext == '':
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 self.folderList.append(f)
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 return infoList,self.folderList
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 def close(self):
"""
close is used to close and end FTP connection
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 Inputs: None
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 Return: void
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 """
self.ftp.close()
class SendByFTP(Operation):
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
def __init__(self, **kwargs):
Operation.__init__(self, **kwargs)
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 self.status = 1
Daniel Valdez
setting a period to send by FTP
r489 self.counter = 0
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 def error_print(self, ValueError):
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 print ValueError, 'Error FTP'
print "don't worry the program is running..."
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Daniel Valdez
This update consider connect and send FTP using multiprocessing with timeout
r517 def worker_ftp(self, server, username, password, remotefolder, filenameList):
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 self.ftpClientObj = FTP(server, username, password, remotefolder)
Daniel Valdez
This update consider connect and send FTP using multiprocessing with timeout
r517 for filename in filenameList:
Miguel Valdez
A new SendToServer Unit has been created to upload files to a remote server....
r573 self.ftpClientObj.upload(filename)
self.ftpClientObj.close()
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Daniel Valdez
This update consider connect and send FTP using multiprocessing with timeout
r517 def ftp_thread(self, server, username, password, remotefolder):
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 if not(self.status):
return
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Miguel Valdez
SendToServer:...
r633 import multiprocessing
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Daniel Valdez
This update consider connect and send FTP using multiprocessing with timeout
r517 p = multiprocessing.Process(target=self.worker_ftp, args=(server, username, password, remotefolder, self.filenameList,))
p.start()
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Daniel Valdez
This update consider connect and send FTP using multiprocessing with timeout
r517 p.join(3)
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Daniel Valdez
This update consider connect and send FTP using multiprocessing with timeout
r517 if p.is_alive():
p.terminate()
p.join()
print 'killing ftp process...'
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 self.status = 0
return
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Daniel Valdez
This update consider connect and send FTP using multiprocessing with timeout
r517 self.status = 1
return
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Daniel Valdez
new utils package:jroutils, jroutils_ftp...
r488 def filterByExt(self, ext, localfolder):
fnameList = glob.glob1(localfolder,ext)
self.filenameList = [os.path.join(localfolder,x) for x in fnameList]
if len(self.filenameList) == 0:
self.status = 0
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Daniel Valdez
setting a period to send by FTP
r489 def run(self, dataOut, ext, localfolder, remotefolder, server, username, password, period=1):
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
self.counter += 1
Daniel Valdez
setting a period to send by FTP
r489 if self.counter >= period:
self.filterByExt(ext, localfolder)
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Daniel Valdez
This update consider connect and send FTP using multiprocessing with timeout
r517 self.ftp_thread(server, username, password, remotefolder)
Daniel Valdez
setting a period to send by FTP
r489 self.counter = 0
José Chávez
cambiados los kwargs para cada operacion/unidad de procesamiento
r897
Daniel Valdez
This update consider connect and send FTP using multiprocessing with timeout
r517 self.status = 1