Erle Robotics Python Networking Gitbook Free

SFTP: File Transfer Over SSH

Version 2 of the SSH protocol includes a sub-protocol called the “SSH File Transfer Protocol” (SFTP) that lets you walk the remote directory tree, create and delete directories and files, and copy files back and forth from the local to the remote machine. The capabilities of SFTP are so complex and complete, in fact, that they support not only simple file-copy operations, but can power graphical file browsers and can even let the remote filesystem be mounted locally.

When talking about SFTP commands than is provided by the bare paramiko documentation for the Python SFTP client( http://www.lag.net/paramiko/docs/paramiko.SFTPClient-class ) ; here are the main things to remember when doing SFTP:

• The SFTP protocol is stateful, just like FTP, and just like your normal shell account. So you can either pass all file and directory names as absolute paths that start at the root of the filesystem, or use getcwd() and chdir() to move around the filesystem and then use paths that are relative to the directory in which you have arrived.

• You can open a file using either the file() or open() method and you get back a file-like object connected to an SSH channel that runs independently of your SFTP channel.

• Because each open remote file gets an independent channel, file transfers can happen asynchronously; you can open many remote files at once and have them all streaming down to your disk drive, or open new files and be sending data the other way.

• Finally, keep in mind that no shell expansion is done on any of the file names you pass across SFTP. If you try using a file name like * or one that has spaces or special characters, they are simply interpreted as part of the file name. This means that any support for pattern-matching that you want to provide to the user has to be through fetching the directory contents yourself and then checking their pattern against each one, using a routine like those provided in fnmatch in the Python Standard Library. fnmatch module provides support for Unix shell-style wildcards, which are not the same as regular expressions.

A very modest example SFTP session is shown in sftp.py. It does something simple that system administrators might often need: it connects to the remote system and copies messages log files out of the /var/log directory, perhaps for scanning or analysis on the local machine. The functools module is for higher-order functions: functions that act on or return other functions. In general, any callable object can be treated as a function for the purposes of this module, as shown in the sftp.py:

import functools
import paramiko

class AllowAnythingPolicy(paramiko.MissingHostKeyPolicy):
    def missing_host_key(self, client, hostname, key):
        return

client = paramiko.SSHClient()
client.set_missing_host_key_policy(AllowAnythingPolicy())
client.connect('127.0.0.1', username='test')  # password='')

def my_callback(filename, bytes_so_far, bytes_total):
    print 'Transfer of %r is at %d/%d bytes (%.1f%%)' % (
        filename, bytes_so_far, bytes_total, 100. * bytes_so_far / bytes_total)

sftp = client.open_sftp()
sftp.chdir('/var/log')
for filename in sorted(sftp.listdir()):
    if filename.startswith('messages.'):
        callback_for_filename = functools.partial(my_callback, filename)
        sftp.get(filename, filename, callback=callback_for_filename)

client.close()

Note that, although I made a big deal of talking about how each file that you open with SFTP uses its own independent channel, the simple get() and put() convenience functions provided by paramiko— which are really lightweight wrappers for an open() followed by a loop that reads and writes—do not attempt any asynchrony, but instead just block and wait until each whole file has arrived. This means that the foregoing script calmly transfers one file at a time, producing output that looks something like this:

root@erlerobot:~/Python_files#  python sftp.py
Transfer of 'messages.1' is at 32768/128609 bytes (25.5%)
Transfer of 'messages.1' is at 65536/128609 bytes (51.0%)
Transfer of 'messages.1' is at 98304/128609 bytes (76.4%)
Transfer of 'messages.1' is at 128609/128609 bytes (100.0%)
Transfer of 'messages.2.gz' is at 32768/40225 bytes (81.5%)
Transfer of 'messages.2.gz' is at 40225/40225 bytes (100.0%)
Transfer of 'messages.3.gz' is at 28249/28249 bytes (100.0%)
Transfer of 'messages.4.gz' is at 32768/71703 bytes (45.7%)
Transfer of 'messages.4.gz' is at 65536/71703 bytes (91.4%)
Transfer of 'messages.4.gz' is at 71703/71703 bytes (100.0%)