It currently runs only on GNU/Linux. It should work on Android/Linux as well, but I haven't tested it yet.
Shell execution doesn't support environment variable setting prior to starting up the shell. I found out that the latest OpenSSH version doesn't allow it. This makes it useless because you don't know whether an environment variable setting will succeed or not and you have to work around if it fails (i.e. setting it via shell commands). So, the best option is to simply skip that step and then do it with shell commands in all cases.
File transfers are done using SFTP, but not all of the SFTP features are exposed. SFTP is not really a file transfer protocol, but a remote file system access protocol. The library provides a higher level interface as found in proper file transfer protocols.
Being such a standalone library, tssh has limited support for various cryptographic algorithms. It supports the following encryption, hash and key exchange algorithms.
The code is written on a GNU/Linux box, but should work with Android/Linux as well. Eventually, I will make a Windows port. The framework is ready for it.
For Linux, I use the gcc compiler. This is needed because of the cothread library. If you want to use something else, you can easily strip the cothread assembly part out of the source and compile it separately with your assembler and then compile the tssh code with any other C compiler.
For Windows, I'll use the MSVC compiler which has a "naked" attribute for functions.
The library internally uses a cooperative thread library. This limits its use to x86-32, x86-64 and ARM-32 CPUs. I will add ARM-64 at some point. I'm not going to target anything old or esoteric.
Rewrite KEX routines to use the cothread framework.
Test re-keying.
user-defined limits for pend_max etc.
Make proper header and code sections.
Allow all functions to be static, for great inclusion.
Make sure imported code doesn't leave behind any macros defined, for great inclusion.
My code is distributed under the BSD license. I have imported some code from elsewhere, namely the encryption functions. These are mostly public domain and some have the BSD license. I have kept their license declarations intact and the overall package should be compatible with the BSD license but you should check it yourself if you want to use it commercially. Please contact cinarus at yahoo dot com if there are any problems.
You may define the CPP macro TSSH_NO_LOGS before including the file or on the command line. This will disable all log events.
tssh is almost completely asynchronous. It has a separate thread for the control loop as well as two other threads for I/O. Results and incoming data are provided as events to the user thread. In addition, tssh is thread-safe. You can call any function from any user thread you want, but it's best to use only one thread to watch for events.
tssh_event_t* tssh_get_event(tssh_t *T);The returned event structure is pretty complex, I will get into its details later. The first field you need to know is the type field:
typedef struct
{
int type;
...
} tssh_event_t;
This tells us what kind of event we have received. Here is a complete
list of events:
enum {
TSSH_EVENT_LOG= 1,
TSSH_EVENT_BANNER,
TSSH_EVENT_AUTH,
TSSH_EVENT_CONNECT_SUCCESS,
TSSH_EVENT_CONNECT_FAILURE,
TSSH_EVENT_DISCONNECTED,
TSSH_EVENT_SHELL_SUCCESS,
TSSH_EVENT_SHELL_FAILURE,
TSSH_EVENT_CHANNEL_DATA,
TSSH_EVENT_CHANNEL_EXTENDED_DATA,
TSSH_EVENT_CHANNEL_WRITE_OK,
TSSH_EVENT_CHANNEL_CLOSED,
TSSH_EVENT_PORT_SUCCESS,
TSSH_EVENT_PORT_FAILURE,
TSSH_EVENT_PORT_CONNECT,
TSSH_EVENT_FTP_DISCONNECTED,
TSSH_EVENT_FTP_SUCCESS,
TSSH_EVENT_FTP_FAILURE,
TSSH_EVENT_FTP_CONNECT_SUCCESS,
TSSH_EVENT_FTP_CONNECT_FAILURE,
};
I will describe each event as we progress. OK, so how to tell when there
is an event to be read? After all, this is an asychronous library and
waiting at get_event() wouldn't be productive at all.
The library creates a socket which can be select()ed or poll()ed to monitor for new events. You shouldn't read from this socket at all, the get_event() function does that. You can obtain the socket descriptor by calling:
int tssh_event_fd(tssh_t *T);If you don't need asynchronous behaviour, you can simply call get_event() and let your user thread block until an event arrives.
After you're done with an event, you shouldn't try to deallocate it. get_event() will do that on the next call.
Prior to reporting the failure of a command, the library will emit a log message thru the TSSH_EVENT_LOG event. This event has the D.str field, as in:
typedef struct
{
union {
...
char *str;
...
} D;
} tssh_event_t;
Log messages
don't contain any user data except for file names. Ignoring these messages
right out is another option, although this can make it hard to troubleshoot
errors because there is no other indication.
tssh_t *tssh_init (char *alias, char *host, int port);If the given port is negative, the default port 22 is used. The connection process also includes a key exchange, which can take some time. When the function returns, the object is ready to supply events but you can't do anything else with it until you get a TSSH_EVENT_CONNECT_FAILURE or TSSH_EVENT_AUTH; the control thread is still working on the background calculating and exchanging keys with the server.
The alias argument is any arbitrary string, which will be used in a later release. You can just as well supply NULL here.
This function will eventually generate one of the following events:
When you receive the AUTH event, you should verify the host key. This verification ensures that the server you're connecting to is really the one you intended and not some intermediary attacker. In order to do so, you may use the following fields in the event structure:
char *host, *alias; uint32_t host_key_size; uint8_t *host_key; char *host_key_alg;If the verification fails, then you should disconnect. This is explained later.
If the verification succeeds, you may proceed with supplying your username and password using:
void tssh_auth(tssh_t *T, char *user, char *passwd);After this call, you will now expect one of the following events:
void tssh_disconnect(tssh_t *T);This function will return immediately and start the disconnection process. This will cause channels to be closed and unfinished FTP transfers to be cancelled. Events will be generated for all these things. The last event to be generated is TSSH_EVENT_DISCONNECTED. When you receive this event, the tssh_t pointer is no longer valid.
If you don't need any CHANNEL_CLOSED events or somehow want to shut it down immediately, you should call:
void tssh_quit(tssh_t *T);After this call, no more events will be generated and the pointer is invalid. This is probably a good idea when the reported host key can't be verified. At that point you already don't have any channels open yet. Therefore, closing the thing immediately and returning with an error makes sense.
void tssh_shell(tssh_t *T, char **env);starts a remote shell after a connection is made. This will result in one of the following events:
This function starts the default shell for the user. The failure event doesn't have any data associated with it. The success event has one data field: the channel.
typedef struct {
...
tssh_channel_t *channel;
...
} tssh_event_t;
When you get the channel from this event, you can then proceed writing to
and reading from it. A shell channel has no further special features.
If you want to close a channel, you should call:
void tssh_channel_close(tssh_channel_t *ch);When a channel is closed either by you or the remote server, the system sends the event TSSH_EVENT_CHANNEL_CLOSED. This event has the channel field and nothing else. You don't need to do anything to deallocate this etc. It's all done automatically.
int tssh_channel_write(tssh_channel_t *ch,
uint8_t *data, uint32_t len);
The return value is non-zero if there was a yet-undiscovered problem with
the channel. Remember, the library runs in the background in a different
thread. Therefore, it can discover that the server has closed a channel
before it gets a chance to tell you thru get_event(). This is useful in
that, you can stop sending data immediately if you get a non-zero result
from channel_write().
This function copies the data given in (data,len). When it returns, you may deallocate data if necessary.
If you have more substantial amount of data, it's more efficient to allocate write buffer and fill that in:
typedef struct tssh_write_buffer
{
uint32_t size;
uint32_t len;
uint8_t *data;
struct tssh_write_buffer *next;
} tssh_write_buffer_t;
tssh_write_buffer_t *tssh_alloc_write_buffer(uint32_t size);
The size field tells us how big the buffer is, it's copied from the
call to alloc_write_buffer(). The len field holds the number of bytes
used in the buffer. The next field will be NULL when allocated. It's
used internally by the library.
In any case, after you have filled in the data you want (updating len along the way of course) you may send the data using:
int tssh_channel_write_buffer
(tssh_channel_t *ch, tssh_write_buffer_t *B);
The return value has the same semantics as for channel_write(). The write
buffer is deallocated automatically when the system is done with it.
In general, a channel write will not block. It's done in another thread and the user thread simply sends a message to that thread. However, if the writing thread is busy at that time, the message will be buffered. Instead of allowing unlimited writes to go unchecked, I implemented a limited buffer in order to stall the user thread. Ideally, such flow control should be done at a higher protocol.
If you want to make sure that the user thread never stalls, you may delay each of your writes (after the first write) until you get the event TSSH_EVENT_CHANNEL_WRITE_OK. This event contains a valid channel pointer and means that a write to the channel would probably not block the calling thread. This event is generated after a chunk of data is sent over the wire and thus some buffer space is freed. Chunk here means an argument to write() or write_block().
typedef struct
{
...
tssh_buffer_t *buffer;
...
} tssh_event_t;
As with other objects, the buffer is free()d on the next call to
get_event(). If you'd like to keep the buffer for longer, just
set the buffer field of the event object to NULL.
After you get the buffer, you can modify its fields however you like, it doesn't effect deallocation. It's a very simple struct:
typedef struct
{
uint8_t *data;
uint32_t len;
} tssh_buffer_t;
Channel data can also be reported as part of an event with type
TSSH_EVENT_CHANNEL_EXTENDED_DATA. This event is used for what's
called extended data for a channel. The SSH RFC describes this
as a separate stream of data for the channel, in addition to the
main stream, as reported by CHANNEL_DATA events. It's supposed to
handle standard error streams but in practice I haven't encountered
its use. When you start a shell with a virtual tty, all output from
the shell is directed to that tty stream, whether it originates from
the standard output or standard error stream. Maybe it's used for
other command invocations other than the shell.
void tssh_port_request(tssh_t *T, uint32_t port);If port is given as zero, then the server will pick an available port number. The server then binds a socket to the given/picked port and responds. When it does, the library produces one of the following events:
typedef struct
{
...
uint32_t port;
...
} tssh_event_t;
In order to cancel such a forwarded port, you may call:
void tssh_port_cancel(tssh_t *T, uint32_t port);When there is a connection to the forwarded port on the server machine, the system generates the TSSH_EVENT_PORT_CONNECT event. This event contains the following fields:
void tssh_ftp_connect(tssh_t *T);This will generate one of the following events:
void tssh_ftp_disconnect(tssh_t *T);This will generate the event TSSH_EVENT_FTP_DISCONNECTED. The same event is generated when the server decides to severe the connection.
FTP operations don't return immediate results, they generate one of the following events when the operation completes:
typedef struct tssh_ftop
{
int cmd;
char *fn; char *oldfn;
char *localfn;
uint8_t *data;
uint32_t size;
struct tssh_ftop *next;
uint8_t *udata;
} tssh_ftop_t;
Here, cmd is one of
void tssh_ftp_download_file(tssh_t *T, char *fn,
char *localfn);
This generates the usual failure/success events with no additional data.
It's also possible to read a file directly into memory using:
void tssh_ftp_download_mem(tssh_t *T, char *fn);When this finishes successfully, the generated success event will have a valid 'buffer' field. Its use is the same as described in the section describing channels.
void tssh_ftp_upload_file
(tssh_t *T, char *fn, char *localfn);
void tssh_ftp_upload_mem
(tssh_t *T, char *fn, uint8_t *data, uint32_t len);
For both of these functions, the generated events have no additional data.
When you use upload_mem(), the data must not be deallocated until the
operation has finished. If you're doing multiple operations without
waiting for a response, you may recover this pointer by accessing
E->ftp_op->udatawhere E is an event pointer.
void tssh_ftp_list(tssh_t *T, char *dir_name);lists the contents of a directory. The generated success event for this operation has the data in its n_fileinfo and fileinfo fields:
typedef struct
{
...
uint32_t n_fileinfo;
tssh_fileinfo_t *fileinfo;
...
} tssh_event_t;
The fileinfo array gets deallocated (names included) when you call
get_event() again, so make sure to create copies if needed.
typedef struct
{
char *name;
char *long_name;
int type;
uint32_t perm;
uint64_t size;
uint32_t mtime;
} tssh_fileinfo_t;
The type field is not used for now. This will change..
Remaining fields are pretty self explanatory. If a server doesn't
report a field such as perm or mtime, the corresponding field will
be all zeros. The long_name field is a string which would be displayed
for the file with a command like "ls -l".
void tssh_ftp_create_dir(tssh_t *T, char *fn); void tssh_ftp_remove_dir(tssh_t *T, char *fn); void tssh_ftp_remove_file(tssh_t *T, char *fn); void tssh_ftp_rename(tssh_t *T, char *oldfn, char* fn);These are all simple commands which generate basic success/failure events with no data.
The query function
void tssh_ftp_query(tssh_t *T, char *fn);will get the fileinfo for a given file name. The returned success event has one fileinfo structure in it. This fileinfo has no valid name or long_name.
The main problem is that, an SSH connection provides multiple channels of data. If I made a library which does only one thing at a time, I could use a synchronous, blocking library. For instance, you would have only one channel on the connection and all events would be related to that channel only. Handling these events within read() or write() requests would be trivial because they all refer to the same channel.
When you have multiple channels, the synchronous idea blows up. You then need to filter events, reset event notifications etc. For instance, let's say that you have two channels on a connection and you want to block until you get an event related to channel #1.
You would then go ahead and wait for an event on channel #1, be it a window increment in order to send more data or just some channel data. While waiting for that, if you get an event related to channel #2, then you need to requeue it. Also, you need to reset the event notification mechanism so that later notifications for events related to channel #2 will still occur. With a synchronous library, it would be very hairy to implement multiple channels on one connection.
First of all, in order to have an asynchronous library which doesn't block all the time, I need at least one thread per connection.
In Linux, you can wait on multiple objects easily because it's easy to create socket pairs which can be fed to calls such as select() or poll(). When a trigger event happens you can also write to one end of the socket thus waking up the waiting thread. For instance, you can wait for a condition variable and a socket simultaneously simply by creating another socket for the condition variable.
However, this is not so in Windows. There are functions for waiting multiple things simulatenously but they are not in any way similar to select(). There is also a way of making socket pairs just like in Unix but it's an ugly workaround which creates a local socket and then connects to it etc. So, a thread can wait on only one type of event at a time.
The main thread and the writer thread wait on condition variables until they receive a message. The reader thread waits on the socket, but reverts to waiting on a condition variable when it's first starting up or when a new key is to be used.
I don't use a separate pthread for FTP because doing so would increase the complexity. I'd need even more locks and message queues. The nice thing about the current setup is that, only one of the main thread and the FTP thread is active at any given time. This simplifies things a lot.
For instance, when you make a connection the server doesn't give you any hint about what kind of operations it supports. It doesn't tell you if you can forward ports or not. It doesn't tell you which versions of the SFTP it can handle. It doesn't tell you whether shell connections can have their own PTYs or have some environment variables set. You have to make a request and see if it fails to find out about these things. SSH wasn't based on any previous work, it didn't have any limitations because of a protocol it was based on, it was created fresh. Then why not include these things among many others? My guess is that they never thought it would be this popular and just assumed that everybody would use whatever they put out.
My second disappointment is with the SFTP protocol. First of all, it's a packet based protocol, encapsulated in a stream-based channel system which is implemented on top of another packet based protocol. This is just garbage. They specify in the RFC that SFTP can be used on top of other protocols other than SSH. Yeah, right. If it didn't suck ass, maybe somebody would go out on a limb and implement it over Bluetooth or something. It's not gonna happen.
The shortcomings of SFTP don't end there. SFTP isn't really a file transfer protocol as its name suggests. It's a remote file system access protocol. You can't just say "hey, here is some data, record it as this file". You need to open the file, write to it and then close the file as if it was a regular file on the disk. The problem is that, on the server side these are implemented as simple system calls. So, if your connection gets severed in the middle of a transfer, you can't roll back. There are no transaction semantics here. Also there is no anonymous FTP either.
I could go on and on. Limited packet sizes for SFTP, lack of request IDs for general SSH commands, lack of replies for write requests, etc. It's basicly a poorly designed hack. However, there is no other choice for secure connections with non-centrally managed keys so I have to stick with it. It's OK for interactive use with a human at the controls, I guess. It sucks for automating things.