OK, a rather long post about effective digital communication. Hopefully an interesting read to folks who would like to add some code to protect communications but haven't gotten around to that TODO item just yet.
A commonly used method for sending messages to others when you need
authentication and privacy is to use an OpenPGP tool such as GNU
Privacy Guard (GnuPG). For real time communications such as instant
messaging, IRC, and socket IO, using Off The Record (OTR) messaging
provides
Perfect
Forward Secrecy and secure identification of the remote party
without the need for a web of trust.
In order to operate without a web of trust,
libotr implements the
Socialist Millionaires' Protocol (SMP). The SMP allows two parties
to verify that they both know the same secret. The secret might be a
passphrase or answer to a private joke that two people will easily
know. The SMP operates fine in the presence of eaves droppers (who
don't get to learn the secret). Active communications tampering is
not a problem, though of course it might cause the protocol not to
complete successfully.
Because the SMP doesn't rely on the fingerprint of the user's
private key for authentication, the private key becomes almost an
implementation detail. Once generated, the user generally doesn't
need to know about the key or it's fingerprint. The only time a user
really cares to know is when a key is created because a bit of
entropy has to go into that process. Of course, an application
should avoid regenerating keys for no reason because each time the
key is replaced the user has to use the SMP again to allow remote
parties to authenticate them.
In this article I'll show you how to use the current release, libotr
3.2.0+, to provide OTR messaging. I'll present two examples which
are both in C++ and use the boost library for socket IO. I have gone
this was so we can focus on the OTR action and not the details of
sockets.
The first example does not use the Socialist Millionaires' Protocol
(SMP). So the new_fingerprint() callback is essential to
establishing a secure session. When not using the SMP,
authentication is performed by comparing the sent fingerprints of
those you are wishing to communicate with against known good values.
These known values must be sent beforehand through a secure
secondary channel, such as a face to face meeting. Once fingerprints
have been accepted, subsequent OTR communications with the same
party can be performed without explicit fingerprint verification.
The second example makes things simpler for the user by using the
SMP for authentication of the remote party. This way, the
information exchanged beforehand becomes shared experiences you and
the other party have had such that a question can be raised that
only you and they can easily answer.
A central abstraction in using the libotr library is the struct
s_OtrlMessageAppOps vtable. This is used by libotr to callback into
your code when something happens such as a cryptographic fingerprint
being received, or libotr wanting to send a message to the other
end. The later happens frequently during OTR session establishment.
If a program monitors it's socket IO using select() or some other
mainloop abstraction, then having these internal protocol messages
being sent is not so much of an issue. Alas, for the simple echo
server I present one must remember that there might be one or more
internal OTR protocol messages sent from what seems like outside of
the normal program flow. I'll get back to this point while
describing the relevant section of the first example.
Many of the callback functions in s_OtrlMessageAppOps might be
simple stubs, but you should be aware of inject_message() which will
be called when libotr itself wants to send something, notify and
display_otr_message can both provide feedback to the user, the
new_fingerprint() method is called when a remote key is discovered
in order to allow you to inform the user and possibly abort the
session. The gone_secure() method is called to allow you to inform
the user that they are off the record. When you call libotr
functions you supply both a pointer to a s_OtrlMessageAppOps
structure
uiops and a void*
opdata. When libotr calls a method
in uiops it will pass
opdata back to you.
Another common three parameters you will pass to libotr functions
are the accountname, protocol and sender or receiver name. The
protocol string can be anything as long as both ends of the system
use the same protocol string. The state data that libotr uses is
stored in an OtrlUserState object which is created with
otrl_userstate_create() and passed to many of the libotr functions
along the way.
The code below loads a private key or creates a new one if none
already exists. Because creating a new key is an entropy heavy
operation, the setupKey() function warns the user that if they are
erratic it the process might move along a bit quicker. Note that the
uiops has a callback create_privkey to generate a key if needed. I
just prefer to make this codepath explicit and out of the main
callback logic.
bool ok( gcry_error_t et )
{
return gcry_err_code(et) == GPG_ERR_NO_ERROR;
}
void setupKey( const std::string& filename )
{
gcry_error_t et;
et = otrl_privkey_read( userstate, filename.c_str() );
if( !ok(et) )
{
cerr << "can't find existing key, generating a new one!" << endl;
cerr << "this needs a bunch of entropy from your machine... so please" << endl;
cerr << "move the mouse around and slap some keys mindlessly for a while" << endl;
cerr << "a message will be printed when keys have been made..." << endl;
et = otrl_privkey_generate( userstate, filename.c_str(),
accountname, protocol );
if( !ok(et) )
{
cerr << "failed to write new key file at:" << filename << endl;
}
cerr << "Have keys!" << endl;
}
}
The main.cpp program implements both the client and server. The
server mode is selected by passing -s at startup. Firstly, a
userstate is created, some variables set depending on if we are a
client or server, and the correct private key is loaded or created.
OTRL_INIT;
userstate = otrl_userstate_create();
keyfile = "client.key";
accountname = "client";
recipientname = "server";
if( ServerMode )
{
keyfile = "server.key";
accountname = "server";
recipientname = "client";
}
setupKey( keyfile );
The core logic for the echo client is to read a string from the user, send it to the
server, grab a reply from the server and show it to the user.
The start of the client code connects to a given port on localhost
and reads a string from the user.
VMSG << "client mode..." << endl;
stringstream portss;
portss << Port;
iosockstream stream( "127.0.0.1", portss.str() );
if (!stream)
{
cerr << "can't connect to server!" << endl;
exit(1);
}
string s;
while( true )
{
getline(cin,s);
cerr << "your raw message:" << s << endl;
cerr << "send plaintext:" << colorsend(s) << endl;
We certainly do not want to send the raw string
s over the wire to
the server though. That would very much be "on the record". So the
next fragment of the client gets libotr to encrypt the string
s so
we can send it off the record to the server. The userstate is the
value created during program initialization using
otrl_userstate_create(). The ui_ops is the vtable
s_OtrlMessageAppOps structure described above, and opdata is the
value we want libotr to pass back to our methods in ui_ops when it
uses them. In this case, we use the address of the iostream for the socket as the
opdata so callbacks can send and receive data on the socket if they
so desire. The newmessage will point to an off-the-record message
that the server can decrypt to read the string
s. The tests on the
return value for message_sending() ensure that we have a new,
encrypted off the record message to send instead of the plaintext
s.
void* opdata = &stream;
OtrlTLV* tlvs = 0;
gcry_error_t et;
char* newmessage;
void* opdata = &stream;
OtrlTLV* tlvs = 0;
gcry_error_t et;
char* newmessage;
et = otrl_message_sending( userstate, &ui_ops, opdata,
accountname, protocol, recipientname,
s.c_str(), tlvs, &newmessage,
myotr_add_appdata, &ui_ops );
cerr << "encoded... ok:" << ok(et) << endl;
if( !ok(et) )
{
cerr << "OTR message_sending() failed!" << endl;
}
if( ok(et) && !newmessage )
{
cerr << "There was no error, but an OTR message could not be made." << endl;
cerr << "perhaps you need to run some key authentication first..." << endl;
}
if( newmessage )
{
VMSG << "have new OTR message:" << newmessage << endl;
s = newmessage;
}
Since we have replaced the plaintext
s with the off the record
version, we send that to the server using the socket iostream and
then wait a moment before reading a response. The while loop is
slightly hairy in that it will block for new messages if we are not
secure. As I mentioned above, libotr can call the inject_message()
callback to write a new off the record message to the socket.
Outgoing messages will be generated and injected during session
establishment. There is no incoming version of inject_message() so
the client needs to keep reading these injected messages before it
tries to send another off the record message. One will find that
there are many messages exchanged between libotr at each end when
the string
s is written to the socket. This only happens the first
time through to setup the OTR protocol.
When reading messages from the server, the encrypted string is read
and passed to otrl_message_receiving(). If the recevied message was
an OTR message that was sent from the other end by libotr using
inject_message() then otrl_message_receiving() will indicate to the
client that it should simply ignore this message. Otherwise a real
message was encrypted and sent by the server and so the client will
show the user the decrypted newmessage.
cerr << "WRITE:" << s << endl;
stream << s << endl;
usleep( 200 * 1000 );
while( !secure && stream.peek() != std::iostream::traits_type::eof()
|| secure && stream.rdbuf()->available() )
{
s = "junk";
VMSG << "reading data from server" << endl;
getline(stream,s);
VMSG << "READ:" << s << endl;
int ignore_message = otrl_message_receiving(
userstate, &ui_ops, opdata,
accountname, protocol, recipientname,
s.c_str(),
&newmessage,
&tlvs,
myotr_add_appdata, &ui_ops );
VMSG << "ignore:" << ignore_message << " newmsg:" << maybenull(newmessage) << endl;
if( ignore_message )
{
VMSG << "libotr told us to ignore this message..." << endl;
VMSG << "available:" << stream.rdbuf()->available() << endl;
VMSG << " in_avail:" << stream.rdbuf()->in_avail() << endl;
continue;
}
if( newmessage )
s = newmessage;
otrl_message_free( newmessage );
cout << color( s ) << endl;
}
Server mode is handled by a thread which executes server_session()
using the std::iostream for the new socket.
if( ServerMode )
{
VMSG << "server mode..." << endl;
boost::asio::io_service io_service;
tcp::acceptor a( io_service, tcp::endpoint( tcp::v4(), Port ));
for (;;)
{
h_iosockstream stream(new iosockstream());
a.accept( *(stream->rdbuf()) );
boost::thread t(boost::bind(server_session, stream));
}
}
The server implementation would look like the below if OTR messaging
was not being used.
void server_session( h_iosockstream streamptr )
{
iosockstream& stream = *(streamptr.get());
while( stream )
{
std::string s;
getline( stream,s );
cout << "server got:" << s << endl;
stream << s << endl;
}
}
The OTR server implementation starts out the same way, reading a
string from the socket. Then our old friend otrl_message_receiving()
is called to decrypt that message. If ignore_message is set then
there is nothing to be done and we simply continue to the top of the
loop to read another string from the client. Also, if we are not yet
secure, there is no point in trying to send a new OTR message back
to the client, so we simply continue at the top of the while loop
again. This way we avoid writing replies to the client when session
establishment messages are sent by libotr on the client side.
This might seem a little strange at first, how will we ever become
secure and start replying to the client if all we do is read from
them and throw away the messages. The thing to keep in mind is that
messages sent with inject_message() on the client will be seen by
libotr when we call otrl_message_receiving() which in turn might
cause libotr on the server to inject_message() with a reply to this
session establishment message. Eventually libotr will call the
gone_secure() OtrlMessageAppOps callback in which we set the global
variable
secure to true, this allowing the server to start replying to the
client as it normally would.
void server_session( h_iosockstream streamptr )
{
iosockstream& stream = *(streamptr.get());
while( stream )
{
gcry_error_t et;
std::string s;
VMSG << "getting more data from the client..." << endl;
getline( stream,s );
VMSG << "READ:" << s << endl;
void* opdata = &stream;
OtrlTLV* tlvs = 0;
char *newmessage = NULL;
int ignore_message = otrl_message_receiving(
userstate, &ui_ops, opdata,
accountname, protocol, recipientname,
s.c_str(),
&newmessage,
&tlvs,
myotr_add_appdata, &ui_ops );
VMSG << "ignore:" << ignore_message << " newmsg:" << maybenull(newmessage) << endl;
if( newmessage )
s = newmessage;
otrl_message_free( newmessage );
if( ignore_message )
{
VMSG << "libotr told us to ignore this message..." << endl;
continue;
}
cout << "ignore:" << ignore_message << " server got:" << s << endl;
cout << "message from client:" << color(s) << endl;
// do not echo back messages when we are establishing the session
if( !secure )
continue;
The remainder of server_session() creates the echo reply message,
encrypts it with otrl_message_sending() and sends the OTR message
over the socket.
static int count = 0;
stringstream zz;
zz << "back to you s:" << s << " count:" << count++;
s = zz.str();
cout << "writing...s:" << s << endl;
cerr << "send plaintext:" << colorsend(s) << endl;
et = otrl_message_sending( userstate, &ui_ops, opdata,
accountname, protocol, recipientname,
s.c_str(), tlvs, &newmessage,
myotr_add_appdata, &ui_ops );
if( !ok(et) )
{
cerr << "OTR message_sending() failed!" << endl;
}
if( ok(et) && !newmessage )
{
cerr << "There was no error, but an OTR message could not be made." << endl;
cerr << "perhaps you need to run some key authentication first..." << endl;
}
if( newmessage )
{
VMSG << "have new OTR message:" << newmessage << endl;
s = newmessage;
}
VMSG << "writing otr...s:" << s << endl;
stream << s << endl;
As the security of the OTR messaging relies on fingerprints in the
first example, the new_fingerprint callback presents our fingerprint
and the remote fingerprint and asks the user if they want to
continue to establish the session or not. Unforuntately this means
the user has to eyeball scan the remote fingerprint against an
expected value they have obtained from the remote party at some
other time in a secure channel.
static void myotr_new_fingerprint( void *opdata, OtrlUserState us,
const char *accountname, const char *protocol,
const char *username, unsigned char fingerprint[20])
{
cerr << "myotr_new_fingerprint(top)" << endl;
char our_fingerprint[45];
if( otrl_privkey_fingerprint( us, our_fingerprint, accountname, protocol) )
{
cerr << "myotr_new_fingerprint() our human fingerprint:" << embold( our_fingerprint ) << endl;
}
cerr << "myotr_new_fingerprint() their human fingerprint:"
<< embold( fingerprint_hash_to_human( fingerprint )) << endl;
cerr << "do the fingerprints match at the remote end (enter YES to proceed)" << endl;
std::string reply;
getline( cin, reply );
if( reply != "YES" )
{
cerr << "You have chosen not to continue to talk to these people... good bye." << endl;
exit(0);
}
}
Simpler authentication with SMP
The second example uses the SMP to avoid having to verify
fingerprints. For good measure, the fingerprints established are
saved and loaded to/from disk so that subsequent conversations do
not need any SMP or user fingerprint verification.
During process startup, fingerprints are read from file if they exist;
std::stringstream fn;
fn << "fingerprints-" << accountname;
gcry_error_t e = otrl_privkey_read_fingerprints( userstate, fn.str().c_str(), 0, 0 );
The otrl_message_sending() and otrl_message_receiving() functions
both have a parameter
OtrlTLV *tlvs. The tlvs allow data to be
sent and received as sideband information that does not effect what
you send with libotr. The SMP uses the tlvs to communicate the
information that it needs in order to authenticate.
In server_session() the main change is a check on the tlvs variable
after calling otrl_message_receiving().
if( tlvs )
{
handle_smp( stream, tlvs, userstate, &extended_ui_ops, opdata );
}
The client initiates the SMP and has heavier changes to it's code.
After creating a iosockstream to localhost, the client calls
run_smp_client() to setup the OTR session and run the SMP to
authenticate. Apart from the call to run_smp_client() the client
mainloop while(true) doesn't need to change. This makes sense
because the SMP is normally only used at session establishment when
we do not know about the remote key (fingerprint) already.
In the run_smp_client function, the first
while( !secure... loop
will establish an OTR session using fingerprints just like the first
example. This time we do not stop to ask the user to verify the
fingerprints, we simply record that a new fingerprint was
seen. This is done by setting runSMP=true to force the SMP if we are
using a fingerprint that we didn't already have on disk.
If runSMP is set then we read a secret from the user and call
otrl_message_initiate_smp() to get the SMP ball rolling with
libotr. This leads to the second
while( !secure loop which will
stop when we are secure again.
void run_smp_client( iosockstream& stream )
{
void* opdata = &stream;
OtrlTLV* tlvs = 0;
// establish session using fingerprints
stream << "?OTR?v2?" << endl;
usleep( 200 * 1000 );
while( !secure && stream.peek() != std::iostream::traits_type::eof() )
client_read_msg_from_server( stream );
if( !runSMP )
{
return;
}
VMSG << "Starting the Socialist Millionaires' Protocol " << endl
<< " to work out who the other guy is..." << endl
<< endl;
VMSG << "please give me a secret that only you and the other guy know..." << endl;
std::string s;
getline( cin, s );
int add_if_missing = true;
int addedp = 0;
ConnContext* smpcontext = otrl_context_find( userstate,
recipientname, accountname, protocol,
add_if_missing, &addedp,
myotr_add_appdata, &ui_ops );
cerr << "addedp:" << addedp << " smpcontext:" << smpcontext << endl;
if( !smpcontext )
return;
otrl_message_initiate_smp( userstate, &ui_ops, opdata, smpcontext,
(const unsigned char*)s.c_str(), s.length() );
// we are only secure if the SMP succeeds
secure = 0;
while( !secure && stream.peek() != std::iostream::traits_type::eof() )
client_read_msg_from_server( stream );
cerr << "secure:" << secure << endl;
if( secure == SMP_BAD )
{
cerr << "couldn't authenticate server, exiting..." << endl;
exit(1);
}
}
The client_read_msg_from_server() function calls
otrl_message_receiving() and checks if tlvs is set and if so calls
handle_smp() with that tlvs value.
As you see from the above, whenever a tlvs is set in the client or
server then handle_smp() is called. If you look at the UPGRADING
file in libotr 3.2.0+ you will see a skeleton code in "3.3.4.
Control Flow and Errors" which the handle_smp() is based on. The
handle_smp() function uses otrl_tlv_find() on tlvs to check for
internal OTR messages sent from libotr itself which describe a stage
in the SMP. handle_smp() is like a primitive state machine working
through from SMP1 (the server asking for the secret to respond to
the client's initial request), through to SMP3 and SMP4 which are
called when the protocol completes with either success or failure
(same or different secrets).
if( tlv = otrl_tlv_find(tlvs, OTRL_TLV_SMP2))
{
if (nextMsg != OTRL_SMP_EXPECT2)
{
cerr << "smp: spurious SMP2 received, aborting" << endl;
otrl_message_abort_smp( userstate, ui_ops, opdata, smpcontext);
otrl_sm_state_free(smpcontext->smstate);
}
else
{
cerr << embold("SMP2 received, otrl_message_receiving will have sent SMP3") << endl;
smpcontext->smstate->nextExpected = OTRL_SMP_EXPECT4;
}
}
If the secrets are proven to be the same when the SMP is used it is
adventagious to save the fingerprints to disk so that future
communications do not require user fingerprint verificaiton or the
SMP.
if( tlv = otrl_tlv_find(tlvs, OTRL_TLV_SMP4)
|| tlv = otrl_tlv_find(tlvs, OTRL_TLV_SMP3))
{
if( smpcontext->smstate->sm_prog_state == OTRL_SMP_PROG_SUCCEEDED )
{
std::stringstream fn;
fn << "fingerprints-" << accountname;
gcry_error_t e = otrl_privkey_write_fingerprints( userstate, fn.str().c_str() );
}
}
Hopefully you are now in a better position to add libotr support to
your real time network programs. The full source code to these
programs as well as the HTML for this post itself is up on
my
github page. Remeber, using off the record messaging doesn't
nessesarily mean you have anything to hide, just that you have
nothing to show.
