Page MenuHomePhabricator

curve25519-sha256 problems under Valgrind on i386
Open, Needs TriagePublic

Description

Here's a trivial server:

#include <libssh/libssh.h>
#include <libssh/server.h>
#include <libssh/callbacks.h>
#include <stdio.h>
#include <assert.h>
#include <stdlib.h>
#include <errno.h>

int
main (int argc, char **argv)
{
  ssh_bind sshbind;
  ssh_event event;
  ssh_session session;
  int port = 4321;
  int r;

  event = ssh_event_new ();
  assert (event != NULL);

  sshbind = ssh_bind_new ();
  session = ssh_new ();

  ssh_bind_options_set (sshbind, SSH_BIND_OPTIONS_BINDADDR, "127.0.0.1");
  ssh_bind_options_set (sshbind, SSH_BIND_OPTIONS_BINDPORT, &port);
  ssh_bind_options_set (sshbind, SSH_BIND_OPTIONS_RSAKEY, "ssh_host_rsa_key");

  /* Broken:     "curve25519-sha256"  (default)
   * Workaround: "ecdh-sha2-nistp256" (for example)
   */
  if (argc > 1)
    ssh_options_set (session, SSH_OPTIONS_KEY_EXCHANGE, argv[1]);

  if (ssh_bind_listen (sshbind) < 0)
    {
      fprintf (stderr, "couldn't listen on socket: %s", ssh_get_error (sshbind));
      return 1;
    }

  r = ssh_bind_accept (sshbind, session);
  if (r == SSH_ERROR)
    {
      fprintf (stderr, "accepting connection failed: %s", ssh_get_error (sshbind));
      return 1;
    }

  if (ssh_handle_key_exchange (session))
    {
      const char *msg = ssh_get_error (session);
      if (!strstr (msg, "_DISCONNECT"))
        fprintf (stderr, "key exchange failed: %s", msg);
      return 1;
    }

  if (ssh_event_add_session (event, session) != SSH_OK)
    abort ();

  do
    {
      ssh_event_dopoll (event, 10000);
    }
  while (ssh_is_connected (session));

  ssh_event_remove_session (event, session);
  ssh_event_free (event);
  ssh_free (session);
  ssh_bind_free (sshbind);

  return 0;
}

Running this under valgrind on i386 causes a connecting ssh to fail. Here's what `ssh -v -p 4321 localhost' looks like:

OpenSSH_7.9p1 Ubuntu-10, OpenSSL 1.1.1b  26 Feb 2019
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 19: Applying options for *
debug1: Connecting to localhost [127.0.0.1] port 4321.
debug1: Connection established.
debug1: identity file /home/builder/.ssh/id_rsa type -1
debug1: identity file /home/builder/.ssh/id_rsa-cert type -1
debug1: identity file /home/builder/.ssh/id_dsa type -1
debug1: identity file /home/builder/.ssh/id_dsa-cert type -1
debug1: identity file /home/builder/.ssh/id_ecdsa type -1
debug1: identity file /home/builder/.ssh/id_ecdsa-cert type -1
debug1: identity file /home/builder/.ssh/id_ed25519 type -1
debug1: identity file /home/builder/.ssh/id_ed25519-cert type -1
debug1: identity file /home/builder/.ssh/id_xmss type -1
debug1: identity file /home/builder/.ssh/id_xmss-cert type -1
debug1: Local version string SSH-2.0-OpenSSH_7.9p1 Ubuntu-10
debug1: Remote protocol version 2.0, remote software version libssh_0.8.6
debug1: no match: libssh_0.8.6
debug1: Authenticating to localhost:4321 as 'builder'
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: algorithm: curve25519-sha256
debug1: kex: host key algorithm: rsa-sha2-512
debug1: kex: server->client cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: none
debug1: kex: client->server cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: none
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug1: Server host key: ssh-rsa SHA256:XQ8a7zGxMFstDrGecBRUP9OMnOUXd/T3vkNGtYShs2w
debug1: Host '[localhost]:4321' is known and matches the RSA host key.
debug1: Found key in /home/builder/.ssh/known_hosts:4
ssh_dispatch_run_fatal: Connection to 127.0.0.1 port 4321: incorrect signature

Running the test server with the argument ecdh-sha2-nistp256 (disabling curve25519-sha256) is an effective workaround.

This bug is present on Ubuntu 19.04 libssh 0.8.6-3, linked against libssl 1.1.1b-1ubuntu2, but is not present on Ubuntu 16.04 libssh 0.6.3-4.3ubuntu0.2 linked against libssl 1.0.2g-1ubuntu4.15.

Event Timeline

For the record, this also happens with the latest 0.8.7. I added a reproducer to the Debian package, to be able to track this easily.

Jakuje added a subscriber: Jakuje.Wed, Jun 19, 1:17 PM

I just installed Fedora 27 i386 image (I was not successful with installing anything newer since it is secondary architecture for several releases and receives close to none testing) and current libssh master and I can not reproduce your issue.

Running your reproducer under valgrind or not, results in successful hostkey verification in all the cases.

Note, that I have OpenSSH_7.5p1 and OpenSSH_7.6p1 (built against openssl-1.1.0i-1.fc27.i686 but that should not matter here) client at this moment, but I do not think it should matter a lot. Can you retest this issue is still present? If so, I will try with newer OpenSSH, but in any case, I will probably need something more to reproduce the issue.

I tried also the 0.8.7 branch as reported in the previous comment, but with the same result. Could it be an issue in valgrind or some other part of the toolchain?

I am using the following verions:
valgrind-3.14.0-1.fc27.i686
gcc-7.3.1-6.fc27.i686

The same thing works for me with the current openssh-portable master that I just built in my Fedora VM. Could this be somehow related to the Ubuntu toolchains or packages, rather than to the 32 bits itsef?

@Jakuje : FTR, we test this in i386 chroots or containers, the base system is 64 bit everywhere. It could be specific to the Debian/Ubuntu valgrind version or toolchain of course.

First I tried to reproduce this in Fedora 30 i386, in mock. Modify the example code above to read the host key from "/tmp/etc/ssh/ssh_host_rsa_key". Then:

mock -r fedora-30-i386 -i libssh-devel valgrind gcc openssh-server
mock -r fedora-30-i386 --shell --unpriv
mkdir -p /tmp/etc/ssh
ssh-keygen -A -f /tmp
gcc -o /tmp/s server.c -lssh
valgrind /tmp/s

Then ssh -v -p 4321 localhost (from a different shell outside seems to work fine. So there's something more subtle going on.

Trying the same on Debian (unfortunately in Fedora there aren't nice tools to build chroots, so doing this manually):

sudo debootstrap --arch=i386 unstable /var/tmp/unstable-i386
sudo chroot /var/tmp/unstable-i386/

# and in the chroot
apt install -y build-essential libssh-dev valgrind openssh-server
mount -t proc proc /proc
su -s /bin/bash - nobody  # don't run build and test as root
cd /tmp/

mkdir -p /tmp/etc/ssh
ssh-keygen -A -f /tmp
gcc -o /tmp/s server.c -lssh
valgrind /tmp/s

This reproduces the bug.

Thank you for the clarification and updated reproducer. I can reproduce it with the latest version installed by the package manager in Debian (0.8.7), but I can not reproduce it when I build the example against current master. I can not reproduce it even if I manually checkout the version 0.8.7 from git.

I am not very familiar with the Debian build process so I do not know what flags were used to build the libssh there, but I really thing this is something in there.

Some more tests really point out to some issue in the curve25519 handling the key exchange, since the signature itself on i386 looks fine (it is also fine using other kex algorithms) and I do not see there any obvious issue in the code.

I tried to rebuild libssh using the guide [1], but without any success. I would like to add -DDEBUG_CRYPTO to the debian build process and generate some more verbose logs and compare some values with the client from

/examples/ssh-client -p 4321 127.0.0.1

If you have a Debian machine around and you can collect these logs, it would be appreciated. I am mostly interested in secret hash: and Hash to be signed with RSA: and Hash to be verified with RSA: from a single session.

[1] https://wiki.debian.org/BuildingAPackage