SOLVED: SSH and Permission denied (publickey,gssapi-keyex,gssapi-with-mic,password)

OpenSSHI ran across the error “Permission denied (publickey,gssapi-keyex,gssapi-with-mic,password).” while ssh’ing to another server today:

$ ssh myhost
Permission denied (publickey,gssapi-keyex,gssapi-with-mic,password).

Usually this means that the permissions of ~/.ssh, ~/.ssh/authorized_keys or your home directory on the other box isn’t setup right The permissions should look like so:

  1. -rwx——. /home/jason
  2. -rwx——. /home/jason/.ssh
  3. -rw——-. /home/jason/.authorized_keys

You would fix with:

$ chmod 0700 ~
$ chmod 0700 ~/.ssh
$ chmod 0600 ~/.ssh/authorized_keys

In my case, the permissions were correct. I ran the ssh command with extra verbose (-v -v)

$ ssh -v -v myhost
OpenSSH_5.3p1, OpenSSL 1.0.1e-fips 11 Feb 2013
debug1: Reading configuration data /home/jason/.ssh/config
debug1: Applying options for *
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: Applying options for *
debug1: /etc/ssh/ssh_config line 62: Deprecated option "RhostsAuthentication"
debug2: ssh_connect: needpriv 0
debug1: Connecting to myhost [192.168.12.6] port 22.
debug1: Connection established.
debug2: key_type_from_name: unknown key type '-----BEGIN'
debug2: key_type_from_name: unknown key type '-----END'
debug1: identity file /home/jason/.ssh/id_rsa type 1
debug1: identity file /home/jason/.ssh/id_rsa-cert type -1
debug2: key_type_from_name: unknown key type '-----BEGIN'
debug2: key_type_from_name: unknown key type '-----END'
debug1: identity file /home/jason/.ssh/id_dsa type 2
debug1: identity file /home/jason/.ssh/id_dsa-cert type -1
debug1: identity file /home/jason/.ssh/id_ecdsa type -1
debug1: identity file /home/jason/.ssh/id_ecdsa-cert type -1
debug1: Remote protocol version 2.0, remote software version OpenSSH_6.0
debug1: match: OpenSSH_6.0 pat OpenSSH*
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_5.3
debug2: fd 3 setting O_NONBLOCK
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug2: kex_parse_kexinit: diffie-hellman-group-exchange-sha256,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha1,diffie-hellman-group1-sha1
debug2: kex_parse_kexinit: ssh-rsa-cert-v01@openssh.com,ssh-dss-cert-v01@openssh.com,ssh-rsa-cert-v00@openssh.com,ssh-dss-cert-v00@openssh.com,ssh-rsa,ssh-dss
debug2: kex_parse_kexinit: aes128-ctr,aes192-ctr,aes256-ctr,arcfour256,arcfour128,aes128-cbc,3des-cbc,blowfish-cbc,cast128-cbc,aes192-cbc,aes256-cbc,arcfour,rijndael-cbc@lysator.liu.se
debug2: kex_parse_kexinit: aes128-ctr,aes192-ctr,aes256-ctr,arcfour256,arcfour128,aes128-cbc,3des-cbc,blowfish-cbc,cast128-cbc,aes192-cbc,aes256-cbc,arcfour,rijndael-cbc@lysator.liu.se
debug2: kex_parse_kexinit: hmac-md5,hmac-sha1,umac-64@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-ripemd160,hmac-ripemd160@openssh.com,hmac-sha1-96,hmac-md5-96
debug2: kex_parse_kexinit: hmac-md5,hmac-sha1,umac-64@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-ripemd160,hmac-ripemd160@openssh.com,hmac-sha1-96,hmac-md5-96
debug2: kex_parse_kexinit: none,zlib@openssh.com,zlib
debug2: kex_parse_kexinit: none,zlib@openssh.com,zlib
debug2: kex_parse_kexinit:
debug2: kex_parse_kexinit:
debug2: kex_parse_kexinit: first_kex_follows 0
debug2: kex_parse_kexinit: reserved 0
debug2: kex_parse_kexinit: ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha1,diffie-hellman-group1-sha1
debug2: kex_parse_kexinit: ssh-rsa,ssh-dss,ecdsa-sha2-nistp256
debug2: kex_parse_kexinit: aes128-ctr,aes192-ctr,aes256-ctr,arcfour256,arcfour128,aes128-cbc,3des-cbc,blowfish-cbc,cast128-cbc,aes192-cbc,aes256-cbc,arcfour,rijndael-cbc@lysator.liu.se
debug2: kex_parse_kexinit: aes128-ctr,aes192-ctr,aes256-ctr,arcfour256,arcfour128,aes128-cbc,3des-cbc,blowfish-cbc,cast128-cbc,aes192-cbc,aes256-cbc,arcfour,rijndael-cbc@lysator.liu.se
debug2: kex_parse_kexinit: hmac-md5,hmac-sha1,umac-64@openssh.com,hmac-sha2-256,hmac-sha2-256-96,hmac-sha2-512,hmac-sha2-512-96,hmac-ripemd160,hmac-ripemd160@openssh.com,hmac-sha1-96,hmac-md5-96
debug2: kex_parse_kexinit: hmac-md5,hmac-sha1,umac-64@openssh.com,hmac-sha2-256,hmac-sha2-256-96,hmac-sha2-512,hmac-sha2-512-96,hmac-ripemd160,hmac-ripemd160@openssh.com,hmac-sha1-96,hmac-md5-96
debug2: kex_parse_kexinit: none,zlib@openssh.com
debug2: kex_parse_kexinit: none,zlib@openssh.com
debug2: kex_parse_kexinit:
debug2: kex_parse_kexinit:
debug2: kex_parse_kexinit: first_kex_follows 0
debug2: kex_parse_kexinit: reserved 0
debug2: mac_setup: found hmac-md5
debug1: kex: server->client aes128-ctr hmac-md5 none
debug2: mac_setup: found hmac-md5
debug1: kex: client->server aes128-ctr hmac-md5 none
debug1: SSH2_MSG_KEX_DH_GEX_REQUEST(1024<1024<8192) sent
debug1: expecting SSH2_MSG_KEX_DH_GEX_GROUP
debug2: dh_gen_key: priv key bits set: 154/256
debug2: bits set: 520/1024
debug1: SSH2_MSG_KEX_DH_GEX_INIT sent
debug1: expecting SSH2_MSG_KEX_DH_GEX_REPLY
Warning: Permanently added 'myhost,192.168.1.66' (RSA) to the list of known hosts.
debug2: bits set: 525/1024
debug1: ssh_rsa_verify: signature correct
debug2: kex_derive_keys
debug2: set_newkeys: mode 1
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug2: set_newkeys: mode 0
debug1: SSH2_MSG_NEWKEYS received
debug1: SSH2_MSG_SERVICE_REQUEST sent
debug2: service_accept: ssh-userauth
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug2: key: /home/jason/.ssh/id_rsa (0x7ff594d8ecb0)
debug2: key: /home/jason/.ssh/id_dsa (0x7ff594d90550)
debug2: key: /home/jason/.ssh/id_ecdsa ((nil))
debug1: Authentications that can continue: publickey,password,keyboard-interactive
debug1: No more authentication methods to try.
Permission denied (publickey,password,keyboard-interactive).

I didn’t see a reason why I wasn’t getting a password prompt but I do see it reading my ssh_config file. A real quick override of the ssh_config showed me that my ssh_config was the culprit:

$ ssh -F /dev/null myhost
jason@myhost's password:

So what is in my ~/.ssh/config file?

ServerAliveInterval 240
BatchMode yes
TCPKeepAlive = yes

Neither ServerAliveInterval or TCPKeepAlive have anything to do with authentication but BatchMode does. From the ssh_config man page:

BatchMode

The argument must be yes or no. If set to yes, passphrase/password querying will be disabled. This option is useful in scripts and other batch jobs where you have no user to supply the password.

So, if my public ssh key is not in the ~/.ssh/authorized_keys, the connection will fail with a permission denied. Let’s verify but removing BatchMode from the ~/.ssh/config file:

ServerAliveInterval 240
TCPKeepAlive = yes
$ ssh -F /dev/null myhost
___$

Success 🙂

Share Button

HOWTO: Find the real and effective users in AIX & Linux when you’re sudo / su’d to another

Retrieving the user that you logged in as while running sudo or su’d into another user can be painful if you don’t have access to root. Here’s a short script that will retrieve the original user that was your session logged in as.

#!/bin/ksh93

OS_NAME=$( uname -s )

if [[ $OS_NAME == "AIX" ]]; then
    typeset var TTY
    REAL_USER=$( TTY=$(tty | sed 's:/dev/::' ) ; ps -t "$TTY" -o ruser=,etime= |sort -r -k2,2 | awk '{ print $1 } ' |head -1 )
elif [[ $OS_NAME = "Linux" ]]; then
    REAL_USER=$( ps T --sort start_time --no-heading -o ruser |head -1 )
else
    echo "ERROR: Requires Linux or AIX"
    exit 1
fi

EFFECTIVE_USER=$( whoami )

echo "I am \"$EFFECTIVE_USER\" but really \"$REAL_USER\""
ssh mybox
..
> sudo su - sybase

AIX Output:

 ./realme.ksh
I am "sybase" but really "jason"

Linux Output:

 ./realme.ksh
I am "sybase" but really "jason"
Share Button

HOWTO: Determine what process is listening on a port (AIX Unix specific)

I needed an easy way to determine which process was listening on a port. For AIX, you need to get the socket id from “netstat -Ana” and use the rmsock “rmsock socket_id tcpcb” to get the PID and command. It would be easy to expand this out to list command line and owner for each PID.

--------------------------------------------------------------------------------------
| Process              | PID             | Protocol | Listening On                   |
--------------------------------------------------------------------------------------
| WEBAPL               |         4915396 |      UDP |                127.0.0.1.32807 |
| WEBAPL               |         4915396 |      UDP |                127.0.0.1.32808 |
| WEBAPL               |        12058770 |      UDP |                127.0.0.1.51714 |
| WEBAPL               |        12058770 |      UDP |                127.0.0.1.51715 |
| backupserver         |        19791994 |      TCP |              192.168.1.4.50021 |
--------------------------------------------------------------------------------------
#!/bin/ksh93

OS_NAME=$( uname -s )

if [[ $OS_NAME == "AIX" ]] ; then
    echo "--------------------------------------------------------------------------------------"
    printf "| %-20s | %-15s | Protocol | %-30s |\n" "Process" "PID" "Listening On";
    echo "--------------------------------------------------------------------------------------"

    netstat -Ana | awk '
    /[0-9\*].[0-9].+LISTEN/ {
        SOCKET=$1;
        IPPORT=$5;
        "rmsock " SOCKET " tcpcb" | getline SOCKOUT;
        split(SOCKOUT, sockarray, " ");
        gsub(/[\.\(\)]/, "", sockarray[10]);
        LISTENERS[ sprintf("| %-20s | %15d | %8s | %30s |", sockarray[10], sockarray[9], "TCP", IPPORT) ] = 1;
    }
    /udp.*.[0-9]/ {
        SOCKET=$1;
        IPPORT=$5;
        "rmsock " SOCKET " inpcb" | getline SOCKOUT;
        split(SOCKOUT, sockarray, " ");
        gsub(/[\.\(\)]/, "", sockarray[10]);
        LISTENERS[ sprintf("| %-20s | %15d | %8s | %30s |", sockarray[10], sockarray[9], "UDP", IPPORT) ] = 1;
    }
    END {
        for (var in LISTENERS)
            print var

    }' | sort | uniq

    echo "--------------------------------------------------------------------------------------"
else
    echo "ERROR: Requires AIX"
    exit 1
fi
Share Button

SAP Sybase IQ: SQLCODE=-1010000, ODBC 3 State=”HY000″: symbolic link to RAW device “already exists”

When you create an instance of IQ, you should use RAW devices for stability and performance. I recently created an instance and ran into the error SQLCODE=-1010000, ODBC 3 State=”HY000″ The file ‘/IQ/myiq/devices/IQ_MAIN/myiqmain001.iq’ already exists on AIX. The symbolic link /IQ/myiq/devices/IQ_MAIN/myiqmain001.iq was pointing to the RAW device /dev/myiqmain001. After many long hours on the phone with SAP, we found the solution, I’m rather embarrassed to say.

On AIX:

  • buffered (cooked) partitions are /dev/myiqmain001
  • unbuffered (REAL) RAW devices are /dev/rmyiqmain001

Note the letter “r” designating a RAW device.

When in doubt, you can use the file utility:
Block Special effectively means cooked partition:

file /dev/myiqmain001 
/dev/myiqmain001: block special (35/1)

Character Special means RAW device:

file /dev/rmyiqmain001
/dev/rmyiqmain001: character special (35/1)

Thanks to John Ting, Richard Weisbrod and Peter Bavin of SAP! It took a long time but we finally figured out what that something really really simple we’re overlooking was.

Feature Request 747716: for IQ to detect non-character raw device during db OR dbfile creation.

Share Button

Howto: Compile VIM 7.3 on AIX 6.x (error: conflicting types for `lseek64′)

If you’ve tried building VIM on AIX 6.x, you’ve probably run into:

In file included from os_unix.h:56,
                 from vim.h:265,
                 from gui_motif.c:36:
/opt/freeware/lib/gcc-lib/powerpc-ibm-aix5.3.0.0/3.3.2/include/unistd.h:178: error: conflicting types for `lseek64'
/opt/freeware/lib/gcc-lib/powerpc-ibm-aix5.3.0.0/3.3.2/include/unistd.h:176: error: previous declaration of `lseek64'
In file included from /opt/freeware/lib/gcc-lib/powerpc-ibm-aix5.3.0.0/3.3.2/include/unistd.h:739,
                 from os_unix.h:56,
                 from vim.h:265,
                 from gui_motif.c:36:
/usr/include/sys/lockf.h:64: error: conflicting types for `lockf64'
/usr/include/sys/lockf.h:62: error: previous declaration of `lockf64'
In file included from os_unix.h:56,
                 from vim.h:265,
                 from gui_motif.c:36:
/opt/freeware/lib/gcc-lib/powerpc-ibm-aix5.3.0.0/3.3.2/include/unistd.h:802: error: conflicting types for `ftruncate64'
/opt/freeware/lib/gcc-lib/powerpc-ibm-aix5.3.0.0/3.3.2/include/unistd.h:800: error: previous declaration of `ftruncate64'
/opt/freeware/lib/gcc-lib/powerpc-ibm-aix5.3.0.0/3.3.2/include/unistd.h:838: error: conflicting types for `truncate64'
/opt/freeware/lib/gcc-lib/powerpc-ibm-aix5.3.0.0/3.3.2/include/unistd.h:836: error: previous declaration of `truncate64'
/opt/freeware/lib/gcc-lib/powerpc-ibm-aix5.3.0.0/3.3.2/include/unistd.h:855: error: conflicting types for `pread64'
/opt/freeware/lib/gcc-lib/powerpc-ibm-aix5.3.0.0/3.3.2/include/unistd.h:852: error: previous declaration of `pread64'
/opt/freeware/lib/gcc-lib/powerpc-ibm-aix5.3.0.0/3.3.2/include/unistd.h:856: error: conflicting types for `pwrite64'
/opt/freeware/lib/gcc-lib/powerpc-ibm-aix5.3.0.0/3.3.2/include/unistd.h:853: error: previous declaration of `pwrite64'
/opt/freeware/lib/gcc-lib/powerpc-ibm-aix5.3.0.0/3.3.2/include/unistd.h:923: error: conflicting types for `fclear64'
/opt/freeware/lib/gcc-lib/powerpc-ibm-aix5.3.0.0/3.3.2/include/unistd.h:920: error: previous declaration of `fclear64'
/opt/freeware/lib/gcc-lib/powerpc-ibm-aix5.3.0.0/3.3.2/include/unistd.h:924: error: conflicting types for `fsync_range64'
/opt/freeware/lib/gcc-lib/powerpc-ibm-aix5.3.0.0/3.3.2/include/unistd.h:921: error: previous declaration of `fsync_range64'
In file included from vim.h:302,
                 from gui_motif.c:36:
auto/osdef.h:120: error: conflicting types for `lseek64'
/opt/freeware/lib/gcc-lib/powerpc-ibm-aix5.3.0.0/3.3.2/include/unistd.h:178: error: previous declaration of `lseek64'
make: The error code from the last command is 1.

Stop.
make: The error code from the last command is 2.

The problem is really an AIX specific change (see http://sourceware.org/bugzilla/show_bug.cgi?id=13558):

Since AIX 6.1, mkdtemp() is declared upon _XOPEN_SOURCE >= 700, and
thus the symbol is available in libc.

The difference is: Upon _ALL_SOURCE (defined in config.h),
AIX 6.1 defines (overrides!) _XOPEN_SOURCE=600, while
AIX 7.1 defines (overrides!) _XOPEN_SOURCE=700,

This particularly isn’t a problem on AIX 7.1, but still the check for mkdtemp()
should be extended to if the declaration is available too.

For the case of VIM, the problem is triggered only when building with Motif support. Since most AIX users are on a server and not a desktop environment, let’s build without it and any other GUI support:

./configure --prefix=/somewhere --without-x --without-gnome --disable-gnome-check --disable-motif-check --disable-athena-check --disable-gui

It should now build and run 🙂

Share Button

Native support of SQLite in Sybase PowerBuilder and PowerDesigner?

What I would love to see is native support for SQLite. http://www.sqlite.org

It is the most installed embedded database on the planet hands down.  Don’t believe me?  You know that Firefox web browser, Thunderbird newsreader, most Adobe products, Miro, etc all have it embedded?  – reference http://www.sqlite.org/mostdeployed.html

We use it extensively at work as:

  1. staging for mass data imports/exports/conversions
  2. local application ‘cache’ for large data sets
  3. projects that don’t require all the features of Sybase ASE (or Oracle for that matter)

One of the best features is that the database itself in platform independent… copy the db on to AIX from your Windows box … then on to your old Amiga … then on to your windows mobile device.  Getting the point?  🙂

I really wish Sybase would make it so that the Sybase ASE databases were truly platform and character set/sort order independent… but that is in another dream 😉

Not a blurb in the PowerDesigner/PowerBuilder manuals or anything

Share Button

How to create a ramdisk on AIX for Sybase

As root:

mkramdisk 2700M
chown sybase /dev/rramdisk0

Granted, the memory would be better off used as a larger cache for tempdb, but some individuals believe that tempdb is faster on a ramdisk (it isn’t).

Share Button