Ambari – user can log in but has no privileges

We ran into some interesting issue – user has all the privileges, was in correct ldap group, can log in to the ambari, but privileges weren’t effective.

I did multiple discoveries – checked log ambari-server.log and found out following

com.google.common.cache.CacheLoader$InvalidCacheLoadException: CacheLoader returned null for key myuser.

So, tried to restart ambari-server, didn’t work. Did further investigation, using ambari API

$ curl -XGET -H"X-Requested-By: ambari" -u admin:... https://ambari.mycluster.com/api/v1/users/myuser
{
  "href" : "http://ambari.mycluster.com/api/v1/users/myuser",
  "Users" : {
    "active" : true,
    "admin" : false,
    "groups" : [
      "ldap-users",
      "cluster-admins"
    ],
    "ldap_user" : false,
    "user_name" : "MYUSER",
    "user_type" : "PAM"
  },

so, this worked. Notice uppercase in user_name. So I tried to fetch user info with privileges

$ curl -XGET -H"X-Requested-By: ambari" -u admin:... https://ambari.mycluster.com/api/v1/users/myuser?fields=privileges/*
{
  "href" : "http://ambari.mycluster.com/api/v1/users/myuser?fields=privileges/*",
  "Users" : {
    "user_name" : "MYUSER"
  },
  "privileges" : [
    {
      "href" : "http://ambari.mycluster.com/api/v1/users/myuser/privileges/153",
      "PrivilegeInfo" : {
        "instance_name" : "INSTANCE",
        "permission_label" : "View User",
        "permission_name" : "VIEW.USER",
        "principal_name" : "cluster-admins",
        "principal_type" : "GROUP",
        "privilege_id" : 153,
        "type" : "VIEW",
        "user_name" : "MYUSER",
        "version" : "2.4.3.0",
        "view_name" : "ADMIN_VIEW"
      }
    },

Nice. Works too. So another check – load only certain privileges – 153.

$ curl -XGET -H"X-Requested-By: ambari" -u admin:... https://ambari.mycluster.com/api/v1/users/myuser/privileges/153
{
  "status": 500,
  "message": "Server Error"
}

and even upcase

$ curl -XGET -H"X-Requested-By: ambari" -u admin:... https://ambari.mycluster.com/api/v1/users/MYUSER/privileges/153
{
  "status": 500,
  "message": "Server Error"
}

Aha! Looks like there’s some inconsitency in the postgres database. So, check it!

ambari=> select * from users where user_name='myuser';
 user_id | principal_id | ldap_user | user_name | create_time | user_password | active | active_widget_layouts | user_type
---------+--------------+-----------+-----------+-------------+---------------+--------+-----------------------+-----------
(0 rows)

ambari=> select * from users where user_name='MYUSER';
 user_id | principal_id | ldap_user | user_name |      create_time       | user_password | active | active_widget_layouts | user_type
---------+--------------+-----------+-----------+------------------------+---------------+--------+-----------------------+-----------
   16005 |        20005 |         0 | MYUSER  | 2019-11-22 04:37:44.18 |               |      1 | [{"id":"29405"}]      | PAM
(1 row)

Gotcha. There’s no such user named myuser in the database – and pg is case sensitive. So, rename the user

ambari=> update users set user_name='myuser' where user_id=16005;
UPDATE 1
ambari=> \q

and restart ambari-server. Now we can try again the api call

$ curl -XGET -H"X-Requested-By: ambari" -u admin:... https://ambari.mycluster.com/api/v1/users/myuser/privileges/153
{
  "href" : "http://ambari.mycluster.com/api/v1/users/myuser/privileges/153",
  "PrivilegeInfo" : {
    "instance_name" : "INSTANCE",
    "permission_label" : "View User",
    "permission_name" : "VIEW.USER",
    "principal_name" : "cluster-admins",
    "principal_type" : "GROUP",
    "privilege_id" : 153,
    "type" : "VIEW",
    "user_name" : "myuser",
    "version" : "2.4.3.0",
    "view_name" : "ADMIN_VIEW"
  }
}

Voila!