Ambari – user can log in but has no privileges

November 28, 2019November 28, 2019 by Tom

We ran into some interesting issue – user has all the privileges, was in correct ldap group, can log in to the ambari, but privileges weren’t effective.

I did multiple discoveries – checked log ambari-server.log and found out following

com.google.common.cache.CacheLoader$InvalidCacheLoadException: CacheLoader returned null for key myuser.

1	com.google.common.cache.CacheLoader$InvalidCacheLoadException: CacheLoader returned null for key myuser.

So, tried to restart ambari-server, didn’t work. Did further investigation, using ambari API

$ curl -XGET -H"X-Requested-By: ambari" -u admin:... https://ambari.mycluster.com/api/v1/users/myuser
{
  "href" : "http://ambari.mycluster.com/api/v1/users/myuser",
  "Users" : {
    "active" : true,
    "admin" : false,
    "groups" : [
      "ldap-users",
      "cluster-admins"
    ],
    "ldap_user" : false,
    "user_name" : "MYUSER",
    "user_type" : "PAM"
  },

$ curl -XGET -H"X-Requested-By: ambari" -u admin:... https://ambari.mycluster.com/api/v1/users/myuser

{

"href" : "http://ambari.mycluster.com/api/v1/users/myuser",

"Users" : {

"active" : true,

"admin" : false,

"groups" : [

"ldap-users",

"cluster-admins"

"ldap_user" : false,

"user_name" : "MYUSER",

"user_type" : "PAM"

so, this worked. Notice uppercase in user_name. So I tried to fetch user info with privileges

$ curl -XGET -H"X-Requested-By: ambari" -u admin:... https://ambari.mycluster.com/api/v1/users/myuser?fields=privileges/*
{
  "href" : "http://ambari.mycluster.com/api/v1/users/myuser?fields=privileges/*",
  "Users" : {
    "user_name" : "MYUSER"
  },
  "privileges" : [
    {
      "href" : "http://ambari.mycluster.com/api/v1/users/myuser/privileges/153",
      "PrivilegeInfo" : {
        "instance_name" : "INSTANCE",
        "permission_label" : "View User",
        "permission_name" : "VIEW.USER",
        "principal_name" : "cluster-admins",
        "principal_type" : "GROUP",
        "privilege_id" : 153,
        "type" : "VIEW",
        "user_name" : "MYUSER",
        "version" : "2.4.3.0",
        "view_name" : "ADMIN_VIEW"
      }
    },

$ curl -XGET -H"X-Requested-By: ambari" -u admin:... https://ambari.mycluster.com/api/v1/users/myuser?fields=privileges/*

{

"href" : "http://ambari.mycluster.com/api/v1/users/myuser?fields=privileges/*",

"Users" : {

"user_name" : "MYUSER"

"privileges" : [

{

"href" : "http://ambari.mycluster.com/api/v1/users/myuser/privileges/153",

"PrivilegeInfo" : {

"instance_name" : "INSTANCE",

"permission_label" : "View User",

"permission_name" : "VIEW.USER",

"principal_name" : "cluster-admins",

"principal_type" : "GROUP",

"privilege_id" : 153,

"type" : "VIEW",

"user_name" : "MYUSER",

"version" : "2.4.3.0",

"view_name" : "ADMIN_VIEW"

}

Nice. Works too. So another check – load only certain privileges – 153.

$ curl -XGET -H"X-Requested-By: ambari" -u admin:... https://ambari.mycluster.com/api/v1/users/myuser/privileges/153
{
  "status": 500,
  "message": "Server Error"
}

and even upcase

$ curl -XGET -H"X-Requested-By: ambari" -u admin:... https://ambari.mycluster.com/api/v1/users/MYUSER/privileges/153
{
  "status": 500,
  "message": "Server Error"
}

$ curl -XGET -H"X-Requested-By: ambari" -u admin:... https://ambari.mycluster.com/api/v1/users/myuser/privileges/153

{

"status": 500,

"message": "Server Error"

}

and even upcase

$ curl -XGET -H"X-Requested-By: ambari" -u admin:... https://ambari.mycluster.com/api/v1/users/MYUSER/privileges/153

{

"status": 500,

"message": "Server Error"

}

Aha! Looks like there’s some inconsitency in the postgres database. So, check it!

ambari=> select * from users where user_name='myuser';
 user_id | principal_id | ldap_user | user_name | create_time | user_password | active | active_widget_layouts | user_type
---------+--------------+-----------+-----------+-------------+---------------+--------+-----------------------+-----------
(0 rows)

ambari=> select * from users where user_name='MYUSER';
 user_id | principal_id | ldap_user | user_name |      create_time       | user_password | active | active_widget_layouts | user_type
---------+--------------+-----------+-----------+------------------------+---------------+--------+-----------------------+-----------
   16005 |        20005 |         0 | MYUSER  | 2019-11-22 04:37:44.18 |               |      1 | [{"id":"29405"}]      | PAM
(1 row)

ambari=> select * from users where user_name='myuser';

---------+--------------+-----------+-----------+-------------+---------------+--------+-----------------------+-----------

(0 rows)

ambari=> select * from users where user_name='MYUSER';

---------+--------------+-----------+-----------+------------------------+---------------+--------+-----------------------+-----------

16005 | 20005 | 0 | MYUSER | 2019-11-22 04:37:44.18 | | 1 | [{"id":"29405"}] | PAM

(1 row)

Gotcha. There’s no such user named myuser in the database – and pg is case sensitive. So, rename the user

ambari=> update users set user_name='myuser' where user_id=16005;
UPDATE 1
ambari=> \q

ambari=> update users set user_name='myuser' where user_id=16005;

UPDATE 1

ambari=> \q

and restart ambari-server. Now we can try again the api call

$ curl -XGET -H"X-Requested-By: ambari" -u admin:... https://ambari.mycluster.com/api/v1/users/myuser/privileges/153
{
  "href" : "http://ambari.mycluster.com/api/v1/users/myuser/privileges/153",
  "PrivilegeInfo" : {
    "instance_name" : "INSTANCE",
    "permission_label" : "View User",
    "permission_name" : "VIEW.USER",
    "principal_name" : "cluster-admins",
    "principal_type" : "GROUP",
    "privilege_id" : 153,
    "type" : "VIEW",
    "user_name" : "myuser",
    "version" : "2.4.3.0",
    "view_name" : "ADMIN_VIEW"
  }
}

$ curl -XGET -H"X-Requested-By: ambari" -u admin:... https://ambari.mycluster.com/api/v1/users/myuser/privileges/153

{

"href" : "http://ambari.mycluster.com/api/v1/users/myuser/privileges/153",

"PrivilegeInfo" : {

"instance_name" : "INSTANCE",

"permission_label" : "View User",

"permission_name" : "VIEW.USER",

"principal_name" : "cluster-admins",

"principal_type" : "GROUP",

"privilege_id" : 153,

"type" : "VIEW",

"user_name" : "myuser",

"version" : "2.4.3.0",

"view_name" : "ADMIN_VIEW"

}

Voila!

Foxtrot: Programy pro ČOV

July 12, 2019July 12, 2019 by Tom

Program pro řídící jednotku Foxtrot a čistírnu odpadních vod Aquatec AT-6. Nabízí 10 programů, dle zatížení domácnosti. Dá se později napojit na vodoměr (počitadlo) a podle toho měnit program.

TYPE
  T_COVProgram :STRUCT
    runs: ARRAY[0..23] OF usint;
  END_STRUCT
END_TYPE
VAR_GLOBAL
  COV_Programs: ARRAY[1..10] OF T_COVProgram := [
  // cas - hodina rozdelena na 12 useku po 5min.
  // hodnoty: 1 -> 1 min bezi, 5-1=4 stoji
//            0  1  2  3  4  5  6  7  8  9 10 11 12 13 14 15 16 17 18 19 20 21 22 23
// 1 - 5.6h celkem, 4 min stoji, 1 min bezi
    (runs := [5, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]),
// 2 - 8.8h celkem, 4 min stoji, 1 min bezi
    (runs := [5, 5, 5, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 5, 5]),
// 3 - 10.4h celkem, 4 min stoji, 1 min bezi
    (runs := [5, 5, 5, 5, 5, 1, 1, 1, 1, 1, 5, 1, 1, 1, 1, 1, 5, 1, 1, 1, 1, 1, 1, 1]),
// 4 - 12h celkem, 4 min stoji, 1 min bezi
    (runs := [5, 5, 5, 5, 5, 1, 1, 1, 1, 1, 5, 1, 1, 1, 1, 1, 5, 1, 1, 1, 1, 1, 5, 5]),
// 5 - 15h celkem, 3 min stoji, 2 min bezi
    (runs := [5, 5, 5, 5, 5, 2, 2, 2, 2, 2, 5, 2, 2, 2, 2, 2, 5, 2, 2, 2, 2, 2, 5, 5]),
// 6 - 18h celkem, 2 min stoji, 3 min bezi
    (runs := [5, 5, 5, 5, 5, 3, 3, 3, 3, 3, 5, 3, 3, 3, 3, 3, 5, 3, 3, 3, 3, 3, 5, 5]),
// 7 - 20.4h celkem, 2 min stoji, 3 min bezi
    (runs := [5, 5, 5, 5, 5, 5, 3, 3, 3, 5, 5, 5, 3, 3, 3, 5, 5, 5, 3, 3, 3, 5, 5, 5]),
// 8 - 21.6h celkem, 2 min stoji, 3 min bezi
    (runs := [5, 5, 5, 5, 5, 5, 5, 3, 5, 5, 5, 5, 5, 3, 5, 5, 5, 5, 5, 3, 5, 5, 5, 5]),
// 9 - 22.8h celkem, 2 min stoji, 3 min bezi
    (runs := [5, 5, 5, 5, 5, 5, 5, 3, 5, 5, 5, 5, 5, 3, 5, 5, 5, 5, 5, 3, 5, 5, 5, 5]),
// 10 - 24h celkem, 24h bezi
    (runs := [5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5])
   ];
END_VAR
VAR_GLOBAL RETAIN
  COV_PROGRAM : usint := 0;  
END_VAR

TYPE

T_COVProgram :STRUCT

runs: ARRAY[0..23] OF usint;

END_STRUCT

END_TYPE

VAR_GLOBAL

COV_Programs: ARRAY[1..10] OF T_COVProgram := [

// cas - hodina rozdelena na 12 useku po 5min.

// hodnoty: 1 -> 1 min bezi, 5-1=4 stoji

// 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23

// 1 - 5.6h celkem, 4 min stoji, 1 min bezi

(runs := [5, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]),

// 2 - 8.8h celkem, 4 min stoji, 1 min bezi

(runs := [5, 5, 5, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 5, 5]),

// 3 - 10.4h celkem, 4 min stoji, 1 min bezi

(runs := [5, 5, 5, 5, 5, 1, 1, 1, 1, 1, 5, 1, 1, 1, 1, 1, 5, 1, 1, 1, 1, 1, 1, 1]),

// 4 - 12h celkem, 4 min stoji, 1 min bezi

(runs := [5, 5, 5, 5, 5, 1, 1, 1, 1, 1, 5, 1, 1, 1, 1, 1, 5, 1, 1, 1, 1, 1, 5, 5]),

// 5 - 15h celkem, 3 min stoji, 2 min bezi

(runs := [5, 5, 5, 5, 5, 2, 2, 2, 2, 2, 5, 2, 2, 2, 2, 2, 5, 2, 2, 2, 2, 2, 5, 5]),

// 6 - 18h celkem, 2 min stoji, 3 min bezi

(runs := [5, 5, 5, 5, 5, 3, 3, 3, 3, 3, 5, 3, 3, 3, 3, 3, 5, 3, 3, 3, 3, 3, 5, 5]),

// 7 - 20.4h celkem, 2 min stoji, 3 min bezi

(runs := [5, 5, 5, 5, 5, 5, 3, 3, 3, 5, 5, 5, 3, 3, 3, 5, 5, 5, 3, 3, 3, 5, 5, 5]),

// 8 - 21.6h celkem, 2 min stoji, 3 min bezi

(runs := [5, 5, 5, 5, 5, 5, 5, 3, 5, 5, 5, 5, 5, 3, 5, 5, 5, 5, 5, 3, 5, 5, 5, 5]),

// 9 - 22.8h celkem, 2 min stoji, 3 min bezi

(runs := [5, 5, 5, 5, 5, 5, 5, 3, 5, 5, 5, 5, 5, 3, 5, 5, 5, 5, 5, 3, 5, 5, 5, 5]),

// 10 - 24h celkem, 24h bezi

(runs := [5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5])

];

END_VAR

VAR_GLOBAL RETAIN

COV_PROGRAM : usint := 0;

END_VAR


FUNCTION_BLOCK fbCistickaProgram
  VAR_INPUT
    prog_num: usint;
  END_VAR
  VAR_OUTPUT
    output: BOOL;
  END_VAR
  VAR_IN_OUT
  END_VAR
  VAR
    ton_stop: ton;
    ton_run: ton;
    light: fb_iLight;
    curr_time: time;
    hour: usint;
    covprg : T_COVProgram;
    mins : usint;
  END_VAR

curr_time := GetTime();
hour := hour_of_time(in := curr_time);
covprg := COV_Programs[prog_num];
mins := covprg.runs[hour];

ton_stop( IN := not ton_run.q, pt := encodeTime(M := 5-mins));
ton_run( IN := ton_stop.q, pt := encodeTime(M:= mins));

light( lightOn := ton_stop.q, lightOff := ton_run.q, name:= 'Cisticka', out => output );
END_FUNCTION_BLOCK

FUNCTION_BLOCK fbCistickaProgram

VAR_INPUT

prog_num: usint;

END_VAR

VAR_OUTPUT

output: BOOL;

END_VAR

VAR_IN_OUT

END_VAR

VAR

ton_stop: ton;

ton_run: ton;

light: fb_iLight;

curr_time: time;

hour: usint;

covprg : T_COVProgram;

mins : usint;

END_VAR

curr_time := GetTime();

hour := hour_of_time(in := curr_time);

covprg := COV_Programs[prog_num];

mins := covprg.runs[hour];

ton_stop( IN := not ton_run.q, pt := encodeTime(M := 5-mins));

ton_run( IN := ton_stop.q, pt := encodeTime(M:= mins));

light( lightOn := ton_stop.q, lightOff := ton_run.q, name:= 'Cisticka', out => output );

END_FUNCTION_BLOCK

PROGRAM prgJson
  VAR
    cisticka : fbCistickaProgram;
  END_VAR

if COV_PROGRAM = 0 then
  COV_PROGRAM := 1;
end_if;

cisticka( prog_num := COV_PROGRAM, output => RO04_COV );
END_PROGRAM

PROGRAM prgJson

VAR

cisticka : fbCistickaProgram;

END_VAR

if COV_PROGRAM = 0 then

COV_PROGRAM := 1;

end_if;

cisticka( prog_num := COV_PROGRAM, output => RO04_COV );

END_PROGRAM

Jsou tam ještě nějaké záseky, např. že pokud je cyklus plný (5 minut) a následuje ten samý (bez pauzy), bylo by fajn vynechat úplně vypínání výstupu, popř. zkrátit cyklus o párset ms. Dále pak přidat reset v případě změny programu atd. Každopádně to ale zatím funguje k mé spokojenosti.

Protected: Gradující demence vedoucích prací na UZS TUL v Liberci, díl první

June 17, 2016June 17, 2016 by Tom

R – Četnost podruhé

March 19, 2016March 19, 2016 by Tom

Mějme následující data:

> head(dotaznik.csv[,c(2,3)])
  type gender
1    H      F
2    H      M
3    A      F
4    A      M
5    A      F
6    H      M

> head(dotaznik.csv[,c(2,3)])

type gender

1 H F

2 H M

3 A F

4 A M

5 A F

6 H M

A potřebujeme zjistit absolutní a relativní četnost výskytu žen a mužů, rozdělenou ještě podle hodnoty v type .

> tbl <- table(dotaznik.csv$gender, dotaznik.csv$type)
> tbl
   
     A  H
  M 18 16
  Ž 17 19

> tbl <- table(dotaznik.csv$gender, dotaznik.csv$type)

> tbl

A H

M 18 16

Ž 17 19

Opět převedeme na data frame

> tbl.frame <- as.data.frame(tbl)
> tbl.frame
  Var1 Var2 Freq
1    M    A   18
2    Ž    A   17
3    M    H   16
4    Ž    H   19

> tbl.frame <- as.data.frame(tbl)

> tbl.frame

Var1 Var2 Freq

1 M A 18

2 Ž A 17

3 M H 16

4 Ž H 19

A máme absolutní četnost. Teď vypočítáme relativní četnost.

> library(plyr) 
> ddply(tbl.frame, .(Var2), transform, prop=Freq/sum(Freq))
  Var1 Var2 Freq      prop
1    M    A   18 0.5142857
2    Ž    A   17 0.4857143
3    M    H   16 0.4571429
4    Ž    H   19 0.5428571

> library(plyr)

> ddply(tbl.frame, .(Var2), transform, prop=Freq/sum(Freq))

Var1 Var2 Freq prop

1 M A 18 0.5142857

2 Ž A 17 0.4857143

3 M H 16 0.4571429

4 Ž H 19 0.5428571

Což znamená: pro každý subset z framu tbl.frame , rozdělený podle proměnné Var2 , proveď transform , a vypočti relativní četnost podle frekvence (abs. četnosti) a součtu hodnot frekvence.

R – četnost a graf

March 10, 2016March 10, 2016 by Tom

Dostal jsem pověření z nejvyšších míst vyrobit statistické zhodnocení dotazníků. A protože bytostně nemám rád Excel, našel jsem R a zkouším.

Udělal jsem data do CSV souboru a nahrál do R (RStudio)

> head(dotaznik.csv)
  num type gender age height weight           edu
1   1    H      M  60    182    100 Vysokoškolské
2   2    H      M  49    188    102      Vyučen/a
3   3    H      M  61    176     75      Vyučen/a
4   4    H      M  56    180    110       Střední
5   5    H      M  47    180     95 Vysokoškolské
6   6    H      M  48    178     95       Střední

> head(dotaznik.csv)

num type gender age height weight edu

1 1 H M 60 182 100 Vysokoškolské

2 2 H M 49 188 102 Vyučen/a

3 3 H M 61 176 75 Vyučen/a

4 4 H M 56 180 110 Střední

5 5 H M 47 180 95 Vysokoškolské

6 6 H M 48 178 95 Střední

Zkusíme vyrobit četnost vzdělání (edu) a potom ji ještě rozdělit podle pohlaví (gender).

> edu <- table(dotaznik.csv$edu)
> edu

      Střední Vysokoškolské Vyšší odborné      Vyučen/a 
           30             7             2            25 
     Základní 
            6

> edu <- table(dotaznik.csv$edu)

> edu

Střední Vysokoškolské Vyšší odborné Vyučen/a

30 7 2 25

Základní

Máme tabulku, ale potřebujeme z ní frame

> fedu <- as.data.frame(edu)
> fedu
           Var1 Freq
1       Střední   30
2 Vysokoškolské    7
3 Vyšší odborné    2
4      Vyučen/a   25
5      Základní    6

> fedu <- as.data.frame(edu)

> fedu

Var1 Freq

1 Střední 30

2 Vysokoškolské 7

3 Vyšší odborné 2

4 Vyučen/a 25

5 Základní 6

Přidáme relativní četnost (mean)

> fedu$mean <- fedu$Freq / sum(fedu$Freq)
> fedu
           Var1 Freq       mean
1       Střední   30 0.42857143
2 Vysokoškolské    7 0.10000000
3 Vyšší odborné    2 0.02857143
4      Vyučen/a   25 0.35714286
5      Základní    6 0.08571429

> fedu$mean <- fedu$Freq / sum(fedu$Freq)

> fedu

Var1 Freq mean

1 Střední 30 0.42857143

2 Vysokoškolské 7 0.10000000

3 Vyšší odborné 2 0.02857143

4 Vyučen/a 25 0.35714286

5 Základní 6 0.08571429

A teď na graf. Používáme ggplot2.

ggplot(fedu, aes(x=Var1, y=Freq)) + geom_bar(stat='identity')

1	ggplot(fedu, aes(x=Var1, y=Freq)) + geom_bar(stat='identity')

Fajn. Ale chceme graf otočit

ggplot(fedu, aes(x=Var1, y=Freq)) + geom_bar(stat='identity') + coord_flip()

1	ggplot(fedu, aes(x=Var1, y=Freq)) + geom_bar(stat='identity') + coord_flip()

Teď ještě smazat popisek osy x a přepsat osu y.

ggplot(fedu, aes(x=Var1, y=Freq)) + geom_bar(stat='identity') +
coord_flip() +
xlab('Vzdělání') + 
theme(axis.title.x = element_blank()) + 
ggtitle('Dosažené vzdělání')

ggplot(fedu, aes(x=Var1, y=Freq)) + geom_bar(stat='identity') +

coord_flip() +

xlab('Vzdělání') +

theme(axis.title.x = element_blank()) +

ggtitle('Dosažené vzdělání')

A finálně přidáme hodnoty frekvence do jednotlivých sloupců.

ggplot(fedu, aes(x=Var1, y=Freq)) + 
geom_bar(stat='identity', position=position_dodge()) + 
geom_text(aes(label=Freq), hjust=1.3, color="white",position = position_dodge(0.9), size=3.5) + 
coord_flip() + 
xlab('Vzdělání') + 
theme(axis.title.x = element_blank()) + 
ggtitle('Dosažené vzdělání')

ggplot(fedu, aes(x=Var1, y=Freq)) +

geom_bar(stat='identity', position=position_dodge()) +

geom_text(aes(label=Freq), hjust=1.3, color="white",position = position_dodge(0.9), size=3.5) +

coord_flip() +

xlab('Vzdělání') +

theme(axis.title.x = element_blank()) +

ggtitle('Dosažené vzdělání')

A teď trochu komplexněji. Ještě to rozdělíme na muže a ženy..

> edu <- table(dotaznik.csv$gender, dotaznik.csv$edu)
> edu
   
    Střední Vysokoškolské Vyšší odborné Vyučen/a Základní
  M      10             4             2       17        1
  Ž      20             3             0        8        5

> fedu <- as.data.frame(edu)
> fedu
   Var1          Var2 Freq
1     M       Střední   10
2     Ž       Střední   20
3     M Vysokoškolské    4
4     Ž Vysokoškolské    3
5     M Vyšší odborné    2
6     Ž Vyšší odborné    0
7     M      Vyučen/a   17
8     Ž      Vyučen/a    8
9     M      Základní    1
10    Ž      Základní    5

> edu <- table(dotaznik.csv$gender, dotaznik.csv$edu)

> edu

Střední Vysokoškolské Vyšší odborné Vyučen/a Základní

M 10 4 2 17 1

Ž 20 3 0 8 5

> fedu <- as.data.frame(edu)

> fedu

Var1 Var2 Freq

1 M Střední 10

2 Ž Střední 20

3 M Vysokoškolské 4

4 Ž Vysokoškolské 3

5 M Vyšší odborné 2

6 Ž Vyšší odborné 0

7 M Vyučen/a 17

8 Ž Vyučen/a 8

9 M Základní 1

10 Ž Základní 5

a přidáme graf

> ggplot(fedu, aes(x=Var2, y=Freq, fill=Var1)) + 
geom_bar(stat='identity', position=position_dodge()) + 
geom_text(aes(label=Freq), hjust=1.6, color="white",position = position_dodge(0.9), size=3.5) + 
coord_flip() + 
xlab('Vzdělání') + 
theme(axis.title.x = element_blank()) + 
ggtitle('Dosažené vzdělání') + 
labs(fill = 'Pohlaví')

> ggplot(fedu, aes(x=Var2, y=Freq, fill=Var1)) +

geom_bar(stat='identity', position=position_dodge()) +

geom_text(aes(label=Freq), hjust=1.6, color="white",position = position_dodge(0.9), size=3.5) +

coord_flip() +

xlab('Vzdělání') +

theme(axis.title.x = element_blank()) +

ggtitle('Dosažené vzdělání') +

labs(fill = 'Pohlaví')

A co když budeme chtít jeden bar, ale rozdělený podle hodnot?

# seridime podle Var2 a Freq
> library(plyr)
> fedu_s <- arrange(fedu, Var2, Freq)
> fedu_s
   Var1          Var2 Freq
1     M       Střední   10
2     Ž       Střední   20
3     Ž Vysokoškolské    3
4     M Vysokoškolské    4
5     Ž Vyšší odborné    0
6     M Vyšší odborné    2
7     Ž      Vyučen/a    8
8     M      Vyučen/a   17
9     M      Základní    1
10    Ž      Základní    5

# a pridame souhrnny soucet, rozdeleny s kazdym jinym Var2
> fedu_s_sum <- ddply(fedu_s, 'Var2', transform, label_ypos=cumsum(Freq))
> fedu_s_sum
   Var1          Var2 Freq label_ypos
1     M       Střední   10         10
2     Ž       Střední   20         30
3     Ž Vysokoškolské    3          3
4     M Vysokoškolské    4          7
5     Ž Vyšší odborné    0          0
6     M Vyšší odborné    2          2
7     Ž      Vyučen/a    8          8
8     M      Vyučen/a   17         25
9     M      Základní    1          1
10    Ž      Základní    5          6

# seridime podle Var2 a Freq

> library(plyr)

> fedu_s <- arrange(fedu, Var2, Freq)

> fedu_s

Var1 Var2 Freq

1 M Střední 10

2 Ž Střední 20

3 Ž Vysokoškolské 3

4 M Vysokoškolské 4

5 Ž Vyšší odborné 0

6 M Vyšší odborné 2

7 Ž Vyučen/a 8

8 M Vyučen/a 17

9 M Základní 1

10 Ž Základní 5

# a pridame souhrnny soucet, rozdeleny s kazdym jinym Var2

> fedu_s_sum <- ddply(fedu_s, 'Var2', transform, label_ypos=cumsum(Freq))

> fedu_s_sum

Var1 Var2 Freq label_ypos

1 M Střední 10 10

2 Ž Střední 20 30

3 Ž Vysokoškolské 3 3

4 M Vysokoškolské 4 7

5 Ž Vyšší odborné 0 0

6 M Vyšší odborné 2 2

7 Ž Vyučen/a 8 8

8 M Vyučen/a 17 25

9 M Základní 1 1

10 Ž Základní 5 6

A graf

ggplot(fedu_s_sum, aes(x=Var2, y=Freq, fill=Var1)) + 
geom_bar(stat='identity') + 
geom_text(aes(label=Freq, y=label_ypos), hjust=1.6, color="white", size=3.5) + 
coord_flip() + 
xlab('Vzdělání') + 
theme(axis.title.x = element_blank()) + 
ggtitle('Dosažené vzdělání') + 
labs(fill = 'Pohlaví')

ggplot(fedu_s_sum, aes(x=Var2, y=Freq, fill=Var1)) +

geom_bar(stat='identity') +

geom_text(aes(label=Freq, y=label_ypos), hjust=1.6, color="white", size=3.5) +

coord_flip() +

xlab('Vzdělání') +

theme(axis.title.x = element_blank()) +

ggtitle('Dosažené vzdělání') +

labs(fill = 'Pohlaví')

STM32F4Discovery + eLua + OSX

February 22, 2016February 22, 2016 by Tom

I fixed few bugs when compiling elua from git on osx – ~~clone my branch~~ clone official repo here.

$ ./build_elua.lua board=stm32f4discovery

1	$ ./build_elua.lua board=stm32f4discovery

create .bin file

$ arm-none-eabi-objcopy -O binary elua_lua_stm32f4discovery.elf elua_lua_stm32f4discovery.bin

1	$ arm-none-eabi-objcopy -O binary elua_lua_stm32f4discovery.elf elua_lua_stm32f4discovery.bin

create openocd config file for stm32f4 board:

$ cat ~/stm32f4discovery.cfg
# stm32f4discover board
source [find interface/stlink-v2-1.cfg]
transport select hla_swd
source [find target/stm32f4x.cfg]
reset_config srst_only

$ cat ~/stm32f4discovery.cfg

# stm32f4discover board

source [find interface/stlink-v2-1.cfg]

transport select hla_swd

source [find target/stm32f4x.cfg]

reset_config srst_only

and upload to the board:

$ openocd -f ~/stm32f4discovery.cfg \
   -c "init" \   
   -c "reset halt" \
   -c "sleep 100" \
   -c "wait_halt 2" 
   -c "echo \"--- Writing elua_lua_stm32f4discovery.bin\"" \
   -c "flash write_image erase elua_lua_stm32f4discovery.bin 0x08000000" \
   -c "sleep 100" \
   -c "echo \"--- Verifying\"" \
   -c "verify_image elua_lua_stm32f4discovery.bin 0x08000000" \
   -c "sleep 100" \
   -c "echo \"--- Done\"" \
   -c "resume" \
   -c "shutdown"

Open On-Chip Debugger 0.9.0 (2015-11-16-01:48)
Licensed under GNU GPL v2
For bug reports, read
	http://openocd.org/doc/doxygen/bugs.html
Info : The selected transport took over low-level target control. The results might differ compared to plain JTAG/SWD
adapter speed: 2000 kHz
adapter_nsrst_delay: 100
none separate
srst_only separate srst_nogate srst_open_drain connect_deassert_srst
Info : Unable to match requested speed 2000 kHz, using 1800 kHz
Info : Unable to match requested speed 2000 kHz, using 1800 kHz
Info : clock speed 1800 kHz
Info : STLINK v2 JTAG v25 API v2 SWIM v14 VID 0x0483 PID 0x374B
Info : using stlink api v2
Info : Target voltage: 2.884520
Info : stm32f4x.cpu: hardware has 6 breakpoints, 4 watchpoints
target state: halted
target halted due to debug-request, current mode: Thread
xPSR: 0x01000000 pc: 0x080063f0 msp: 0x20000c78
--- Writing elua_lua_stm32f4discovery.bin
auto erase enabled
Info : device id = 0x10076413
Info : flash size = 1024kbytes
wrote 262144 bytes from file elua_lua_stm32f4discovery.bin in 7.783807s (32.889 KiB/s)
--- Verifying
target state: halted
target halted due to breakpoint, current mode: Thread
xPSR: 0x61000000 pc: 0x2000002e msp: 0x20000c78
verified 220776 bytes in 2.229936s (96.685 KiB/s)
--- Done
shutdown command invoked

$ openocd -f ~/stm32f4discovery.cfg \

-c "init" \

-c "reset halt" \

-c "sleep 100" \

-c "wait_halt 2"

-c "echo \"--- Writing elua_lua_stm32f4discovery.bin\"" \

-c "flash write_image erase elua_lua_stm32f4discovery.bin 0x08000000" \

-c "sleep 100" \

-c "echo \"--- Verifying\"" \

-c "verify_image elua_lua_stm32f4discovery.bin 0x08000000" \

-c "sleep 100" \

-c "echo \"--- Done\"" \

-c "resume" \

-c "shutdown"

Open On-Chip Debugger 0.9.0 (2015-11-16-01:48)

Licensed under GNU GPL v2

For bug reports, read

http://openocd.org/doc/doxygen/bugs.html

Info : The selected transport took over low-level target control. The results might differ compared to plain JTAG/SWD

adapter speed: 2000 kHz

adapter_nsrst_delay: 100

none separate

srst_only separate srst_nogate srst_open_drain connect_deassert_srst

Info : Unable to match requested speed 2000 kHz, using 1800 kHz

Info : clock speed 1800 kHz

Info : STLINK v2 JTAG v25 API v2 SWIM v14 VID 0x0483 PID 0x374B

Info : using stlink api v2

Info : Target voltage: 2.884520

Info : stm32f4x.cpu: hardware has 6 breakpoints, 4 watchpoints

target state: halted

target halted due to debug-request, current mode: Thread

xPSR: 0x01000000 pc: 0x080063f0 msp: 0x20000c78

--- Writing elua_lua_stm32f4discovery.bin

auto erase enabled

Info : device id = 0x10076413

Info : flash size = 1024kbytes

wrote 262144 bytes from file elua_lua_stm32f4discovery.bin in 7.783807s (32.889 KiB/s)

--- Verifying

target state: halted

target halted due to breakpoint, current mode: Thread

xPSR: 0x61000000 pc: 0x2000002e msp: 0x20000c78

verified 220776 bytes in 2.229936s (96.685 KiB/s)

--- Done

shutdown command invoked

Now you can connect to the board using /dev/tty.usbmodem621, 115200/8/n/1. You should get this prompt:

eLua#
eLua# help
Shell commands:
  help   - shell help
  lua    - start a Lua session
  ls     - lists files and directories
  dir    - lists files and directories
  cat    - list the contents of a file
  type   - list the contents of a file
  recv   - receive files via XMODEM
  cp     - copy files
  mv     - move/rename files
  rm     - remove files
  ver    - show version information
  mkdir  - create directories
  exit   - exit the shell
For more information use 'help <command>'.
eLua# lua
Press CTRL+Z to exit Lua
Lua 5.1.4  Copyright (C) 1994-2011 Lua.org, PUC-Rio
> print(pd.board() .. "/" .. pd.platform() .. "/" .. pd.cpu())
STM32F4DISCOVERY/STM32F4/STM32F407VG

eLua#

eLua# help

Shell commands:

help - shell help

lua - start a Lua session

ls - lists files and directories

dir - lists files and directories

cat - list the contents of a file

type - list the contents of a file

recv - receive files via XMODEM

cp - copy files

mv - move/rename files

rm - remove files

ver - show version information

mkdir - create directories

exit - exit the shell

For more information use 'help <command>'.

eLua# lua

Press CTRL+Z to exit Lua

> print(pd.board() .. "/" .. pd.platform() .. "/" .. pd.cpu())

STM32F4DISCOVERY/STM32F4/STM32F407VG

enjoy :)

nodemcu – simple webserver

February 11, 2016March 13, 2019 by Tom

I’ve modified and updated this source:

local SSID = "my_wifi_ssid"
local SSID_PASSWORD = "my_wifi_password"

local function http_header(conn)
  conn:send('HTTP/1.1 200 OK\n\n')
  conn:send('<!DOCTYPE HTML>\n')
  conn:send('<html>\n')
  conn:send('<head><meta  content="text/html; charset=utf-8">\n')
  conn:send('<title>ESP8266 znouza test</title></head>\n')
end

local function connect (conn, data)
   local query_data

   conn:on ("receive",
      function (cn, req_data)
         query_data = get_http_req (req_data)
         print (query_data["METHOD"] .. " " .. " " .. query_data["User-Agent"])
         http_header(cn)
         cn:send ("<body>")
         cn:send ("<h1>Hello World from ESP8266 and NodeMCU!!</h1>")
         cn:send ("</body></html>")
         -- Close the connection for the request
         cn:close ( )
      end)
end

function wait_for_wifi_conn ( )
   tmr.alarm (1, 1000, 1, function ( )
      if wifi.sta.getip ( ) == nil then
         print ("Waiting for Wifi connection")
      else
         tmr.stop (1)
         print ("ESP8266 mode is: " .. wifi.getmode ( ))
         print ("The module MAC address is: " .. wifi.sta.getmac ( ))
         print ("Config done, IP is " .. wifi.sta.getip ( ))
      end
   end)
end

-- Build and return a table of the http request data
function get_http_req (instr)
   local t = {}
   local first = nil
   local key, v, strt_ndx, end_ndx

   for str in string.gmatch (instr, "([^\n]+)") do
      -- First line in the method and path
      if (first == nil) then
         first = 1
         strt_ndx, end_ndx = string.find (str, "([^ ]+)")
         v = trim (string.sub (str, end_ndx + 2))
         key = trim (string.sub (str, strt_ndx, end_ndx))
         t["METHOD"] = key
         t["REQUEST"] = v
      else -- Process and remaining ":" fields
         strt_ndx, end_ndx = string.find (str, "([^:]+)")
         if (end_ndx ~= nil) then
            v = trim (string.sub (str, end_ndx + 2))
            key = trim (string.sub (str, strt_ndx, end_ndx))
            t[key] = v
         end
      end
   end

   return t
end

-- String trim left and right
function trim (s)
  return (s:gsub ("^%s*(.-)%s*$", "%1"))
end

-- Configure the ESP as a station (client)
wifi.setmode (wifi.STATION)
wifi.sta.config (SSID, SSID_PASSWORD,1)

-- Hang out until we get a wifi connection before the httpd server is started.
wait_for_wifi_conn ( )

-- Create the httpd server
svr = net.createServer (net.TCP, 30)

-- Server listening on port 80, call connect function if a request is received
svr:listen (80, connect)

local SSID = "my_wifi_ssid"

local SSID_PASSWORD = "my_wifi_password"

local function http_header(conn)

conn:send('HTTP/1.1 200 OK\n\n')

conn:send('<!DOCTYPE HTML>\n')

conn:send('<html>\n')

conn:send('<head><meta content="text/html; charset=utf-8">\n')

conn:send('<title>ESP8266 znouza test</title></head>\n')

end

local function connect (conn, data)

local query_data

conn:on ("receive",

function (cn, req_data)

query_data = get_http_req (req_data)

print (query_data["METHOD"] .. " " .. " " .. query_data["User-Agent"])

http_header(cn)

cn:send ("<body>")

cn:send ("<h1>Hello World from ESP8266 and NodeMCU!!</h1>")

cn:send ("</body></html>")

-- Close the connection for the request

cn:close ( )

end)

end

function wait_for_wifi_conn ( )

tmr.alarm (1, 1000, 1, function ( )

if wifi.sta.getip ( ) == nil then

print ("Waiting for Wifi connection")

else

tmr.stop (1)

print ("ESP8266 mode is: " .. wifi.getmode ( ))

print ("The module MAC address is: " .. wifi.sta.getmac ( ))

print ("Config done, IP is " .. wifi.sta.getip ( ))

end

end)

end

-- Build and return a table of the http request data

function get_http_req (instr)

local t = {}

local first = nil

local key, v, strt_ndx, end_ndx

for str in string.gmatch (instr, "([^\n]+)") do

-- First line in the method and path

if (first == nil) then

first = 1

strt_ndx, end_ndx = string.find (str, "([^ ]+)")

v = trim (string.sub (str, end_ndx + 2))

key = trim (string.sub (str, strt_ndx, end_ndx))

t["METHOD"] = key

t["REQUEST"] = v

else -- Process and remaining ":" fields

strt_ndx, end_ndx = string.find (str, "([^:]+)")

if (end_ndx ~= nil) then

v = trim (string.sub (str, end_ndx + 2))

key = trim (string.sub (str, strt_ndx, end_ndx))

t[key] = v

end

return t

end

-- String trim left and right

function trim (s)

return (s:gsub ("^%s*(.-)%s*$", "%1"))

end

-- Configure the ESP as a station (client)

wifi.setmode (wifi.STATION)

wifi.sta.config (SSID, SSID_PASSWORD,1)

-- Hang out until we get a wifi connection before the httpd server is started.

wait_for_wifi_conn ( )

-- Create the httpd server

svr = net.createServer (net.TCP, 30)

-- Server listening on port 80, call connect function if a request is received

svr:listen (80, connect)

nodemcu + el capitan

February 11, 2016February 11, 2016 by Tom

download and install CH340g driver (check your FTDI chip)
install CoolTerm (http://freeware.the-meiers.org/)
install ESPlorer (http://esp8266.ru/esplorer/)
open CoolTerm, options, choose appropriate /dev/cu.* port and connect

To flash

download firmware
Install esptool

git clone https://github.com/themadinventor/esptool.git 
cd esptool
sudo python ./setup.py install

git clone https://github.com/themadinventor/esptool.git

cd esptool

sudo python ./setup.py install

upgrade firmware

$ python ./esptool.py --port=/dev/cu.wchusbserial620  write_flash  -fm=dio -fs=32m 0x00000 /tmp/nodemcu_integer_0.9.6-dev_20150704.bin

Connecting...
Erasing flash...
Took 3.92s to erase flash block
Wrote 450560 bytes at 0x00000000 in 54.1 seconds (66.6 kbit/s)...

Leaving...

$ python ./esptool.py --port=/dev/cu.wchusbserial620 write_flash -fm=dio -fs=32m 0x00000 /tmp/nodemcu_integer_0.9.6-dev_20150704.bin

Connecting...

Erasing flash...

Took 3.92s to erase flash block

Wrote 450560 bytes at 0x00000000 in 54.1 seconds (66.6 kbit/s)...

Leaving...

iPhone5 power button repaired

February 1, 2016February 22, 2016 by Tom

Common issue. Power button got broken after a while, I switched to iPhone6, so this ‘old’ phone stayed forgotten. Then my son wanted to use it, so I had to repair it. Looked around, for the first time it as eligible to Apple repair program, but after a while my SN was removed, not sure why. Anyway, the off-warranty repair costs more than 2500CZK (w/o VAT), quite a lot. So I looked around and found few links. Ordered spare button/ribbon cable, few tools and .. got it repaired (but I have to admit, it was a tough fight) :)

Price to repair was (excluding tools) around $4.

Things ordered from Aliexpress:

screwdriver (philips etc) with small bits
spudger tool
iSesamo opening tool
switch flex table [spare part only needed]

Cool links to follow when repairing

PS: a small advice at the end. Sort your screws to small groups, on some sticker or a stick tape, put them in that order as removed – as there’s a bunch of really small screws and almost every one differs from others

Side by side images

January 21, 2016 by Tom

Quick script. As I’ve got FLIR images of my house, I wanted to have them side by side – original image + FLIR. So I wrote some small (fast) script in bash to do that.

#!/bin/bash

ir=IR_
img=DC_

im_start=$1
ir_start=$((im_start-1))

echo "Starting from $start - $(printf "%s%04d.jpg" $img $im_start)"
echo "IR will be from $ir_start - $(printf "%s%04d.jpg" $ir $ir_start)"

if [[ ! -f "$(printf "%s%04d.jpg" $img $im_start)" ]]; then
  echo "Image doesn't exist!"
  exit
fi

if [[ ! -f "$(printf "%s%04d.jpg" $ir $ir_start)" ]]; then
  echo "IR doesn't exist!"
  exit
fi

loop=1

while [ $loop -ne 0 ];do

  image=$(printf "%s%04d.jpg" $img $im_start)
  ir_image=$(printf "%s%04d.jpg" $ir $ir_start)

  combined_image=$(printf "flir-%04d-%04d.jpg" $im_start $ir_start)

  echo -n "$image + $ir_image --> $combined"
  montage -mode concatenate -tile x1 $image $ir_image $combined_image
  echo "..Done!"

  # increase
  im_start=$((im_start+2))
  ir_start=$((ir_start+2))

  #echo "Next file $(printf "%s%04d.jpg" $img $im_start)"

  # test if file?
  if [[ ! -f "$(printf "%s%04d.jpg" $img $im_start)" ]]; then
    loop=0
  fi
done

#!/bin/bash

ir=IR_

img=DC_

im_start=$1

ir_start=$((im_start-1))

echo "Starting from $start - $(printf "%s%04d.jpg" $img $im_start)"

echo "IR will be from $ir_start - $(printf "%s%04d.jpg" $ir $ir_start)"

if [[ ! -f "$(printf "%s%04d.jpg" $img $im_start)" ]]; then

echo "Image doesn't exist!"

exit

if [[ ! -f "$(printf "%s%04d.jpg" $ir $ir_start)" ]]; then

echo "IR doesn't exist!"

exit

loop=1

while [ $loop -ne 0 ];do

image=$(printf "%s%04d.jpg" $img $im_start)

ir_image=$(printf "%s%04d.jpg" $ir $ir_start)

combined_image=$(printf "flir-%04d-%04d.jpg" $im_start $ir_start)

echo -n "$image + $ir_image --> $combined"

montage -mode concatenate -tile x1 $image $ir_image $combined_image

echo "..Done!"

# increase

im_start=$((im_start+2))

ir_start=$((ir_start+2))

#echo "Next file $(printf "%s%04d.jpg" $img $im_start)"

# test if file?

if [[ ! -f "$(printf "%s%04d.jpg" $img $im_start)" ]]; then

loop=0

done

And the resulting image:

tom's blog

some thoughts…

Ambari – user can log in but has no privileges

Foxtrot: Programy pro ČOV

Protected: Gradující demence vedoucích prací na UZS TUL v Liberci, díl první

R – Četnost podruhé

R – četnost a graf

STM32F4Discovery + eLua + OSX

nodemcu – simple webserver

nodemcu + el capitan

iPhone5 power button repaired

Side by side images