iPhone5 power button repaired

Common issue. Power button got broken after a while, I switched to iPhone6, so this ‘old’ phone stayed forgotten. Then my son wanted to use it, so I had to repair it. Looked around, for the first time it as eligible to Apple repair program, but after a while my SN was removed, not sure why. Anyway, the off-warranty repair costs more than 2500CZK (w/o VAT), quite a lot. So I looked around and found few links. Ordered spare button/ribbon cable, few tools and .. got it repaired (but I have to admit, it was a tough fight) :)

Price to repair was (excluding tools) around $4.

Things ordered from Aliexpress:

Cool links to follow when repairing

PS: a small advice at the end. Sort your screws to small groups, on some sticker or a stick tape, put them in that order as removed – as there’s a bunch of really small screws and almost every one differs from others

Side by side images

Quick script. As I’ve got FLIR images of my house, I wanted to have them side by side – original image + FLIR. So I wrote some small (fast) script in bash to do that.

#!/bin/bash

ir=IR_
img=DC_

im_start=$1
ir_start=$((im_start-1))

echo "Starting from $start - $(printf "%s%04d.jpg" $img $im_start)"
echo "IR will be from $ir_start - $(printf "%s%04d.jpg" $ir $ir_start)"

if [[ ! -f "$(printf "%s%04d.jpg" $img $im_start)" ]]; then
  echo "Image doesn't exist!"
  exit
fi

if [[ ! -f "$(printf "%s%04d.jpg" $ir $ir_start)" ]]; then
  echo "IR doesn't exist!"
  exit
fi

loop=1

while [ $loop -ne 0 ];do

  image=$(printf "%s%04d.jpg" $img $im_start)
  ir_image=$(printf "%s%04d.jpg" $ir $ir_start)

  combined_image=$(printf "flir-%04d-%04d.jpg" $im_start $ir_start)

  echo -n "$image + $ir_image --> $combined"
  montage -mode concatenate -tile x1 $image $ir_image $combined_image
  echo "..Done!"

  # increase
  im_start=$((im_start+2))
  ir_start=$((ir_start+2))

  #echo "Next file $(printf "%s%04d.jpg" $img $im_start)"

  # test if file?
  if [[ ! -f "$(printf "%s%04d.jpg" $img $im_start)" ]]; then
    loop=0
  fi
done

And the resulting image:

flir-0941-0940

sidekiq getting stuck?

We had quite strange scenario. Two hosts with sidekiq, one working well, second one gets stuck after few seconds or minutes. Happened in workers with opening new TCP/UDP connections. So, I started to review all the files, where ‘max files’ is set.

$ cat /etc/security/limits.conf
* soft nofile 30000
* hard nofile 60000

tom@web:~$ ulimit -Hn
60000
tom@web:~$ ulimit -Sn
30000

tom@web:~$ /sbin/sysctl -a | grep "file-max"
fs.file-max = 60000

So, all this looks OK so far. After numerous checks I’ve found, that our eye  process is running for a while, maybe since the beginning and probably it took old file-max  settings. How to check? Simply run eye info  or get PID of the sidekiq  process and run following command (assume 22613 is the PID):

tom@web:~$ cat /proc/22613/limits
Limit                     Soft Limit           Hard Limit           Units
Max cpu time              unlimited            unlimited            seconds
Max file size             unlimited            unlimited            bytes
Max data size             unlimited            unlimited            bytes
Max stack size            8388608              unlimited            bytes
Max core file size        0                    unlimited            bytes
Max resident set          unlimited            unlimited            bytes
Max processes             516140               516140               processes
Max open files            1024                 4096                 files
Max locked memory         65536                65536                bytes
Max address space         unlimited            unlimited            bytes
Max file locks            unlimited            unlimited            locks
Max pending signals       516140               516140               signals
Max msgqueue size         819200               819200               bytes
Max nice priority         0                    0
Max realtime priority     0                    0
Max realtime timeout      unlimited            unlimited            us

Gotcha! 1024 as soft-limit, 4096 hard limit. The fix is pretty simple – just quit eye , load it’s config and restart sidekiq.

tom@web:~$ eye quit
quit...

tom@web:~$ eye load config.eye
eye started!
config loaded!

tom@web~$ eye restart sidekiq

tom@web:~$ cat /proc/4624/limits
Limit                     Soft Limit           Hard Limit           Units
Max cpu time              unlimited            unlimited            seconds
Max file size             unlimited            unlimited            bytes
Max data size             unlimited            unlimited            bytes
Max stack size            8388608              unlimited            bytes
Max core file size        0                    unlimited            bytes
Max resident set          unlimited            unlimited            bytes
Max processes             516140               516140               processes
Max open files            30000                60000                files
Max locked memory         65536                65536                bytes
Max address space         unlimited            unlimited            bytes
Max file locks            unlimited            unlimited            locks
Max pending signals       516140               516140               signals
Max msgqueue size         819200               819200               bytes
Max nice priority         0                    0
Max realtime priority     0                    0
Max realtime timeout      unlimited            unlimited            us

voila! We’re back on track :)

sendmail + osx 10.10.4/ios 8.4/ios 9

After upgrade I was unable se to send mails through my sendmail, from both OSX 10.10.4 and upgraded iOS9 …

this was in my sendmail log:

Jul 12 19:58:03 orin sendmail[12164]: STARTTLS=server, error: accept failed=0, SSL_error=1, errno=0, retry=-1
Jul 12 19:58:03 orin sendmail[12164]: STARTTLS=server: 12164:error:14094410:SSL routines:SSL3_READ_BYTES:sslv3 alert handsh
ake failure:s3_pkt.c:1053:SSL alert number 40

fix is amazingly easy and effective:

generate dh2048 certificate:

openssl gendh -out dh_2048.pem -2 2048

and add to sendmail.mc:

define(`confDH_PARAMETERS',`/etc/mail/certs/dh_2048.pem')

then recompile your .mc and restart sendmail.

BCD Boot troubles and WinMagic encryption

I’ve got a notebook to repair – HP ProBook, where Windows boot ended up with an error message – something about BCD. Tried to repair with standard tools, like recovery DVD, but without any success.

It was because main partition with OS was encrypted with WinMagic. So, now what?

Got some tools and info from local HP service, so you need to find our or get:

  • SDWinPEx64.iso
  • SDWmgRecovery package
  • encryption key or password (key is better)

So, now steps how to fix it:

  1. create bootable USB/DVD media with WinPE
  2. unpack and store SDWmgRecovery somewhere (USB flash)
  3. copy key to the same place
  4. boot with WinPE (takes a while, be patient)
  5. first of all, run TB Launcher/SD Recovery
  6. expand your hdd with all the partition and check Lock column
  7. select those with Lock as true (one by one)
  8. click Unlock and enter your password or open a key
  9. back to TB Launcher, open a file manager to check which disk letter belongs to Windows (eg. “d:”)
  10. open Console
  11. type bootsect /nt60 c: /force /mbr
  12. type bcdboot d:\windows /s c:
  13. reboot

It should be working fine now. :-)

Rubymotion – new bug found

I’m using this motion-addressbook in my project. Reported http://hipbyte.myjetbrains.com/youtrack/issue/RM-630

fixed in 2.37


= RubyMotion 2.37 =

* Fixed a regression where Dispatch.once did not work correctly when it was
invoked in some points.
* Fixed a bug where the compiler would crash with an assertion message
`[BUG] Object: SubtypeUntil ...' when compiling certain Ruby files.
* Fixed a bug in the compiler when compiling for ARM64 where certain types
would still be emitted with a 32-bit architecture in mind.
* Fixed a bug in the compiler when compiling for ARM64 where certain structs
would not be properly available (such as NSDecimal).

AWS S3 IAM user policy setup for bucket

First of all, setup your AWS account – and login into AWS Management Console. Then create any bucket you want.

Next step is to add new IAM user – go to https://console.aws.amazon.com/iam/home?#users, select Users and click Create new user button.

iam_create

Click Create and copy/paste or download your access keys.

iam_access

Then close window and click to summary tab – you’ll need to copy ARN notation of this user

iam_summary

 

s3cmd

To be able to sync some folders and/or use s3cmd commandline tool, you need to setup some more policies. Click Permission tab and then Attach User Policy. Then choose Policy Generator and follow these steps:

manage_user_policySet permissions as shown in following image – we need ListAllMyBuckets for :::* and ListBucket and PutObject for the one, specified bucket.

user_policy_editor

 

Click Continue and save new policy

set_permission

You can always edit the policy using Manage Policy. You should create this kind of policy file:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "Stmt1402603222000",
      "Effect": "Allow",
      "Action": [
        "s3:ListAllMyBuckets"
      ],
      "Resource": [
        "arn:aws:s3:::*"
      ]
    },
    {
      "Sid": "Stmt1402603306000",
      "Effect": "Allow",
      "Action": [
        "s3:ListBucket",
        "s3:PutObject"
      ],
      "Resource": [
        "arn:aws:s3:::sample_bucket/*"
      ]
    }
  ]
}

That’s all. Lets move to bucket policy.

Bucket policy

Now you can open Policy Generator – http://awspolicygen.s3.amazonaws.com/policygen.html and start adding new items.

First of all, select your policy type – S3

policy_select_type

and then fill new policy – we need two policies – one for all users to read objects (GetObject) and second for your new user, to be able to upload, delete and get object as well.

public_policy_form

and click Add Statement. The list below the form should look like

public_policy_list

So, add your user the same way. As principal add your ARN notation (which you copied before) and add Actions as shown below.

policy_user

All set. You need to click Generate Policy button and copy freshly generated JSON policy to your clipboard

{
  "Id": "Policy1402594421281",
  "Statement": [
    {
      "Sid": "Stmt1402593998750",
      "Action": [
        "s3:GetObject"
      ],
      "Effect": "Allow",
      "Resource": "arn:aws:s3:::sample_bucket/*",
      "Principal": {
        "AWS": [
          "*"
        ]
      }
    },
    {
      "Sid": "Stmt1402594323930",
      "Action": [
        "s3:AbortMultipartUpload",
        "s3:DeleteObject",
        "s3:GetObject",
        "s3:GetObjectAcl",
        "s3:PutObject",
        "s3:PutObjectAcl"
      ],
      "Effect": "Allow",
      "Resource": "arn:aws:s3:::sample_bucket/*",
      "Principal": {
        "AWS": [
          "arn:aws:iam::024698735884:user/sample_user"
        ]
      }
    }
  ]
}

Last step is to add this policy to the bucket. Go back to your S3 console (https://console.aws.amazon.com/s3/home), click loupe icon next to bucket name and open Permission section. Click on Edit Bucket Policy and paste the generated policy into that window. Save and you’re done.

bucket_permission

 

You’ll need API keys for use with Paperclip or any other S3 storage engine, so keep them safe.

PS: when you plan to use s3 as storage for your static files, avoid using underscore in your bucket name – as it does not comply with valid FQDN. ‘sample_bucket’ was used only as an example, in the real word you should use ‘sample-bucket’.

ATtiny13 – Hello World! :)

Finally managed to get into AVR programming. Using OSX, which I found as the worst platform for doing any kind of embed programming :( Sad. Anyway, there’s my first ATtiny schematics and code. I’m using AVR Dragon to flash code into the MCU.

tiny13-spi

The circuit is quite simple. I just confirm SPI flashing works and I’m able to turn the LED on. So here’s some code (using avr-gcc to compile and avrdure to upload). Create new project using avr-project and add this to main.c file:

#include <avr/io.h>
#include <util/delay.h>

int main(void)
{
    const int msDelay = 50;

    // PortB pin4 to output (set bit to 1 using SHL)
    DDRB = 1<<DDB4;

    // PortB to low
    PORTB = 0;

    while (1) {
        // XOR on pin 4
        PORTB ^= 1<<PB4;
        _delay_ms(msDelay);
    }

    return 0;
}

Now we can compile the firmware and upload to our MCU.

$ make
avr-gcc -Wall -Os -DF_CPU=8000000 -mmcu=attiny13 -c main.c -o main.o
avr-gcc -Wall -Os -DF_CPU=8000000 -mmcu=attiny13 -o main.elf main.o
rm -f main.hex
avr-objcopy -j .text -j .data -O ihex main.elf main.hex
avr-size --format=avr --mcu=attiny13 main.elf
AVR Memory Usage
----------------
Device: attiny13

Program:      72 bytes (7.0% Full)
(.text + .data + .bootloader)

Data:          0 bytes (0.0% Full)
(.data + .bss + .noinit)

No errors. Great. Connect AVR Dragon and upload our new firmware:

$ make flash
avrdude -c dragon_isp -P usb -p attiny13 -U flash:w:main.hex:i

avrdude: AVR device initialized and ready to accept instructions

Reading | ################################################## | 100% 0.15s

avrdude: Device signature = 0x1e9007
avrdude: NOTE: "flash" memory has been specified, an erase cycle will be performed
         To disable this feature, specify the -D option.
avrdude: erasing chip
avrdude: reading input file "main.hex"
avrdude: writing flash (110 bytes):

Writing | ################################################## | 100% 0.46s

avrdude: 110 bytes of flash written
avrdude: verifying flash memory against main.hex:
avrdude: load data flash data from input file main.hex:
avrdude: input file main.hex contains 110 bytes
avrdude: reading on-chip flash data:

Reading | ################################################## | 100% 0.47s

avrdude: verifying ...
avrdude: 110 bytes of flash verified

avrdude: safemode: Fuses OK (H:FF, E:FF, L:6A)

avrdude done.  Thank you.

tiny13-spi-foto

For those, interested in .hex file – this is how it looks like:

$ cat main.hex
:1000000009C00EC00DC00CC00BC00AC009C008C09A
:1000100007C006C011241FBECFE9CDBF02D012C059
:10002000EFCF80E187BB18BA90E188B3892788BBFE
:100030002FE738E381E0215030408040E1F700C0F5
:080040000000F3CFF894FFCF9C
:00000001FF

and I made some commented assembler output (base is taken from avr-objdump -S main.elf).

 cat main.S

main.elf:     file format elf32-avr


Disassembly of section .text:

00000000 <__vectors>:
   0:	09 c0       	rjmp	.+18     	; 0x14 <__ctors_end>
   2:	0e c0       	rjmp	.+28     	; 0x20 <__bad_interrupt>
   4:	0d c0       	rjmp	.+26     	; 0x20 <__bad_interrupt>
   6:	0c c0       	rjmp	.+24     	; 0x20 <__bad_interrupt>
   8:	0b c0       	rjmp	.+22     	; 0x20 <__bad_interrupt>
   a:	0a c0       	rjmp	.+20     	; 0x20 <__bad_interrupt>
   c:	09 c0       	rjmp	.+18     	; 0x20 <__bad_interrupt>
   e:	08 c0       	rjmp	.+16     	; 0x20 <__bad_interrupt>
  10:	07 c0       	rjmp	.+14     	; 0x20 <__bad_interrupt>
  12:	06 c0       	rjmp	.+12     	; 0x20 <__bad_interrupt>

00000014 <__ctors_end>:             ; set stack to SRAM
  14:	11 24       	eor	r1, r1      ; set 0 to r1
  16:	1f be       	out	0x3f, r1	  ; 0 -> SPH
  18:	cf e9       	ldi	r28, 0x9F	  ; RAMEND
  1a:	cd bf       	out	0x3d, r28	  ; r28 -> SPL
  1c:	02 d0       	rcall	.+4      	; call main loop at 0x22
  1e:	12 c0       	rjmp	.+36     	; then call exit at 0x44

00000020 <__bad_interrupt>:
  20:	ef cf       	rjmp	.-34     	; 0x0 <__vectors>

00000022 <main>:
  22:	80 e1       	ldi	r24, 0x10	  ; 1<<DDB4 (0x10) -> r24
  24:	87 bb       	out	0x17, r24	  ; set r24 -> DDRB
  26:	18 ba       	out	0x18, r1	  ; set r1 (0) -> PORTB
  28:	90 e1       	ldi	r25, 0x10	  ; 1<<DDB4
  2a:	88 b3       	in	r24, 0x18	  ; read PORTB -> r24 to XOR
  2c:	89 27       	eor	r24, r25    ; XOR pin 4
  2e:	88 bb       	out	0x18, r24	  ; set r24 -> PORTB
  30:	2f e7       	ldi	r18, 0x7F	  ; 0x01387f = wait 79999 ticks (50ms at 4MHz)
  32:	38 e3       	ldi	r19, 0x38	  ;
  34:	81 e0       	ldi	r24, 0x01	  ;
  36:	21 50       	subi	r18, 0x01	; decrease r18 by 1
  38:	30 40       	sbci	r19, 0x00	; and decrease other by carry bit
  3a:	80 40       	sbci	r24, 0x00	;
  3c:	e1 f7       	brne	.-8      	; repeat until zero set - jump to 0x36
  3e:	00 c0       	rjmp	.+0      	; relative jump to 0x40
  40:	00 00       	nop             ; do nothing
  42:	f3 cf       	rjmp	.-26     	; jump back to 0x2a (loop)

00000044 <_exit>:
  44:	f8 94       	cli

00000046 <__stop_program>:
  46:	ff cf       	rjmp	.-2      	; 0x46 <__stop_program>

 

Optimizing fluentd

We’re currently using (for one part of our infrastructure) logging into elasticsearch. We have fluentd collectors and kibana interface for viewing and searching through the logs. fluentd This is how it works. Logs are sent to fluentd forwarder and then over the network to fluentd collector, which pushes all the logs to elasticsearch. As we have plenty of logs, we need to incorporate some buffering – on both sides – using buffer_file statement in the fluentd config. Here is a part of our fluentd config from forwarder

<match ***>
  type forward
  send_timeout 60s
  recover_wait 10s
  heartbeat_interval 1s
  phi_threshold 16
  hard_timeout 120s

  # buffer
  buffer_type file
  buffer_path /opt/fluentd/buffer/
  buffer_chunk_limit 8m
  buffer_queue_limit 4096
  flush_interval 10s
  retry_wait 20s

  # log to es
  <server>
    host 10.0.0.1
  </server>
  <secondary>
    type file
    path /opt/fluentd/failed/
  </secondary>
</match>

and the same for the collector

<source>
  type forward
  bind 10.0.0.1
</source>

<match log.**>
  type elasticsearch
  logstash_format true
  # elastic host
  host 10.0.0.3
  port 9200
  logstash_prefix log
  include_tag_key

  # buffering
  buffer_type file
  buffer_path /opt/fluentd/buffer/
  flush_interval 5m
  buffer_chunk_limit 16m
  buffer_queue_limit 4096
  retry_wait 15s
</match>

So. For the forwarder, we’re using buffer with max 4096 8MB chunks = 32GB of buffer space. Forwarder is flushing every 10secs. For collector, we use bigger chunks, as elasticsearch is capable to handle it – but not using default 256MB chunks due to memory limitations. Flushing period is longer – and should be – recommended value is 5minutes. We can keep up to 64Gigs of buffer data.

What happens if one of the fluentd dies. Some data will be probably lost, when unsaved to buffer. But. When there’s connection lost or collector fluentd isn’t running, all logs, collected by forwarder, are stored into the buffer – and sent later. Which is great. The same when ES is down for some reason, collector node is still receiving data and is able to continue sending into ES after full recovery.

PS: don’t forget to make some tweaks to the system itself, like raise the limit for max files opened and some tcp tunning.