eye killed?

Ever seen

If so, solution is quite simple. Run  rm ~/.eye -rf and remove eye folder. Then re-run eye load  and it should all work :)

AWS S3 IAM user policy setup for bucket

First of all, setup your AWS account – and login into AWS Management Console. Then create any bucket you want.

Next step is to add new IAM user – go to https://console.aws.amazon.com/iam/home?#users, select Users and click Create new user button.

iam_create

Click Create and copy/paste or download your access keys.

iam_access

Then close window and click to summary tab – you’ll need to copy ARN notation of this user

iam_summary

 

s3cmd

To be able to sync some folders and/or use s3cmd commandline tool, you need to setup some more policies. Click Permission tab and then Attach User Policy. Then choose Policy Generator and follow these steps:

manage_user_policySet permissions as shown in following image – we need ListAllMyBuckets for :::* and ListBucket and PutObject for the one, specified bucket.

user_policy_editor

 

Click Continue and save new policy

set_permission

You can always edit the policy using Manage Policy. You should create this kind of policy file:

That’s all. Lets move to bucket policy.

Bucket policy

Now you can open Policy Generator – http://awspolicygen.s3.amazonaws.com/policygen.html and start adding new items.

First of all, select your policy type – S3

policy_select_type

and then fill new policy – we need two policies – one for all users to read objects (GetObject) and second for your new user, to be able to upload, delete and get object as well.

public_policy_form

and click Add Statement. The list below the form should look like

public_policy_list

So, add your user the same way. As principal add your ARN notation (which you copied before) and add Actions as shown below.

policy_user

All set. You need to click Generate Policy button and copy freshly generated JSON policy to your clipboard

Last step is to add this policy to the bucket. Go back to your S3 console (https://console.aws.amazon.com/s3/home), click loupe icon next to bucket name and open Permission section. Click on Edit Bucket Policy and paste the generated policy into that window. Save and you’re done.

bucket_permission

 

You’ll need API keys for use with Paperclip or any other S3 storage engine, so keep them safe.

PS: when you plan to use s3 as storage for your static files, avoid using underscore in your bucket name – as it does not comply with valid FQDN. ‘sample_bucket’ was used only as an example, in the real word you should use ‘sample-bucket’.

Optimizing fluentd

We’re currently using (for one part of our infrastructure) logging into elasticsearch. We have fluentd collectors and kibana interface for viewing and searching through the logs. fluentd This is how it works. Logs are sent to fluentd forwarder and then over the network to fluentd collector, which pushes all the logs to elasticsearch. As we have plenty of logs, we need to incorporate some buffering – on both sides – using buffer_file statement in the fluentd config. Here is a part of our fluentd config from forwarder

and the same for the collector

So. For the forwarder, we’re using buffer with max 4096 8MB chunks = 32GB of buffer space. Forwarder is flushing every 10secs. For collector, we use bigger chunks, as elasticsearch is capable to handle it – but not using default 256MB chunks due to memory limitations. Flushing period is longer – and should be – recommended value is 5minutes. We can keep up to 64Gigs of buffer data.

What happens if one of the fluentd dies. Some data will be probably lost, when unsaved to buffer. But. When there’s connection lost or collector fluentd isn’t running, all logs, collected by forwarder, are stored into the buffer – and sent later. Which is great. The same when ES is down for some reason, collector node is still receiving data and is able to continue sending into ES after full recovery.

PS: don’t forget to make some tweaks to the system itself, like raise the limit for max files opened and some tcp tunning.

capistrano3: run gem binary

I need to setup deploy task to run eye. Tried to deal with gem-wrappers, but had no success. As capistrano3 is using non-interactive ssh, without loading user’s environment (.profile, .bashrc etc), then command, which is not in PATH, it’s not workin.

So, after searching and reading capistrano (capistrano/rvm) source, then sshkit source, I got into this simple solution.

It’s not dependent on any other settings nor knowing what, where rvm is installed.

before change in deploy.rb (not working)

after change in deploy.rb

original code from capistrano is

redis sentinel with ruby (on rails)

In the last article I introduced how to install and use redis sentinel. As I’m using ruby, I need to use this new redis configuration with ruby (on rails).

For ruby on rails use redis-sentinel gem.

Then your redis initializer will look like

You can use your redis then as usual.

When using sidekiq, configuration is pretty simple too

You can test your configuration. Run rails console and test with

if you see “127.0.0.1:6379”, something is probably wrong. Then try to set/get some key and check Redis.current once again.

rails + passenger + nginx maintenance mode

I need to add maintenance page to some rails app, running with passenger and nginx. Here’s some config and steps.

You just need to add static html file to app_root/public/maintenance.html – and I assume css files on /assets url.

so, here’s nginx config:

setting maintance mode is really simple – set app_root/tmp/maintenance.txt file – when escaping, just remove that file.

Change image in UIImageView – RubyMotion

I needed to change image in my UIImageView, but simply setting new image using myImageView.setImage didn’t work. There’s simple workaround

and now set the image

you can remove resizableImageWithCapInsets – I’m using square images to make UIImageView of any size.

Print simple x*x table in ruby

I’ve needed to print simple table as

as you can see, x = y, cell in table = x*y (x*x to be exact)

Calculate prime numbers

Tried simple Sieve of Eratosthenes in Ruby …

we can add some tests