Rails Starling Setup, Options, and Usage Documentation

After spending too many frustrating hours and days fighting BackgrounDRb to connect to multiple servers, I decided to try option #2, the Starling/Workling combo, as of yesterday evening. So far, I’ve been really pleased with most of the setup. I had it working preliminarily in about an hour (compare to about 2 full days for one of our developers to get BackgroundRb setup) locally. However, if there is one thing that’d made Starling difficult so far, though, it’s the lack of useful results I get when I Google anything I can think to Google about Starling documentation. Thus, I am going to try to keep updating this blog as I get Starling tweaked and working, with all of the information I can gather to try to make future Starlingers have a bit easier proposition.

Installation

On your Starling server(s), run “sudo gem install starling” to install it.

On your clients, you talk with Starling via memcached, so you’ll need the memcached client installed. “sudo gem install memcache-client” should do the trick.

Basic Usage Example

I found this morsel in the readme.txt for the Starling gem:

# Start the Starling server as a daemonized process:
# Note by Bill -- I believe that starling will only run as root currently
sudo starling -h 192.168.1.1 -d

# Put messages onto a queue:
require 'memcache'
starling = MemCache.new('192.168.1.1:22122')
starling.set('my_queue', 12345)

# Get messages from the queue:
require 'memcache'
starling = MemCache.new('192.168.1.1:22122')
loop { puts starling.get('my_queue') }

The first line starts a starling that listens on 192.168.1.1, i.e., on your local network. The next lines actually connect to that Starling and test it. They can be run from within irb to verify that your Starling setup is working as expected.

Other options

In general, you can see memcached methods to figure out what Starling is capable of (well documented here). The key difference between Starling and Memcached, from my testing, is that Starling’s get() returns the hash value and deletes it from the cache. Memcached’s get() just returns the value, which is still in the cache until you call delete().

Otherwise, Starling is pretty similar to Memcached. For example (from Starling tests):

starling = MemCache.new('192.168.1.1:22122')
starling.set('test_set_with_expiry', 5 + 2, now)
starling.set('test_set_with_expiry', 5)
sleep(now + 1 - Time.now.to_f)
starling.get('test_set_with_expiry') # returns 5

Like memcached, if you want to get stats on your Starling, such as number of items in the queue, bytes used, starling version, log size, cache hits/misses, total connections, stuff that’s currently cached, and more, you can run stats() on your starling connection object, like so:

starling = MemCache.new('192.168.1.1:22122')
starling.stats # returns hash of statistics on the starling connection

Bugs and Caveats

When you’re setting your Starling up on a remote server, you’ll want to remember to change the address you bind to when you change Starling, e.g., “sudo starling -h my_ip_address -d”. If you’re not root, you’ll probably need the sudo, since Starling binds to network ports.

A couple more apparent bugs I’ve experienced with Starling (running gem version 0.9.3 on Gutsy):

  • Even if I specify a log file on the command line, no log file is generated
  • Though when you get() a message it is removed from the queue, it still exists in the /var/spool/starling directory, from what I can tell, indefinitely – Blaine explains this in my Starling update

If anyone who has been a longer-time Starling user has seen and gotten past these issues, I’d sure like to hear them.

Update

I was able to talk with Blaine at Railsconf about some of my specific questions about Starling, and about the future of Starling in general. Read about it here.

7 Replies to “Rails Starling Setup, Options, and Usage Documentation”

  1. Thanks for posting this. I also had some problems finding documentation when I was setting up my initial starling environment.

    I have a note regarding logging that might help someone else out.

    as of starling .0.9.3, there is a bug in the runner script found under GEM_PATH/starling-0.9.3/lib/starling/runner.rb

    line 17 is: @process = ProcessHelper.new(options[:log_file], options[:pid_file], options[:user], options[:group])

    This will eventually set up the log file at the path found in options[ :log_file ].

    Unfortunately, the option parsing code sets the :log key in the options hash when -l or –log-file is specified on the command line. This is at line 86.

    options[:log] = log_path

    So, the runner script will never find a path. This is further compounded by the fact that, in the absence of said path, all output is redirected to /dev/null.

    If someone is anything like me, they may have cheerfully followed the setup instructions and created directories for their queues and such in the default locations. Then, before running starling for the first time, they may have decided that it would be nice to put them in their own custom locations. But they would have forgotten to make make the new directories in these locations. They would then spend hours wondering why starling didn’t seem to work, because they wouldn’t be getting the extremely helpful output that inidcated that directories were missing. 🙂

    They would then probably learn 2 lessons.

    1) Always follow instructions. Each. And. Every. Time.

    2) Don’t silently pipe important output to /dev/null. 🙂
    I just hacked runner locally for my setup.

    Aside from this minor, partially self-inflicted issue, I’ve been extremely pleased with starling thus far.

    I can also verify your last point about messages remaining in the queue files.

    — gmc

  2. Wow, what fantastic information. Thanks a lot Gabriel!

    I am on heitus from Starling from the moment, since I needed a solution that I could query for status, and since Starling deletes any message that you get() from the queue, it didn’t seem like the best answer for this (I could always put message back into the queue when they are found by the status-getting mechanism, but that seemed too hacky).

    Thus, I’m back to BackgroundRb for now, until I want to run my BdRb processes on multiple servers, at which time I’ll presumably set up a Starling/BDRB combo. And at which time, hopefully the Twitter folks will have a chance to update the gem with fixes to some of these sort of obvious problems.

    I still think Starling could be a good solution for someone who wants an easily scalable way to get a persistent queue, and doesn’t need to check the status of their messages while they’re being processed.

  3. How do I kill starling -d? I tried killall starling but it said it could not find the process, but i did ps -ef | grep starling and it found it:

    0 186 1 0 0:08.29 ?? 0:11.37 /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/bin/ruby /usr/bin/starling -d

    when i do killall memcached it kills memcached -d.

  4. Hmm.. I don’t personally remember have that particular problem with starling. I usually used “skill starling” (or skill [process number]) though I doubt there’s a difference between that and killall. Hopefully someone else who’s had the problem can chime in with some insight.

  5. Dear Matt, kill the pid, if not killall the binary, which is not starling, but ruby. Ruby is the real executable, where as starling is running inside the instance of ruby.

  6. I was wanting to know if setting the sleep time is the time it waits between sending each message in the queue? I have a server that only allows 500 messages an hour and I want to limit certain messages that would go out the entire database.

    Thank you,

    Sean McGilvray

  7. Root privileges should only be required for binding to ports below 1024. As starling generally is bound to a high port, one above 1024, that does not require root to run it.

    The reason that starling seems to need root to run is that it tries to write files in /var/spool/starling and if that directory is not accessible by the user running starling it silently aborts the startup after writing a pid file. This is only noticeable if you run it without the -d option. So chown the /var/spool/starling to the user you want to run starling as and it should work without an issue.

Leave a Reply to Matt Cancel reply

Your email address will not be published. Required fields are marked *