Implementing Web Push Notifications

Long gone are the days of push notifications being an exclusive feature of native mobile applications. With the advent of the Push API, web browsers now have the ability to receive notifications as well. Users can even receive notifications when they don’t have your website open. We have recently implemented support on meetme.com.

What is the current browser support?

Web push works on both desktop and mobile; however, not all browsers support the standard yet.

  • Chome 50+ (both desktop and mobile Android)
  • Chrome 44 introduced partial support but notification payloads were not supported
  • Firefox 44+ desktop, Firefox 48+ mobile Android
  • Safari does NOT support web push (although the desktop version supports Apple’s own proprietary protocol for push)
  • Coming soon to Microsoft Edge

High level overview

From the perspective of a user, web push notifications work largely the same as on native mobile. However, before a user can start receiving push notifications they must accept permissions for your website. When a user arrives at a website that supports web push (and they are using a browser that supports it), the website can then prompt the user to enable notifications:

Chrome Desktop
Chrome desktop notifications prompt

From this point forward, whenever the website has a notification it will make a REST call to the push provider (more on this below), which will in turn generate the notification on the user’s device. Note that the user does not need to have the website open in order to receive notifications.

Notification
Chrome desktop notification

Behind the scenes the website needs to implement a service worker which will handle receiving the push event and generating a notification UI element.

Components of Web Push

Service Workers

A service worker runs in the background separately from your web application and is required in order to receive push notifications. It has no direct access to the DOM and any communication with your web application must occur using postMessage() to send and receive messages. Some other aspects of service workers include:

  • Can be stopped/restarted by the browser as necessary (you cannot depend on global state)
  • Has access to the IndexedDB API
  • Does NOT have access to LocalStorage or cookies
  • Receives the actual push notification message
  • Generates the visible notification message and handles the click

Use of Promises

The API for service workers relies heavily on promises, so you will need to have some familiarity with them. In a nutshell, promises are designed to make working with asynchronous code easier by wrapping success and error callbacks in chainable functions to allow for greater readability. Although this article is not going to give a tutorial on promises, some good documentation can be found here. One thing that should be noted, however, is that if you are accustomed to using promises in jQuery or Q, the interface for JavaScript ES6 Promises is slightly different. For example, here is how you might use a promise in jQuery:

The preceding code will wait for 5 seconds and then print “Wait Complete!” to the JavaScript console. Let’s look at the equivalent using ES6 Promises:

As you can see, the concept is the same but the syntax is slightly different. It should also be noted that, unlike jQuery, an ES6 Promise does not have a done() method.

How is a service worker used?

A service worker needs to be registered; however, we must first check for browser support.

Once our service worker is registered we will need to subscribe via the PushManager in order to start receiving push notifications.

For users who have not previously enabled notifications a prompt will appear.

Chrome android prompt
Chrome Android notifications prompt

Note that if you want to check for the existence of a push subscription prior to invoking the prompt, you can do so by calling getSubscription() on the PushManager instance instead. If the Promise it returns resolves to a null value then there is no existing subscription. Calling getSubscription() first can be useful if you want to display your own custom message to better inform the user of the enable notifications prompt they are about to see.

It’s possible to determine if the user has pressed cancel on the “enable notifications” prompt if you call subscribe(); the error thrown will have a “name” property of “PermissionDeniedError”.

You may have noticed that when we called subscribe() a “userVisibleOnly” property was specified as true. This is currently required in Chrome. The option indicates that for every push notification received we will show a notification. If you were to not generate a visible notification when a push message was received then the browser will generate a default notification with the text “This site has been updated in the background.” This can also occur if you attempt to perform an asynchronous operation prior to showing the notification without returning a promise.

In our handlePushSubscription() function, we will receive a single argument of type PushSubscription. This contains the data that is required in order for our backend to generate push notifications. The data that PushSubscription contains is a bit different from what you may be used to if you’ve worked with native mobile push notifications (which generate a token hash string). Here is an example of how it is formatted if we stringify the PushSubscription object:

The “keys” portion is what is used to encrypt the payload of the push message. There are libraries that can handle this portion for you (we’ll show an example later on).

The “endpoint” property is unique and is used to identify your browser. Endpoints take the following form (where “unique-key” is a key that identifies the user’s specific browser):

Chrome
https://android.googleapis.com/gcm/send/unique-key

FireFox
https://updates.push.services.mozilla.com/wpush/v1/unique-key

Now let’s define our “handlePushSubscription” function. Its job is going to be informing our backend of the push subscription (which can then be used to generate notifications to the browser itself).

The URL endpoint used above will need to store our web push subscription data (and associate it with the current user).

Generating Push Notifications on our backend

When we have a notification that we would like to send to our user, we will need to encrypt the notification’s payload and make a POST request to the endpoint that we received. To accomplish this, we’re going to use Node.js and the excellent web-push package.

Chrome requires some additional setup steps such as defining an application manifest with a “gcm_sender_id” (for more details see here).

Handling Push Notifications in our Service Worker

We’re now going to setup the “serviceWorker.js” that we referenced earlier. This script is going to run in the background listening for notifications.

There are other useful events that we can listen for in a service worker (see a more thorough guide here). The “push” event is fired when a push notification is received by the browser. You do not need to have a tab open (of the web application) to receive a notification. Let’s define our event handlers:

Note that the notification click handler above will open a new browser tab regardless of whether or not one is already open (there are ways around this by finding an existing client tab using clients.matchAll(), sending it a message via postMessage(), and then handling the message in the web application).

Conclusion

Browser adoption of the push standard has taken an important step towards moving the web platform forward. Although the discussion of service workers in this article was limited to push notifications, service workers can be used for other purposes such as adding offline functionality. A full example of the code used here has been posted on Github.

PaletteBar – A color picker for Android

Try it out

As a developer, I’m often skimming tech blogs in search of a solution, so if you’re here to find a color picker for your Android app, here’s the Github link: https://github.com/MeetMe/PaletteBar

paletteBarScreenshot

PaletteBar is a minimal color picker, implemented as a custom Android View, done entirely in code in one class file. The border shows the selected color – in this case, light green. Here’s a grainy gif of it in action:

paletteBarDemo

From a development perspective, it’s super easy to add to a project. Just copy the single PaletteBar.java file, no extra resources or xml needed. It’ll automatically scale that color spectrum to any height and width, and the border size can be changed or removed entirely.

From the UI side, this layout allows any color to be chosen by moving horizontally, and any hue by moving vertically. Grays and browns have been tacked on to the left. The notable downside is that it’s nigh impossible to choose the exact same color repeatedly.

Here’s PaletteBar in our MeetMe app on Android, used to edit Chat photos:

selfiePaletteBar

Color pickers are tricky

The trouble with mobile devices is that screens are small and fingers get in the way, and the trouble with mobile developers is that we haven’t had many years to establish conventions for UI elements.

In helping build MeetMe’s drawing feature, and in my personal quest to find useable apps for my toddlers, I’ve tried a ton of apps with palettes, and most of them fail terribly. If a drawing app is just for entertainment, then you don’t need to specify RGB values or cram in a “recent favorite colors” section for your palette. We recently downloaded a [beloved copyrighted character] coloring book app that was particularly awful. It took me and my young kids a couple days to realize that the palette only appeared to be limited to six colors because it could scroll horizontally.

Ultimately, there are just two UI problems for a color picker to solve:

  1. Show the user all the available colors
  2. Allow the user to choose a color

Some apps use a ROYGBIV spectrum, which is great for making rainbows, but you can’t draw people and robots without browns and grays. Check out how many palettes make that faux pas in this Google images search for “color picker”: https://www.google.com/search?q=color+picker&tbm=isch  For apps that only need a dozen or so colors, like a simple text editor with font colors, the easy solution is to have a button for each color. It’s when the palette grows that the challenge arises for both displaying and accessing colors. Wikipedia has a small article under “Color tool” that shows some complex, but common, solutions for art applications: https://en.wikipedia.org/wiki/Color_tool  Even in a sophisticated app, these desktop solutions would be unwieldy.

ColorPalette is an attempt to provide pretty much all the colors in limited real estate, with quick interaction, at the cost of being able to specify a color beyond eyeballing. For an entertaining scenario like MeetMe’s Chat, it’s a great fit . I also integrated ColorPalette in an extremely simple drawing app for my kids, and can confirm that even a 2-year-old can use it. And it’s rife with opportunities for feature creep or even legitimate enhancements, so have at it.

simpleDrawUse2

Troubleshooting Rundeck Remote Resource Issues

rundeck-logoAs part of an ongoing effort to migrate a significant part of our stack into the cloud, we recently moved our job scheduling system, Rundeck, from an on-premise bare metal instance to AWS. With the exception of a few minor annoyances–from an operational standpoint–pretty much everything worked. It didn’t take long, however, to notice jobs weren’t running on-time. Minutely jobs were running between 30-50 seconds off, skipping minutes, etc. As we had a near-identical configuration, we weren’t really sure what the issue was. Given the lack of documentation for Rundeck architecture and having seen similar unanswered questions across various message boards, after three of us had spent a couple days on the issue, we wanted to post the steps we took to troubleshoot and resolve the issue.

Job Concurrency Limitations
As the most common cause of items not running on-time in a scheduling system is related to the permitted level of concurrency, and we saw no CPU-related run queue issues, we first expected the delay to be limited by the number of available worker threads. So, per the Rundeck Administration tuning guide, we increased the Quartz scheduler’s thread pool size (org.quartz.threadPool.threadCount), related JVM heap settings, and restarted the service. We noticed zero effect on jobs and verified the thread pool was operating as expected.

AWS Instance Sizing/Performance
At this point, we knew the Quartz scheduler was not the issue. As everything else in the configuration was identical, while we hadn’t seen any CPU-related issues, we figured it could just be related virtualization slowness. So, just to rule it out, we rolled the next largest box. As expected, given we had seen nothing to indicate CPU issues, we saw no difference in performance on a larger instance; jobs were still running 30-50 seconds behind.

Java Version Differences
After comparing packages between our on-premise box and AWS box, we found our on-prem box to be running Java 1.8.0, while our AWS instance was running 1.7.0. We upgraded the AWS instance to 1.8.0 and noticed a five-or-so-second improvement. The UI was much faster. But no huge improvement in runtime.

Waiting for Execution
During one of the times we launched a job manually, we noticed it became queued immediately, but sat there in the UI for over 30s saying, “Waiting for execution to start.” We thought this was odd, as the box wasn’t exhibiting any CPU-related issues and there were hundreds of free threads in the scheduler to execute the job. We searched and came across this thread, which seems very similar and was unanswered.

Local vs SSH Job
In an effort to make sure SSH wasn’t the issue, we created a local job that simply echoed the current timestamp into a file in /tmp. This job too ran between 30-50 seconds behind the time it was supposed to start. Obviously, this isn’t an SSH issue.

MySQL Optimization
As we’re using MySQL as the backend of Rundeck, we figured there may be something up with database performance. After taking a look, we noticed several queries taking several seconds to complete. Accordingly, we realized the tables weren’t indexed properly for those queries. Rather than creating them ourselves, we searched and found a ticketed performance issue specifically related to missing MySQL indices. We created these indices and, while we no longer saw any slow queries, we saw no improvement in job execution.

Reducing the Number of Concurrent Jobs
To see whether this issue occurred with the same frequency with fewer jobs, we disabled all but one minutely-executed job. While it was still off by ~6s, it was far better than 30-50s. As we enabled other jobs one-by-one, we saw increasing start delays. As it was still 6s off, we figured something else must still be going on.

Watching the Filesystem
During the time we had been watching the single job logs, we noticed a new job log was being created at EXACTLY the right time the job was supposed to start. That is, at the first second of each minute. Unfortunately, when we looked at all other logs, none indicated this job had even started. So, how did this file get created at the right time, but no scheduler log showed this job had actually yet started? Let’s eliminate all the variables.

Eliminating Outside Variables
Our Rundeck configuration includes multiple servers, which have nodes resourced from a local XML file (resources.source.N.config.file) as well as multiple servers via a URL in the project resource configuration (resources.source.N.config.url). We disabled all URL-based ones and noticed our job ran perfectly on-time over and over again. So, something was up with the URL-based configs. We noticed, for some reason, caching was disabled (resources.source.N.config.cache=false), so we set that to true, reenabled our URL-based resources, and restarted. Same old performance issues. Looking at the code, that’s only because this adds additional cache headers to the HTTP request. As everyone who knows HTTP knows, dynamic endpoints still have to do all the work, even if they don’t return any results. So, we tried to fetch that resource directly via cURL. It took ~2s to fetch the resource file from each server. Yikers! We then researched the resources.source.N.config.timeout option, only to find it wouldn’t really help in this case. Lastly, we decided to try saving those resource files locally and using them directly. After that, all jobs ran perfectly on time.

So, when a job gets kicked off by Rundeck, it appears it’s not officially “started” until after the resource configuration is fetched. As multiple jobs run per minute, each one fetching N number of URL-based resources, this caused a stampeding herd to the other Rundeck servers and, accordingly, a significant lag in execution. This also explains why–in the more-jobs-per-minute case–we saw greater lag with each added job.

Update: Response from Rundeck

The URL source is a blocking call so unfortunately it can lead to this kind of problem. We have discussed the idea to enable an asynchronous URL request to help avoid this kind of lag problem.

With the exception of making calls to multiple URL sources concurrently, thereby reducing startup-time from the sum of all response times to the maximum response time, I’m not sure how this could be improved in any asynchronous manner. As all nodes are required at the start of a job–to me–node retrieval must be remain synchronous, but it could be improved with smarter caching on the HTTP server. ¯\_(ツ)_/¯

A look at Apple’s new programming language: Swift

There has been a lot of buzz over Apple’s new programming language, Swift, and many questions as well. Why did Apple need a new language? How is it different from Objective-C? As a developer, should I learn Swift? We’ll dive into these topics in this post.

A Brief History

Swift is the heir to Objective-C, so it’s very difficult to talk about Swift without first talking in detail about Objective-C. Objective-C was created in the early 1980s and descended from both C and the Smalltalk Language.

While most every programmer is familiar with C, Smaltalk is more obscure. It’s an object-oriented language and uses a message-passing style to call methods. (This will come into play later when we discuss Swift syntax.)

During his exile from Apple, Steve Jobs formed a company called NeXT. This company licensed Objective-C in 1988, and they used the language to build an operating system called NextStep. Flash forward to 1996, Apple’s in bit of a rut and they’re looking to update their operating system…Apple acquires NeXT and uses it as the basis of Mac OS X.

Things were fine for a few years and a small amount of developers quietly made Mac desktop applications. In 2007, everything changed when the iPhone was released. Developers were soon able to release third party apps in the App Store which led to an explosion of interest in Objective-C.

What’s wrong with Objective-C?

So what’s the problem? Why can’t we just use Objective-C until the end of time?
In short, Objective-C was never intended to be as popular as it became. It was used for the OS and Mac apps, but there wasn’t a mobile app industry at the time and no one could anticipate what a game changer the iPhone would be.

Objective-C has a somewhat tough barrier to entry because it doesn’t look like a lot of the languages that descended from C like C++, Java or C#. As far as aesthetics go, Objective-C is fairly hideous.

Here’s a sample of Objective-C code:

For comparison, this is how you could accomplish the same thing in Java:

Objective-C can be intimidating to programmers who aren’t used to this syntax. Let’s take a look at some of the issues here.

  • We have square brackets all over there place
  • An asterisk before the variable name
  • @ symbols
  • Lots of extra words cluttering up like alloc and initWithFormat
  • What’s with this “NS” and why is that cluttering up the String?

You’ll see this “NS” in many classes in Objective-C: NSArray, NSString, NSDictionary, etc. It’s a nightmare for autocomplete.

Because Objective-C doesn’t have a concept of namespaces, class names have to be unique, so you need to add a prefix like “NS”. If we were to create a custom string in the MeetMe app, it might have a name like MMString.

The prefix “NS” is a relic from when Objective-C and its Foundation framework were built using NextStep. Again, this was created way before anyone could anticipate how popular it would be to code in the language. If they had a crystal ball and saw the future, there’s a good chance the team at NeXT would have made the entire language easier to use.

Over the years, Objective-C has gotten friendlier with more features you’d expect out of a programming language (like blocks and literals), but Apple was looking for a lighter language with less barrier to entry and to clean up some of the relics of the past — Swift is their answer.

Concise

One of the main benefits of Swift is that it is a much more concise language than Objective-C. Gone are:

  • Semicolons ;
  • [Square brackets ]
  • #import statements for classes
  • Alloc/init, lengthy method names

Back to the previous code samples with the strings — here is the same line in Swift, and you can see it has done away with all the weirdness we pointed out in the previous example, looking a lot more like a modern language:

You can also overload the arithmetic operators with your own custom functionality. In Objective-C, there is no overloading; you have to create a new method name if you want to have different types of parameters for the same method. As you may know, autocomplete gets pretty cluttered with all these extra methods.

In Objective-C you need to access properties with the “self” keyword and also call methods on self. In Swift, the self is implied in properties; you can write it out if you want to be explicit, but you can omit self for conciseness. If you call a function without specifying a variable to call it on, the function is implied to call on self so you don’t need to declare that either.

Objective-C:

Swift:

This is an example they used in the Apple Keynote when they first introduced Swift.

Here, you see some code you’d commonly see in an Objective-C app where you spend a lot of time checking if the delegate is nil and checking if the delegate has a particular method before you call it.

In Swift, they’ve cut down on a lot of typing with Optionals:

The use of Optionals is indicated by the question marks. (Optionals will come up later when I discuss safety.)

Another change that results in more concise code is the use of strict types in collections.

Here is a sample method from Objective-C:

You pass in an array of donuts, but the compiler doesn’t know that the array only contains NSDonuts. We know it because of clues from the method name (and maybe the documentation) but, for all the compiler knows, there could be strings in that donut array.

So, we create a donut variable and cast it as NSDonut to assure the compiler that it’s ok to call eatDonutMethod on it. Is there a better way? Here’s how this same method could be implemented in Swift:

The syntax is a little flipped, but we have the same name of the array — donuts. Instead of a general array type, this has an array strictly of donuts. This allows us to call donut methods on any object in the array without needing to cast it because it has to be a donut; the code won’t compile if you try to send an array of strings to this method.

It’s a great win for more concise code, but this also demonstrates another feature of Swift — its emphasis on safety!

Safety

All variables are statically typed in Swift, meaning you have to declare if a variable is an Int or a String; for example, you can’t just say it’s a variable and call any method you want on it later.

While Objective-C is also statically typed, it uses the generic id property in collections as we previously discussed. This sometimes leads to errors happening at runtime when a collection ends up having a type that you weren’t expecting. In Swift, because collections are of a particular type, there will be an error before you compile rather than at runtime if you do pass in the wrong type.

Type can also be inferred; as you see in this example, you don’t need to explicitly declare the variable type.

Here, it is inferred that donutArray is an array of donuts based on the assignment. Under the hood, this is still statically typed; donutArray is an array of Donuts, even though you didn’t have to explicitly type it out like the line above. You get the safety of static typing, but it’s also concise.

In Swift, variables are only able to be changed if you declare them a var. The preferred practice is to declare your variables as constants using the ‘let’ keyword. Not only does it optimize the code and make it faster, it adds safety by throwing you a warning if you try to change the contents.

Swift really tries to get you to think about how you’re going to use each variable, which we also see with Optionals. Nil or null pointers cause a lot of problems in programming, and different languages have different ways of dealing with it. In a lot of languages, your program will crash if you try to call a method on a null object.

Because Objective-C was inspired by the SmallTalk language, it uses message passing to invoke methods (told you we would come back to that!). A feature of message passing is that you can get away with calling a method on a variable that doesn’t have the method and the compiler won’t complain. In Objective-C, messaging nil returns nil, so no harm, no foul, no crash.

And yet, this isn’t ideal. What if you need the return value from this method and getting back nil will crash your program? You could wrap this in a nil check, or use the respondsToSelector method, and that is what we have to do all the time in Objective-C. Swift is looking for a better way to handle the nil problem.

Enter Optionals. Here’s an example from earlier — let’s say later on you want to clear out the donuts and set the variable to nil:

You cannot do this because then donutArray would no longer be an array of donuts. A var expects a value at all points of its life cycle, from initialization until it is deallocated. How you can accomplish this is by making donutArray an Optional:

An Optional behaves like how variables in Objective-C behave in that they can be nil at any time. They also do not need to be initialized to a value.

Optionals should be used only when necessary because they work around the safety Swift has created to try to eliminate runtime errors that crop up around surprise nil variables. The preferred order to create variables is:

  1. let constant
  2. var if it needs to change
  3. optional var if it needs to be nil at some point

Optionals are generally meant to be used with a question mark, meaning that there may be an object there or it may be nil. Sometimes you need to guarantee that there’s a value there. The easiest was to use the underlying value of an option (and least safe) is to to force unwrap it by using an exclamation point.

This is a brute force way of saying, “Just trust me, there’s a value there.” This should be avoided unless absolutely necessary because it goes around the safety of optionals and the program will crash if it turns out you were wrong and there’s no object there.

The preferred way to handle this is using a new statement, “if let” (also known as Optional Binding). This creates a new local constant equal to your optional only if there’s a value there. You then safely use the new constant inside the block.

Optional binding is similar to a nil check, so that seems to go against Swift’s push for concise code. I think that, as Swift develops, the syntax for this situation will become more concise. For now, this is the way to safely handle optionals when you need to guarantee they’re non-nil.

Other Safety Features

Swift has eliminated several features that allowed programmers to be a little lazy at the expense of safety. Here are two conditional statements in Objective-C that both do the same thing:

If you only have one line inside the body of an if statement, you have the option of omitting the curly braces in Objective-C. The problem is that it’s really easy to think that you’re just going to need one line, then come back to your code sometime later and add more lines not realizing that you also needed to add braces. Those extra lines will not be tested by the conditional statement; they will always execute!

This is an insidious error because the compiler will not complain; it just always executes the extra code. You may not even know there’s a problem until months or years later when unusual things happen. The most infamous recent example of this is Apple’s “goto fail” SSL bug, which happened while using this same shortcut:

Apple decided that this convenience was just not worth the risk, so the brace-less conditional checking in Swift is disallowed. Another area Apple made safer while actually letting us be lazier is in switch statements. Here is an example switch statement in Swift:

Notice there are no “break” statements anywhere. They are no longer an option in Swift — every case breaks automatically. While this takes away some of the flexibility of being able to fall through to the next case, it eliminates the common problem of forgetting to type “break” after every case and accidentally falling through when you didn’t intend to.

Instead of falling through to additional cases, Swift allows adding several options to the same case (see the “left”, “right” case). And, thankfully, you can now use non-number types inside switch statements such as strings (as you see in the above example), classes, and optionals.

Finally, switch statements must now be exhaustive, which usually means that a default statement is required. In the case of an enum you can usually evaluate all cases; however, there is no way you can evaluate every possible string combination, so use default if you don’t want to get a compiler error!

There are lots of other examples of safety in Swift, too many to go through in this post. The main takeaway is that Swift was designed to be easier to learn, faster to run, and safer to code in than Objective-C. It is evolving and has had some significant changes in its short lifetime. It remains to be seen how quickly Swift will supplant Objective-C, but there seems to be justified enthusiasm for non-Objective-C programmers looking to get into iOS app development.

Hackd project – Remote Car Starter

It’s HACKD season again at MeetMe! For those of you that are not familiar with the concept, hackathons are events that some companies, schools, and other organizations put together where people work on the projects they want, like new technologies, inventions, ideas, etc., that they usually doesn’t have time to spend on. I’ve had a project on the back burner for a while that I wanted to work on but, because lack of experience and knowledge (a.k.a self confidence), I never went forward with it.

MeetMe offers HACKD every couple of months and during one of the recent events, I decided to finally work on this project, which I call “Really Remote Starter”.

THE PROBLEM

About 3 years ago, I lived in Northwest Mexico, near Arizona, US. Since then I moved to the US East Coast. So I basically moved from here:

16068683

…to here:

1549573_10202914907332087_349482691_n

I found out that in this weather I need to sit in my car for a while until it gets warm. Sitting in the car does not provide an ideal comfort level. Even if you turn the heat on, it’s useless for those first few minutes.

THE IDEA

By the end of the first winter season, I desperately wanted a remote starter for my car. I thought, “it would be nice to start my car just from the warmth of my home or office and then just go to my car into a warm interior and go”. Then, when I got a remote starter for my car, I quickly realized that it wasn’t always going to be that useful.

At work, the parking lot is located at the back of the business center, and it fills pretty quickly. It can take you a good 5 to 10 minutes to walk to your car. Also, in other cases like parking at malls, movie theaters, etc. makes the Remote Starter less useful. Why? Remote starters and car alarms have RF remotes, which are very limited in terms of reach.

In my other car, in Mexico, I have a high end car alarm with a two-way RF remote that promises a one mile reach. Well it has a one mile reach in a place like this:

200013474-001

If you start adding cars, walls, and other barriers, the reach decreases dramatically. So, I came up with the idea to make my remote starter to start from wherever, without a distance restriction.

THE SOLUTION

I wanted to start my car from wherever with my cellphone, sending the car some kind of signal. I wasn’t sure exactly how to achieve this, so I did some research and looked at a bunch of Youtube videos and written tutorials. It seemed that I wasn’t the first one to try this. It seemed that everyone wanted to use bluetooth. It makes sense–your cellphone has bluetooth and you can use an arduino or a raspberry pi with a bluetooth module and voilá. But that wasn’t enough for me because that doesn’t solve my problem. Although bluetooth could have longer reach than RF, it still has that distance restriction. I wanted something that doesn’t care how far are you from your car. Then the WIFI idea came to me. “What if I put a WIFI module in the arduino or the raspberry pi?” Nope, that is not going to work either.

It should have WIFI coverage everywhere and there’s no WIFI coverage everywhere; you also have to deal with password-protected networks and stuff like that. Same happens with 3G or 4G–there’s no 100% 3G/4G coverage everywhere, and it will turn this to a very expensive solution since 3G/4G mobile plans are expensive. But, what if I send an SMS to my starter? Let’s think about it: cellular signal coverage is pretty much 100% (only remote places are deficient) and SMS can be received pretty much everywhere. Also, very basic, cheap plans can give you this advantage. Got it!

Then HACKD weekend was approaching. I researched a little bit about which board to use, there’s a bunch of options out there, not just the arduino and the raspberry pi, but settled on the arduino basically because it was the right price and the complexity level required.

HOW TO

After deciding what to use and how to use it, I got everything I thought I would need.

BOM

  • Arduino Uno Rev 3 (link)
  • GPRS Module (Geeetech SIMCOM SIM900 Quad-band GSM GPRS Shield ) (link)
  • Breadboard and Jumper wires (link)
  • Soldering Kit (link)
  • Multimeter (link)
  • Cables
  • Voltage Regulator (link)
  • Switch Relay (link)

* The breadboard, jumper wires, soldering kit and multimeter links are just for reference, you can get other model, styles, brands, etc. You also don’t need to buy anything from Amazon, you can get everything from ebay or electronics specialized stores.

* Voltage regulator. Arduino documentation says that it works with 5V to 20V, but it can became unstable under 7V and you can burn it over 12V. Common car voltage is 12V so I needed this to convert down the voltage to turn on the arduino. However after 2 or 3 days @ 5V it became unstable. I switched since then to a 9V Voltage Regulator and that solved the problem.

Well that’s pretty much everything I needed. Now the process.

PROCESS

Installing a remote starter to a modern car isn’t an easy task. There are now transponders in the keys that you have to know how to bypass, you need to know different signals for several different instruments of the car, and you also need some security restrictions, like make it so the car shuts off some time after being started without a key (you could forget it was started, for example) or make it so the car can’t be driven without a key. (A thief could take advantage of the remote starter to just jump in and drive.) Because all of that logic was already implemented on the remote starter my car already had and the hackathon was a 3 day event, I decided just to hang my stuff off the installation already there.

My plan then was that I would send an SMS from my cell to the arduino (received via the GPRS Module). Then the arduino would send a signal thru a cable to the remote starter via it’s remote control. All of this would be located on a hidden place inside the car. Here’s the diagram.

 

alarma1 (1)

So I took out the remote from it’s case and connected one cable on each pole of each button

IMG_20141008_182952 IMG-20141011-WA0009 IMG-20141011-WA0040

Then I connected one switch relay to each button on the switches I had on the COM and N.O. pins

IMG-20141011-WA0003

Then, all three switches connected to all three buttons of the remote need to connect one of their left pins to the negative of the arduino and each of the left pins on each of the three switches have to be connected to the three 5v pins that we are going to use to send the signals to it. So at the end all connected looked like this:

IMG-20141011-WA0042 IMG-20141011-WA0045

This module has to be connected to a continuous power source from the car; each car is different so anyone else trying has to figure that out. I added a voltage regulator because the reasons I already stated above.

SOFTWARE

The language Arduino basically uses is C++. Arduino has an IDE that we can use, downloadable from here. I used it for this project and it helps and works fine. Then we also need a Serial Terminal Software to communicate with the Arduino and the Geeetech SIMCOM SIM900 Quad-band GSM GPRS Shield, to send and receive SMS messages. On the Shield documentation located here, they offer some different options for different platforms, I downloaded CoolTerm for my Mac OSX from here and it worked perfectly. There’s some setup you need to do in it to communicate with the Arduino and the Shield. Ah! You also need a SIM Card with a plan installed on the Geeetech Shield.

Terminal Pre-setup

  1. Connect to the computer USB port and turn on Arduino (with the Geeetech Shield mounted).
  2. Go to CoolTerm and click options.
  3. Pick port, usually usbmodemfd something, if doesn’t appear click Re-Scan Serial Ports at the bottom.
  4. Pick 19200 Baud Rate.
  5. Ready to send and receive commands and read them.

There’s a second “language” or set of commands to learn here, and those are the ones that are used to actually send and receive the SMS messages. There’s a whole bunch of commands, not only for SMS, there’s commands to make a call, to receive it, to store, delete, refresh SMS, and all SIM card activities, even internet browsing. That reference can be located here.

Arduino Code

That code can be also viewed here and is going to be running in a neverending loop in the arduino and basically waits for the SIM card to get an SMS. Once the SMS arrives, it sends a double pulse (needed by the remote buttons) to the desired remote button, depending of the content of the SMS received. After that it deletes all the messages to avoid filling up the SIM Card.

Well, after “uploading” that to our Arduino, we can manually send SMS messages to our installation and it should work. This is a little video when I was testing it:

This is with the car already connected to the module:

I have to thank for this part of the code to my friends and coworkers Bob Dauth and Diana Shkolnikov, who helped me a lot with C++ because I had no C++ coding experience at all. After that, it was mostly done, however, I wanted an app that with a tap of a button I could tell my car to start or to open and close the door locks. I asked my also friend and coworker David Breneisen to help me with that. He came up in less than half hour with a very simple app that does the work. Here’s a screenshot:

Screenshot_2015-02-25-09-36-17Pretty much what it does is to send the SMS when you tap the button and that’s it. It also gets confirmation and shows a toast message when the SMS has been sent. Because its so many files you can find the code here. After putting that together, we finished our project.

I was able to use it all winter long and it was awesome!

Now, for this most recent HACKD, I wanted to improve on it. I projected the following changes:

  1. Remove the phone number from the code in the app and put a settings page so the user can change the phone number of the car.
  2. Get better-looking buttons with icons instead of just text.
  3. Get reporting back from the car when the car has been started, closed, or opened after the SMS is received.
  4. Add an alert to be sent by the car when a door has been opened.

This last one representing the biggest challenge because I needed to hook up more stuff to the car. I wanted to get reporting back from the car when it has started or closed or opened because there were times where my cell plan has expired and the SMS weren’t going out, I didn’t noticed that the car wasn’t started, and I was going out in the cold just to find out that the car hasn’t been started. Or vice-versa, the SMS was sent, but the car didn’t get the order, because the plan on the Arduino’s chip was expired.

For these new challenges I had to construct a new, more complex circuit represented in the following diagram:

schemeit-project

I have to thank here to my friend from México, Carlos Pinto; he is an Electronics Engineer and helped me with coming up with this circuit. A little bit of description.

1. The icons on the top left corner are switches on the doors, this switches send the signal to the car computer to let it know a door has been opened, you can see an light coming off on your dashboard when a door is open, these are the responsibles.

We need to connect those sensors to the Arduino somehow to get the signal. There’s some considerations to make — these are negative signals, and we need diodes in order to avoid the signal to go between them.

2. To the center (Doors Relay), it’s basically a signal converter. That relay basically takes the negative signal from any of the doors and sends a positive signal to a voltage regulator.

3. The signal from the Doors Relay is converted to a 12V signal so that’s why we need a voltage regulator, to convert the 12V to a 5V signal that is going to be connected to an input pin on the Arduino (we convert it to 5V because that’s what the Arduino supports).

The rest of the circuit is pretty much the same as it was, just the connection from the car to power up the Arduino and the switches needed for sending the signals to the remote control of the starter. Then we need to update our code for the Arduino as follows:

This code is also on my Github and I have to thank Drew Denardo also friend and coworker for helping me with this code and putting together the new version of the Android app that now looks like this:

Screenshot_2015-04-22-17-56-40 Screenshot_2015-04-22-17-56-55

The code for this version can be also viewed here. After presenting and wrapping up the new version of the remote starter in the HACKD event, I realized an issue on the way it is programmed now; it sends me an SMS when a door has been opened or closed, even when I open or close the door. Obviously I don’t need an SMS telling me a door has been opened/closed if I’m the one doing it. I need to add some kind of logic to tell it when it is me doing or not. But I guess that’s going to be for the next HACKD?