On Functional Reactive Programming…

FRP has been getting more and more attention lately in different areas of our industry, with good reason, it’s a tool that wasn’t obvious before.

Skipping that discussion, let’s talk about how we think about state. You try and design your models as immutable structures, right? This means state is fixed. If you want to mutate something, you create a new thing by copying and update the state before you return that copy. Alternatively you create a whole new thing with the data you want changed at the time of creation. It’s the same thing, just an implementation detail.

If you model your data as mutable structures, let me reserve judgement for the moment and just say that for all you know, a changed value has been that value since the object was created. You don’t know when it changed, or at least if you did, you’ll forget at some point. This usually leads to bugs.

I want to take an example of a car, as a piece of state. Let’s model it. A car has a frame, it’s got a body type, a body colour, some wheels of a specific size, shape and appearance. It’s got windows on the sides and the front/back. It probably has a trunk, an engine, a gas tank, seats, ignition, steering wheel, gas pedal and break pedal, as well as a driver at a minimum. There are lots of pieces of state in a car. How much of it changes? Let’s address the obvious items:

  1. If you have a flat tire, you need to change it
  2. If your tire has a lot of wear, you need to change it
  3. If your windshield cracks, you need to change it
  4. If your breaks stick, they must be repaired
  5. If you run out of gas, you need to add more
  6. If you lend your car to your brother, it has a new driver

So on, so on… One thing you’ll quickly notice when you stop and think about modelling a vehicle is, all the things that can effect your state, can themselves be modelled as a stream of events. Also, the pieces of your vehicle themselves generate streams of change events, whenever something happens to them. All of a sudden, we’re not modelling state so much as we’re modelling the actions that occur on subjects. The actions represent the state changes across time, and we, as an observer of the vehicle, can see these changes as they happen. Our model is reactive.

This brings up another fun point, if we start viewing state as time-varying variables, how can I apply transformations to this data, i.e., when its read?

What if we thought of streams as infinite lists? Lists we could apply common techniques like enumeration, folding, filtering, mapping, all of a sudden these streams are becoming more and more useful as we introduce regular concepts we use daily even in the imperative world.

The key takeaway here is that the first impediment to understanding FRP is not to change the way you write your code, not at all. But rather, instead of writing first, and asking questions later, ask questions first, think “How can this be modelled as a stream of signals” and then write code to cover that model.

This can work on more types of problems in more areas than I can cover in a single blog post. I hope this helps.

small dogs / big dogs: I Am Being Watched

smalldogsbigdogs:

About 8 months ago, my wife gave birth to a beautiful little girl named Margot. As she becomes more aware of the world around her, I’ve noticed that she stares at me whenever I’m in the room. Constantly. She always wants to make eye contact. She’s always looking for me to engage with her in some…

Shoulders of giants…

Tracking in iOS

When Apple deprecated the UDID, seemingly developers everywhere were in such panic because they’d written their software to use it so they can track users for whatever purposes, i.e., analytics, high scores, advertising, etc. Soon thereafter, we saw a plethora of applications pop up which created their own little UDID system basically creating a universal ID out of items like MAC addresses.

That approach is fraught with problems as developers will soon find out. For reasons I can’t explain here right now, it’s not tenable going forward. So what can you do?

I’m going to lay out a method by which you should have been doing things before. A way that will continue to work in past operating systems which support these APIs, and a way that going forward will continue to work for the foreseeable future as it is using APIs Apple recommends for this purpose.

I’m not going to focus on advertising applications, but the +[UIDevice advertisingIdentifier] is an API that will interest you.

Instead what I’m going to do is address all other applications. Here’s the basic algorithm:

  1. On startup, check the keychain for the last UDID you saved on the device.
  2. If it’s not there, skip to 4, otherwise load it in memory.
  3. Compare the saved UDID with the current +[UIDevice identifierForVendor]
  4. Save the new one in the keychain overwriting the old one. You’ll also want to update your backend (analytics or whatever) linking the old UDID with the new UDID.

That’s it, all you have to do.

The reason this algorithm is important to use is because the vendor ID is only unique for the set of your applications the user has installed at the time. If they delete the last app they have of yours on their device and reinstall it some time later, they’ll have a different UDID. Therefore to be useful, you have to link the old data to the new data.

Whether or not you should be tracking devices is an entirely different matter for discussion. It’s usually not the right tool for the job.

Be Great

This post deviates from my usually highly technical nature, but I write this for myself. Whenever I feel like I’m not being great, I will return and read my words again. You should too.

If I can leave one bit of wisdom on this world after I long depart, it’s to strive for greatness, pick yourself up and dust yourself off when you fail; keep going. Whatever great means to you, go out and be it.

For me, it’s my family. I’ve been addicted to the computer for so long I had to think about the last time I didn’t worry about when I’d be able to experiment with this idea I’ve been having. To me, being great is being able to take care of my kids, but experience their journey with them, as an active participant, not an on-looker.

I’ve not been the best dad I could be to my kids, but I’m going to change that, starting today.

Retain cycles, ARC and GC: A thought

Anybody who’s been building Mac or iOS apps for a while, or who are even fleetingly familiar with retain counting as a way of managing resources, will be at least familiar with the concept of a retain cycle. I’m only talking to those people, so I don’t end up writing a primer about that topic.

clang for a while now, has been telling us when it thinks we’ve got a retain cycle in some cases, and the static analyzer can give us some info too. We already have some visibility, and it can become better over time. The more we learn how to identify these cases statically, the more it makes sense to build a runtime collector to handle those objects.

Let me explain how this might work.

ARC already inserts retain and release function calls in places it defines as appropriate according to its rules. In cases we know where there’s a likely possibility of a retain cycle, we can promote objects to a “garbage collected” heap for the purposes of reclaiming objects caught in a cycle by some accurate mechanism. This could involve a call like objc_promoteRetainedObject or otherwise being inserted into the code where a retain call might be. This lets a concurrent GC know about it.

Now, I know what you’re thinking, haven’t we already been down this road before? This isn’t pure GC, this is a hybrid approach which might actually stand a chance of working. The GC would serve just objects we’re likely are caught in a retain cycle, which should be a really small portion of the heap at any given time, a minor fraction of it even.

This does leave out an interesting problem though, which is how to demote objects. I only have rough ideas on how to make that happen, but for instance, the compiler isn’t going to know when exactly an object is promoted to be tracked by the GC or not, but it can ask the GC if it has been promoted. If so, it can effectively be a noop. This would also allow us to have the GC demote objects that have survived a cycle through references elsewhere, after the block goes away; thus removing them from tracking by the GC.

This is all very rough and maybe there are technical considerations I’m not thinking about.

What do you think?

Introducing Swarm

We’ve all had cases where we need to communicate between computers or people electronically. There are a plethora of tools built on various protocols that suit the task to varying definitions of well. I’ve decided to throw my hat into the matter with a real simplified protocol built for this purpose.

'Why would you want to do this?' you might ask. 'What's wrong with X, Y or Z?' you might also ask. The fact is, for your purposes, other tools may be better, I built this because I believe it to be better for my purposes.

So what are my purposes you might ask? Well, I’m setting out to build an inherently fault-tolerant local and remote communication protocol. Nothing extremely fancy or clever, just something simple and well suited for the task.

Anyway, enough of that, let’s get into some details.

How does it work?

After deciding what I wanted to build, I went thinking about how I wanted to do it. Almost immediately traditional peer-based networks came to mind as being something reasonably well suited to what I wanted to build. However, most P2P overlay networks tend to be unable to guarantee there’s no data loss when half of the network + 1 node disappear. This is a problem for me. I want the network to appear up to every node, even if it’s entirely detached.

I know I could have solved this appearances problem by using local caching, but in much the same way as Skype, for instance, locally caches, if you go offline before the other user comes online, they won’t get your messages until you’re both online at the same time. This is a problem I’d like to avoid.

The network is structured from the vantage point of a node, as a member of a swarm. A swarm is an independent group of nodes which communicate with one another. All the nodes in a swarm do not necessarily connect to the same node, they can connect to whomever they wish to. A swarm is the formation of all the nodes connected however they are.

You can probably infer, at least for those of you who have built such systems in the past, that the ideal operating mode for this network in terms of latency and ease of finding other nodes, would be limited to a local area network if no bootstrapping process occurs, and you’d be right. Locally, Swarm looks for any other swarm services advertised locally, and connects to those nodes. The ultimate result here, is a directed acyclic graph (DAG for short) that forms the swarm.

The nodes communicate between one another by multicast (primarily) or unicast (explained later) messaging. Messages are constructed with a given purpose, and its payload (if applicable). These messages, upon receipt by another node, it checks if it’s got a message with the given UUID before, if so it ignores the message. Otherwise, it will look at all the nodes it is directly connected to, forward this message on, and record that it has seen the message before it does this.

Let me stop the explanation at this point and explain that how this works was designed for smaller to mid-sized swarms, and not 10 thousand+ nodes in one swarm. While it’s certainly possible to create very large swarms, that’s not what this was designed for.

What will happen is the last node to receive the message, is:

  1. It will forward to all its connected peers;
  2. Each node that then receives the message, will record it;
  3. Nodes don’t care if the receiver received the message.

If the intended receiver later connects, it can request a replay of all the messages sent since a given time. It does not have to request from a specific node, since all nodes in the swarm will receive the message, any node will do.

Unicast messaging

A swarm is inherently multicast. Implementing unicast semantics on top of that is a bit hard given the current way the network works.

To alleviate this problem, we can use use external direct connections to retrieve data.

Let’s assume we are building a chat application and in this chat application, someone can drop a youtube video into the chat. We can recognize this and embed an image of the video into the stream. The receiving node would go out to the internet, and fetch this image. Additionally, when the user clicks on the image to start the video, we’d make another connection out to stream it. These connections operate externally to the control network.

Connecting Swarms

While up to this point we’ve just talked about local connections, the same rules apply for remotes. There may be servers in different data centres which need to communicate with one another in the same swarm. A node may open a socket to these remote services as well on a public listening port, to join two swarms together.

This will be a single point of failure, so if either node X or Y go offline, communications between swarms R and S will be disconnected.

For myself, I’m considering approaching implementing remotes using DNS-SD so I can use bonjour locally and remotely. Additionally, I’m considering using true multicast groups instead of a DAG where possible once I can explore the possibilities of doing so in a fault-tolerant way.

Conclusion

Swarm is an open source protocol to help connect your nodes, whatever they happen to do, in a fault-tolerant manner.

While message replay is not presently implemented, you can fetch Swarm on my GitHub.

: Talk about the problem not the solution

katgleason:

My favorite thing about building a product is that it is really hard. It is hard for a lot of different reasons, but one that has become blatantly obvious is that communication is not something that comes naturally to human beings. We like to think it does, but it doesn’t.

And in most cases in…

Working for results

I’ve been working remotely almost exclusively since I left my telecom company in 2009. Remote work was not a foreign concept to me when I left that company, I had done it for many years, but in a part time capacity.

For me, after reading Richard Branson’s blog post today, I agree vehemently. While I don’t necessarily think that offices are without merit, I do question forcing your employees to work the same way you do, it usually doesn’t end up well.

It’s true that not everyone does well working from home, there’s a very fine line. Being able to see it and know when you’re not being as productive as you could be in an office is an acquired skill any remote employee should have. Your “office” doesn’t have to be an office of course, it could be a beach or a closet for all it matters. It’s a different place than your house, that’s quiet, where you can get some work done differently. It’s essential.

After all, we work towards some goal, some result. If we change our way of thinking from performing tasks to achieving results, I think that is the first step in the right direction to getting rid of the traditional office.

Inheriting Legacy Code Gracefully

Amazing little write up that all developers, not just young ones should take to heart and try and live by.

As a fun surprise, doesn’t just apply to developers, many points apply to everyone.

Women in Tech

I will readily admit, before I was a father of one daughter, let alone 3, I had different views. Views that mostly resembled something like “If more women want to be in the ‘club’ they’ll do what they can to get here, like the rest of us.” However, when women do those things, and then are, as has been well documented, treated the way they often are, it sometimes drives them out of the ‘club’.

What changed my opinion on the matter was having children. Almost instantly one starts thinking about the future we’ll make for our children. I don’t want to have one of my daughters calling me one day in tears that she just can’t take it anymore and wants to leave the thing she loves doing, because of the types of stupid things that shouldn’t happen. I’m pretty sure, most men don’t realize what they put women through with the kinds of comments they make day after day.

For me, I try and think not: “What should I say so that I don’t offend people,” but more to the point: “What would I want others saying to my kids.”

Stop next time before you make that cheeky comment, and think: “Would I want hear someone say this to my daughter?”