Building the clone
This is the largest clone built in the book and has many components. Unlike the
previous chapters where all the source code are listed in the chapter itself, some of
the less interesting parts of the code are not listed or described here. To get access to
the full source code please go to http://github.com/sausheong/Colony
Configuring the clone
We use a few external APIs in Colony so we need to configure our access to these
APIs. In a Colony all these APikeys and settings are stored in a Ruby file called
config.rb , shown as below:
S3_CONFIG = {}
S3_CONFIG['AWS_ACCESS_KEY'] = ‘<AWS ACCESS KEY>’
S3_CONFIG['AWS_SECRET_KEY'] = ‘<AWS SECRET KEY>’
RPX_API_KEY = ‘<RPX APiKEY>’
Modeling the data
This is the chapter with the largest number of classes and relationships. A few major
classes you see here are similar but not exactly the same as the ones in the previous
chapters, so if you have gone through those chapters you would roughly know
how it works.
The following diagram shows how the clone is modeled:
User
As before the first class we look at is the User class. If you have followed the
previous chapters you will have realized that that this class is very similar to the ones
before. However, the main differences are that there are more relationships with
other classes and the relationship with other users follows that of a friends model
rather than a followers model.
class User
include DataMapper::Resource
property :id, Serial
property :email, String, :length => 255
property :nickname, String, :length => 255
property :formatted_name, String, :length => 255
property :sex, String, :length => 6
property :relationship_status, String
property :provider, String, :length => 255
property :identifier, String, :length => 255
property :photo_url, String, :length => 255
property :location, String, :length => 255
property :description, String, :length => 255
property :interests, Text
property :education, Text
has n, :relationships
has n, :followers, :through => :relationships, :class_name =>
‘User’, :child_key => [:user_id]
has n, :follows, :through => :relationships, :class_name => ‘User’,
:remote_name => :user, :child_key => [:follower_id]
has n, :statuses
belongs_to :wall
has n, :group s, :through => Resource
has n, :sent_messages, :class_name => ‘Message’, :child_key =>
[:user_id]
has n, :received_messages, :class_name => ‘Message’, :child_key =>
[:recipient_id]
has n, :confirms
has n, :confirmed_events, :through => :confirms, :class_name =>
‘Event’, :child_key => [:user_id], :date.gte => Date.today
has n, :pendings
has n, :pending_events, :through => :pendings, :class_name =>
‘Event’, :child_key => [:user_id], :date.gte => Date.today
has n, :requests
has n, :albums
has n, :photos, :through => :albums
has n, :comments
has n, :activities
has n, :pages
validates_is_unique :nickname, :message => “Someone else has taken
up this nickn ame, try something else!”
after : create, :create_s3_bucket
after :create, :create_wall
def add_friend(user)
Relationship.create(:user => user, :follo wer => self)
end
def friends
(followers + follows).uniq
end
def self.find(identifier)
u = first(:identifier => identifier)
u = new(:identifier => identifier) if u.n il?
return u
end
def feed
feed = [] + activities
friends.each do |friend|
feed += friend.activities
end
return feed.sort {|x,y| y.created_at <=> x.created_at}
end
def possessive_pronoun
sex.downcase == ‘male’ ? ‘his’ : ‘her’
end
def pronoun
sex.downcase == ‘ male’ ? ‘he’ : ‘she’
end
def create_s3_bucket
S3.create_ bucket(“fc.#{id}”)
end
def create_wall
self.wall = Wall.create
self.save
end
def all_events
confirmed_eve nts + pending_events
end
def friend_events
events = []
friends.each do |friend|
events += friend.confirmed_events
end
return events.sort {|x,y| y.time <=> x.time}
end
def friend_groups
groups = []
friends.each do |friend|
groups += friend.groups
end
groups – self.groups
end
end
As mentioned in the design section above, the data used in Colony is user-centric. All
data in Colony eventually links up to a user. A user has the following relationships
with other models:
- A user has none, one, or more status updates
- A user is associated with a wall
- A user belongs to none, one, or more groups
- A user has none, one, or more sent and received messages
- A user has none, one, or more confirmed and pending attendances at events
- A user has none, one, or more user invitations
- A user has none, one, or more albums and in each album there are none, one,
or more photos - A user makes none, one, or more comments
- A user has none, one, or more pages
- A user has none, one, or more activities
- Finally of course, a user has one or more friends.
Once a user is created, there are two actions we need to take. Firstly, we need to
create an Amazon S3 bucket for this user, to store his photos.
after :create, :create_s3_bucket
def create_s3_bucket
S3.create_bucket(“fc.#{id}”)
end
We also need to create a wall for the user where he or his friends can post to.
after :create, :create_wall
def create_wall
self.wall = Wall.create
self.save
end
Adding a friend means creating a relationship between the user and the friend.
def add_friend(user)
Relationship.create(:user => user, :follower => self)
end
You might realize that this was a follows relationship in the previous chapters, so you
might ask how could it go both ways? The answer to this question will be clearer
once the discussion turns to sending a request to connect. In short, Colony treats both
following and follows relationships as going both ways—they are both considered as a
friends relationship. The only difference here is who will initiate the request to join. This
is why when we ask the User object to give us its friends, it will add both followers and
follows together and return a unique array representing all the user’s friends.
def friends
(followers + follows).uniq
end
The Relationship class is almost the same as the one used in the other chapters,
except that each time a new relationship is created, an Activity object is also created
to indicate that both users are now friends.
class Relationship
include DataMapper::Resource
property :user_id, Integer, :key => true
property :follower_id, Integer, :key => true
belongs_to :user, :child_key => [:user_id]
belongs_to :follower, :class_name => ‘User’, :child_key =>
[:follower_id]
after :save, :add_activity
def add_activity
Activity.create(:user => user, :activity_type => ‘relationship’,
:text => “<a href=’/user/#{user.nickname}’>#{user.formatted_name}</a>
and <a href=’/user/#{follower.nickname}’>#{follower.formatted_name}</
a> are now friends.”)
end
end
Finally we get the user’s news feed by taking the user’s activities and going through
each of the user’s friends, their activities as well.
def feed
feed = [] + activities
friends.each do |friend|
feed += friend.activities
end
return feed.sort {|x,y| y.created_at <=> x.created_at}
end
Request
We use a simple mechanism for users to invite other users to be their friends. The
mechanism goes like this:
- Alice identifies another Bob whom she wants to befriend and sends
him an invitation. - This creates a Request class which is then attached to Bob.
- When Bob approves the request to be a friend, Alice is added as a friend
(which is essentially making Alice follow Bob, since the definition of a friend
in Colony is someone who is either a follower or follows another user).
class Request
include DataMapper::Resource
property :id, Serial
property :text, Text
property :created_at, DateTime
belongs_to :from, :class_name => User, :child_key => [ :from_id]
belongs_to :user
def approve
self.user.add_friend(self.from)
end
end
Message
Messages in Colony are private messages that are sent between users of Colony.
As a result, messages sent or received are not tracked as activities in the user’s
activity feed.
class Message
include DataMapper::Resource
property :id, Serial
property :subject, String
property :text, Text
property :created_at, DateTime
property :read, Boolean, :default => false
property :thread, Integer
belongs_to :sender, :class_name => ‘User’, :child_key => [:user_id]
belongs_to :recipient, :class_name => ‘User’, :child_key =>
[:recipient_id]
end
A message must have a sender and a recipient, both of which are users.
has n, :sent_messages, :class_name => ‘Message’, :child_key => [:user_
id]
has n, :received_messages, :class_name => ‘Message’, :child_key =>
[:recipient_id]
The read property tells us if the message has been read by the recipient, while the
thread property tells us how to group messages together for display.
Album
The photo sharing capabilities of Colony is transplanted from Photoclone in the
previous chapter and therefore the various models involved in photo sharing are
almost the same as the one in Photoclone. The main difference is that each time an
album is created, an activity is logged.
class Album
include DataMapper::Resource
property :id, Serial
property :name, String, :length => 255
property :description, Text
property :created_at, DateTime
belongs_to :user
has n, :photos
belongs_to :cover_photo, :class_name => ‘Photo’, :child_key =>
[:cover_photo_id]
after :save, :add_activity
def add_activity
Activity.create(:user => user, :activity_type => ‘album’, :text =>
“<a href=’/user/#{user.nickname}’>#{user.formatted_name}</a> created a
new album <a href=’/album/#{self.id}’>#{self.name}</a>”)
end
end
Photo
The Photo class is the main class in the photo-sharing feature of Colony. Just like
the Album class, this is very similar to the one in Photoclone, except for some
minor differences.
class Photo
include DataMapper::Resource
include Commentable
attr_writer :tmpfile
property :id, Serial
property :title, String, :length => 255
property :caption, String, :length => 255
property :privacy, String, :default => ‘public’
property :format, String
property :created_at, DateTime
belongs_to :album
has n, :annotations
has n, :comments
has n, :likes
after :save, :save_image_s3
after :create, :add_activity
after :destroy, :destroy_image_s3
def filename_display; “#{id}.disp”; end
def filename_thumbnail; “#{id}.thmb”; end
def s3_url_thumbnail; S3.get_link(s3_bucket, filename_thumbnail,
Time.now.to_i+ (24*60*60)); end
def s3_url_display; S3.get_link(s3_bucket, filename_display, Time.
now.to_i+ (24*60*60)); end
def url_thumbnail
s3_url_thumbnail
end
def url_display
s3_url_display
end
def previous_in_album
photos = album.photos
index = photos.index self
return nil unless index
photos[index - 1] if index > 0
end
def next_in_album
photos = album.photos
index = photos.index self
return nil unless index
photos[index + 1] if index < album.photos.length
end
def save_image_s3
return unless @tmpfile
img = Magick::Image.from_blob(@tmpfile.open.read).first
display = img.resize_to_fit(500, 500)
S3.put(s3_bucket, filename_display, display.to_blob)
t = img.resize_to_fit(150, 150)
length = t.rows > t.columns ? t.columns : t.rows
thumbnail = t.crop(CenterGravity, length, length)
S3.put(s3_bucket, filename_thumbnail, thumbnail.to_blob)
end
def destroy_image_s3
S3.delete s3_bucket, filename_display
S3.delete s3_bucket, filename_thumbnail
end
def s3_bucket
“fc.#{album.user.id}”
end
def add_activity
Activity.create(:user => album.user, :activity_type => ‘photo’,
:text => “<a href=’/user/#{album.user.nickname}’>#{album.user.
formatted_name}</a> added a new photo – <a href=’/photo/#{self.
id}’><img class=’span-1′ src=’#{self.url_thumbnail}’/></a>”)
end
end
First of all, we removed the feature of storing temporary file caches on the filesystem
of the server. The main reason is that of economy—we want to be able to deliver
everything from Amazon S3 deploy on the Heroku cloud platform (which does
not serve files). Of course this can be changed easily if you’re planning to
customize Colony.
Next, and related to the first difference, is that we no longer store the original
photo. Instead, we only keep a reduced-size display photo and a thumbnail of the
original photo. The rationale for this is the same as with Facebook. Colony is not a
full-fl edged photo-sharing site for photographers and is meant to share photos with
friends only. Therefore storing large original files is unnecessary.
Photos can be commented on so it includes the Commentable module (explained
later). Also each photo has none, one, or more comments and likes.
Finally as with many of the classes in Colony, creating a Photo is considered an
activity and is logged for streaming on the activity feed. Note that we don’t log an
activity after a save, but only after we create an object photo. This is because save
will be called each time the photo object is edited, annotated, or has its caption or
description modified. This is not an activity we want to log in to the activity stream.
class Annotation
include DataMapper::Resource
property :id, Serial
property :description,Text
property 
, Integer
property :y, Integer
property :height, Integer
property :width, Integer
property :created_at, DateTime
belongs_to :photo
after :create, :add_activity
def add_activity
Activity.create(:user => self.photo.album.user, :activity_type
=> ‘annotation’, :text => “<a href=’/user/#{self.photo.album.user.
nickname}’>#{self.photo.album.user.formatted_name}</a> annotated
a photo – <a href=’/photo/#{self.photo.id}’><img class=’span-1′
src=’#{self.photo.url_thumbnail}’/></a> with ‘#{self.description}’”)
end
end
Annotation is another class, part of the photo-sharing feature that is transplanted
from Photoclone, with activity logging added in. We will not go into this, if you want
a refresher please read Chapter 4.
Status
Just as the Album, Photo, and Annotation classes are transplanted from Photoclone,
the Status and Mention classes are derived from Tweetclone.
class Status
include DataMapper::Resource
include Commentable
property :id, Serial
property :text, String, :length => 160
property :created_at, DateTime
belongs_to :recipient, :class_name => “User”, :child_key =>
[:recipient_id]
belongs_to :user
has n, :mentions
has n, :mentioned_users, :through => :mentions, :class_name =>
‘User’, :child_key => [:user_id]
has n, :comments
has n, :likes
before :save do
@mentions = []
process
end
after :save do
unless @mentions.nil?
@mentions.each {|m|
m.status = self
m.save
}
end
Activity.create(:user => user, :activity_type => ‘status’, :text
=> self.text )
end
# general scrubbing
def process
# process url
urls = self.text.scan(URL_REGEXP)
urls.each { |url|
tiny_url = RestClient.get “http://tinyurl.com/api-create.
php?url=#{url[0]}”
self.text.sub!(url[0], “<a href=’#{tiny_url}’>#{tiny_url}</a>”)
}
# process @
ats = self.text.scan(AT_REGEXP)
ats.each { |at|
user = User.first(:nickname => at[1,at.length])
if user
self.text.sub!(at, “<a href=’/#{user.nickname}’>#{at}</a>”)
@mentions << Mention.new(:user => user, :status => self)
end
}
end
def starts_with?(prefix)
prefix = prefix.to_s
self.text[0, prefix.length] == prefix
end
def to_json(*a)
{‘id’ => id, ‘text’ => text, ‘created_at’ => created_at, ‘user’ =>
user.nickname}.to_json(*a)
end
end
As before, each time a user updates his status, an activity will be logged. Statuses
can be commented upon and also liked. The Mention class is unchanged from
Tweetclone. For an in-depth description of this class please refer to Chapter 3.
class Mention
include DataMapper::Resource
property :id, Serial
belongs_to :user
belongs_to :status
end
URL_REGEXP = Regexp.new(‘\b ((https?|telnet|gopher|file|wais|ftp) :
[\w/#~:.?+=&%@!\-] +?) (?=[.:?\-] * (?: [^\w/#~:.?+=&%@!\-]| $ ))’,
Regexp::EXTENDED)
AT_REGEXP = Regexp.new(‘@[\w.@_-]+’, Regexp::EXTENDED)
Group
Each user can belong to none, one, or more groups. Each group that is created also
belongs to a user and it’s this user that the activity is logged to. Each group has a set
of features:
- A group can have none, one, or more pages.
- A group has a wall where other users can post to. This wall is created right
after the group is created.
class Group
include DataMapper::Resource
property :id, Serial
property :name, String
property :description, String
has n, :pages
has n, :members, :class_name => ‘User’, :through => Resource
belongs_to :user
belongs_to :wall
after :create, :create_wall
def create_wall
self.wall = Wall.create
self.save
end
after :create, :add_activity
def add_activity
Activity.create(:user => self.user, :activity_type => ‘event’,
:text => “<a href=’/user/#{self.user.nickname}’>#{self.user.
formatted_name}</a> created a new group – <a href=’/group/#{self.
id}’>#{self.name}</a>.”)
end
end
Note that the User-Group relationship is a many-to-many relationship, and we use the
DataMapper::Resource class as an anonymous class to represent the relationship.
For convenience we also provide a method in the User object to retrieve all groups a
user’s friends belong to. This becomes useful for us later when suggesting groups for
users to join.
def friend_groups
groups = []
friends.each do |friend|
groups += friend.groups
end
groups – self.groups
end
Event
Events are quite similar to Groups but with a twist. As before we log it as an activity
each time the event is created. Each event has an administrative user who is the
person who created the event.
class Event
include DataMapper::Resource
property :id, Serial
property :name, String
property :description, String
property :venue, String
property :date, DateTime
property :time, Time
belongs_to :user
has n, :pages
has n, :confirms
has n, :confirmed_users, :through => :confirms, :class_name =>
‘User’, :child_key => [:event_id], :mutable => true
has n, :pendings
has n, :pending_users, :through => :pendings, :class_name => ‘User’,
:child_key => [:event_id], :mutable => true
has n, :declines
has n, :declined_users, :through => :declines, :class_name =>
‘User’, :child_key => [:event_id], :mutable => true
belongs_to :wall
after :create, :create_wall
def create_wall
self.wall = Wall.create
self.save
end
after :create, :add_activity
def add_activity
Activity.create(:user => self.user, :activity_type => ‘event’,
:text => “<a href=’/user/#{self.user.nickname}’>#{self.user.formatted_
name}</a> created a new event – <a href=’/event/#{self.id}’>#{self.
name}</a>.”)
end
end
In addition, each event has three types of members depending on their current
attendance status:
- Users confirmed to attend the event
- Users who are still undecided on attending the event
- Users who have declined to attend the event
For this implementation we use a separate class for each type of user, that is we
have a Confirm class for confirmed users, a Pending class to indicate users who
are undecided, and a Decline class to indicate users who have declined to
attend the event.
class Pending
include DataMapper::Resource
property :id, Serial
belongs_to :pending_user, :class_name => ‘User’, :child_key =>
[:user_id]
belongs_to :pending_event, :class_name => ‘Event’, :child_key =>
[:event_id]
end
class Decline
include DataMapper::Resource
property :id, Serial
belongs_to :declined_user, :class_name => ‘User’, :child_key =>
[:user_id]
belongs_to :declined_event, :class_name => ‘Event’, :child_key =>
[:event_id]
end
class Confirm
include DataMapper::Resource
property :id, Serial
belongs_to :confirmed_user, :class_name => ‘User’, :child_key =>
[:user_id]
belongs_to :confirmed_event, :class_name => ‘Event’, :child_key =>
[:event_id]
end
As with Group, we have a convenient method in the User class to help us find the
events the user’s friends are attending. We only retrieve confirmed events for this
list, which is then sorted according to ascending chronological order.
def friend_events
events = []
friends.each do |friend|
events += friend.confirmed_events
end
return events.sort {|x,y| y.time <=> x.time}
end
Page
Pages are a simple means for users to publish their own web pages. A Page can be
owned directly by a user, through a group, or through an event.
class Page
include DataMapper::Resource
include Commentable
property :id, Serial
property :title, String
property :body, Text property :created_at, DateTime
has n, :comments
has n, :likes
belongs_to :user
belongs_to :event
belongs_to :group
after :create, :add_activity
def add_activity
if self.event
Activity.create(:user => self.user, :activity_type => ‘event
page’, :text => “<a href=’/user/#{self.user.nickname}’>#{self.user.
formatted_name}</a> created a page – <a href=’/event/page/#{self.
id}’>#{self.title}</a> for the event <a href=’/event/#{self.event.
id}’>#{self.event.name}</a>.”)
elsif self.group
Activity.create(:user => self.user, :activity_type => ‘group
page’, :text => “<a href=’/user/#{self.user.nickname}’>#{self.user.
formatted_name}</a> created a page – <a href=’/group/page/#{self.
id}’>#{self.title}</a> for the group <a href=’/group/#{self.group.
id}’>#{self.group.name}</a>.”)
else
Activity.create(:user => self.user, :activity_type => ‘page’,
:text => “<a href=’/user/#{self.user.nickname}’>#{self.user.formatted_
name}</a> created a page – <a href=’/page/#{self.id}’>#{self.title}</
a>.”)
end
end
end
Page also logs activities according to whichever object that owns it.
Wall
A wall is a place where users can place their posts. A wall can belong to a user, event,
or group. In fact each time a user, event, or group is created, we will automatically
create a wall on its behalf.
class Wall
include DataMapper::Resource
property :id, Serial
has n, :posts
end
The implementation of a wall by itself has no definite properties other than being a
container for posts. A post is the actual content that a user will submit to a wall and
it is something that can be commented and liked. A post on a wall can come from
any user, so a post is also associated with the user who created the post.
class Post
include DataMapper::Resource
include Commentable
property :id, Serial
property :text, Text
property :created_at, DateTime
belongs_to :user
belongs_to :wall
has n, :comments
has n, :likes
end
Activity
An activity is a log of a user’s action in Colony that is streamed to the user’s activity
feed. Not all actions are logged as activities, for example messages are considered
private and are therefore not logged.
class Activity
include DataMapper::Resource
include Commentable
property :id, Serial
property :activity_type, String
property :text, Text
property :created_at, DateTime
has n, :comments
has n, :likes
belongs_to :user
end
Activities are commented and can be liked by other users.
Comment
Comments in Colony are stored and managed through the Comment class. All usergenerated
content including pages, posts, photo, and statuses can be commented by
users in Colony. Activities can also be commented on.
class Comment
include DataMapper::Resource
property :id, Serial
property :text, Text
property :created_at, DateTime
belongs_to :user
belongs_to :page
belongs_to :post
belongs_to :photo
belongs_to :activity
belongs_to :status
end
Like
Like and Comment classes are very similar. The main difference between them is that
the Like mechanism is binary (either you like the content or you don’t) whereas you
need to provide some content to comment.
class Like
include DataMapper::Resource
property :id, Serial
belongs_to :user
belongs_to :page
belongs_to :post
belongs_to :photo
belongs_to :activity
belongs_to :status
end
The implementation of the Like mechanism in Colony requires each class of objects
that can be liked or commented on to include the Commentable module.
module Commentable
def people_who_likes
self.likes.collect { |l| “<a href=’/user/#{l.user.nickname}’>#{l.
user.formatted_name}</a>” }
end
end
This allows you to retrieve an array of people of who likes the content, which are
then formatted as HTML links for easy display.
This wraps up the data models that we will be using in Colony. In the next chapter
we will cover Colony’s application fl ow and deployment.
Summary
This is the second last chapter in this book and also the first one in a series of two
chapters describing how we can clone a social networking service like Facebook.
Social networking services are the next step in the evolution of Internet applications
and Facebook is currently the most successful incarnation of this service. Cloning
Facebook is not difficult though, as can be attested in this chapter and also in the
many Facebook ‘clones’ out there on the Internet. Let’s look at what we have covered
in this chapter.
First, we went through a whirlwind tour of social networking services and their
history, before discussing the most dominant service, Facebook. Next, we described
some of its more essential features and we categorized the features into User,
Community, and Content sharing features. After that, we went into a high level
discussion on these various features and how we implement them in our Facebook
clone, Colony. After that, we went briefl y into the various technologies used in
the clone.
After the technology discussion, we jumped straight into the implementation,
starting with a detailed discussion of the data models used in this chapter. Our
next chapter is the last chapter in this book. We will finish what we have started in
this chapter with a detailed step-by-step description of Colony’s application fl ow,
followed with the deployment of Colony on the Heroku cloud platform.
October 28, 2010
Ruby