Let your users mark up images on iOS with Drawsana

We recently released Drawsana, an open source iOS library in Swift that makes it easy to add image markup features to your own iOS app! This post explains why we did it and how it works.

Asana’s iOS app helps you capture thoughts and information at the moment you think to do it, using the device you carry with you. One way the app serves that purpose is by allowing you to attach images to tasks. Sometimes the image itself isn’t enough; you want to call out a specific part of the image. So we built a feature called markup, which lets you add shapes, drawings, and text to images before attaching them to a task or conversation. Here’s how it works:

Of course, you could always do this kind of thing using a separate app like Annotable, and Apple built a native markup feature into iOS for specific situations, but we want to let people capture important information effortlessly right inside the Asana app.

When we started looking at options for implementing this feature, we did a survey of open source libraries that would handle the hard parts for us: efficiently rendering freehand strokes with Core Graphics, object-oriented tool design, undo/redo stacks, and interactive text entry. Given that there are so many illustration apps on the store, we thought it likely that one of them would have published an open source component. In practice, there were no options that fit all of our criteria:

1. Allows us to build a great UI
2. High-quality rendering
3. Maintainable code
4. Either a minimum set of tools (pen, box, arrow, text), or good extensibility

The one that came close, ACEDrawingView, had all the necessary features, but also came with some hard-to-debug issues, inflexible design, and inflexible UI elements that weren’t a fit for our app. We also considered using a web view and a JavaScript library I wrote a few years ago called Literally Canvas, but the complexity of maintaining a bridge and an embedded web page would have been as burdensome as writing a native component in Swift from scratch.

So during our mid-year Roadmap Week, I used my time away from the sprint cycle to write a high quality, extensible library for building iOS markup features that would be useful not only to us, but also to other iOS developers adding similar features to their own apps. It was more work than using an existing library, but the result was exactly what we wanted.

Before I talk about the library itself, I’d like to call out Let’s Build: Freehand Drawing in iOS by Miguel Angel Quinones, which I used as a starting point for learning the basics. It’s a well-written, useful resource.

Design considerations

Besides collaboration apps like Asana, markup can be leveraged for anything where a feature requires communicating an idea outside of the core content of an image. Telegram, a messaging app, lets you mark up images before you send them. File sharing apps like Dropbox could also benefit. It’s silly that all the individual developers at these companies have to go on the same journey of discovery to learn how to build freehand drawing and editable text box features.

Serving such a variety of apps means that extensibility and customization are necessary. It’s not enough to build a decent UI and a framework for defining custom tools. The UI itself is a liability for an open source project that’s trying to fill this niche. Mobile designers at tech companies are rightfully opinionated about feature sets and UI conventions.

So the design goals when choosing the library were distinct from the goals we had when writing a library:

1. Avoid built-in UI
2. Good-looking, performant rendering
3. Useful set of common tools
4. Exemplary Swift code
5. Extensible design

We think we achieved those goals, and we call the result Drawsana.

Using Drawsana

Here’s a short example showing how to use Drawsana to let a user draw a picture and then save it to JSON or a UIImage.

import Drawsana
 
class MyViewController: UIViewController {
  let drawsanaView = DrawsanaView()
  let penTool = PenTool()
 
  func viewDidLoad() {
    /* ... */
    drawsanaView.set(tool: penTool)
    drawsanaView.userSettings.strokeWidth = 5
    drawsanaView.userSettings.strokeColor = .blue
    drawsanaView.userSettings.fillColor = .yellow
    drawsanaView.userSettings.fontSize = 24
    drawsanaView.userSettings.fontName = "Marker Felt"
  }
 
  func save() {
    let jsonEncoder = JSONEncoder()
    jsonEncoder.outputFormatting = [.prettyPrinted, .sortedKeys]
    let jsonData = try! jsonEncoder.encode(drawingView.drawing)
    // store jsonData somewhere
  }
 
  func load() {
    let data = // load data from somewhere
    let jsonDecoder = JSONDecoder()
    let drawing = try! jsonDecoder.decode(Drawing.self, from: jsonData)
    drawsanaView.drawing = drawing
  }
 
  func showFinalImage() {
    imageView.image = drawsanaView.render()
  }
}

You can find a complete working example on GitHub.

What does it do?

If you’re just trying to ship a markup-like view with as little effort as possible, Drawsana has a nice set of built-in features:

• A simple API that always needs a little bit of configuration, but not too much
• Most of the important tools from MS Paint: pen, eraser, box, ellipse, text, and selection
• Undo/redo
• JSON saving and loading

For those who need more tools outside the built-in set or want to build sophisticated interfaces, Drawsana provides public APIs for everything it can:

• Building tools
• Customizing the text tool’s appearance and behavior
• Adding new kinds of shapes and undo/redo actions

struct AddShapeOperation: DrawingOperation {
  let shape: Shape
 
  func apply(drawing: Drawing) {
    drawing.add(shape: shape)
  }
 
  func revert(drawing: Drawing) {
    drawing.remove(shape: shape)
  }
}

public class LineTool: DrawingTool {
  public var name: String { return "LineTool" }
  public var shapeInProgress: LineShape?
  public var isProgressive: Bool { return false }
 
  public init() { }
 
  public func handleTap(context: ToolOperationContext, point: CGPoint) { }
 
  public func handleDragStart(context: ToolOperationContext, point: CGPoint) {
    shapeInProgress = LineShape()
    shapeInProgress?.a = point
    shapeInProgress?.b = point
    shapeInProgress?.apply(userSettings: context.userSettings)
  }
 
  public func handleDragContinue(context: ToolOperationContext, point: CGPoint, velocity: CGPoint) {
    shapeInProgress?.b = point
  }
 
  public func handleDragEnd(context: ToolOperationContext, point: CGPoint) {
    guard let shape = shapeInProgress else { return }
    shape.b = point
    context.operationStack.apply(operation: AddShapeOperation(shape: shape))
    shapeInProgress = nil
  }
 
  public func handleDragCancel(context: ToolOperationContext, point: CGPoint) {
    handleDragEnd(context: context, point: point)
  }
 
  public func renderShapeInProgress(transientContext: CGContext) {
    shapeInProgress?.render(in: transientContext)
  }
}

private struct AnyEncodable: Encodable {
  let base: Encodable
 
  func encode(to encoder: Encoder) throws {
    try base.encode(to: encoder)
  }
}
 
public class Drawing: Codable {
  /* ... */
  public func encode(to encoder: Encoder) throws {
    /* ... */
    try container.encode(shapes.map({ AnyEncodable(base: $0) }), forKey: .shapes)
  }
}

What’s next?

We hope that by releasing this library, we help unlock the creativity of app developers everywhere by breaking down barriers to turning ideas into apps. We also hope that if you like it, you help us improve it—Drawsana is MIT licensed and open for contributions! We’re interested in your feedback about how well it works for your needs, as well as any suggestions for API improvements or implementations of new tools and features. Come on over to GitHub and get started!