Dollarydoo’s Back End

Originally I built the back end, or The Server, for Dollarydoo using Typescript, Node.js, and Express. I had a single app.ts file that exposed the different endpoints via Express and I kept this file pretty lean. It did input validation before calling functions on an instance of my Blockchain class.

It got the job done, it worked, but it cost money. It costs money to run EC2 instances all the time. This is a Potato Developer system. It had to be cheap if not free! After getting the whole back end up and running using AWS Elastic Beanstalk I noticed some pretty bad architectural decisions had been made in my haste to get something built. That’s fine. Cheap prototypes are something to strive for, and the best code you can write is code that can be deleted.

When the Node.js app restarted my blockchain was wiped from memory and everyone’s accomplishments (read: me, mining my own toy crypto) was lost forever. Kinda bad.

I rebuilt it in 2020.

Thankfully the vast majority of my Blockchain class was copied over verbatim, and all of my unit tests still passed. I just changed the way the blockchain was persisted. I know we said NO DATABASES earlier, and so with that on mind my method of persistence changed to serialising the JSON object (the blockchain’s chain property) to a file in S3.

The application itself was cut up. The app.ts file for the Express app was turned into multiple discrete serverless lambdas. Those are free if you don’t get much traffic, and I could fill a football stadium with the amount of traffic I wasn’t getting. Perfect.

I mucked around for a while getting the Cloud Formation written, IAM policies are a fun hurdle to get over and probably the hardest part. Oh, except for API Gateway, that was not very easy. Definitely that was hardest part except for the Github actions. I was spiralling into devops hell for a few days, and then everything was gravy.

…

I still don’t like yaml.

Anyway the really really high level setup looks like this:

Route 53 -> Cloudfront -> API Gateway -> Lambda ---sometimes---> S3 Bucket

Let’s talk about how Dollarydoo’s Blockchain worked. This is the entire Blockchain class. It’s imported by each of my lambdas and manages the whole show - except for business logic that I’ve extracted out for unit testing purposes.

// blockchain.ts
import * as crypto from "crypto"
import { decryptData, getHash } from "../../../common/crypto_helpers"
import { getBalance, proportionalRewardForMember, isValidProof } from "../../../common/business_logic"
import {
  INVALID_DETAILS,
  FRAUDULENT_TRANSACTION,
  INSUFFICIENT_FUNDS,
  STRING_TO_FIND,
  REWARD_PER_PROOF,
  MiningMember,
  Transaction,
  Block,
  DOLLARYDOO_PRIME_MINISTER
} from "../../../common/constants"
import { S3 } from "aws-sdk"
import fetch from "node-fetch"
import { TRANSACTIONS_URL } from "./constants"
import { PutObjectOutput } from "aws-sdk/clients/s3"

export class Blockchain {
  public chain: Array<Block>
  public readonly publicKey: string
  private ecdh: crypto.ECDH

  constructor() {
    this.ecdh = crypto.createECDH("secp256k1")
    this.ecdh.generateKeys()
    // there is no way that this can go bad
    this.ecdh.setPrivateKey(
      "Believe it or not, Secrets Manager costs money to keep a single secret. This is not the real private key",
      "hex"
    )
    this.publicKey = this.ecdh.getPublicKey("hex", "compressed")
  }

  /**
   * Grab the blockchain from S3
   * @param url file in S3
   */
  public async loadChain(url: string): Promise<boolean> {
    return fetch(url).then(
      (response) => {
        return response.json().then((json) => {
          this.chain = json
          return true
        })
      },
      (error) => {
        console.log(error)
        return false
      }
    )
  }

  /**
   * Append a block to the blockchain
   * @param proof proof of work (number)
   * @param previous_hash the hash of the previous block in the blockchain
   * @param transactions the list of transactions to be baked into this block
   */
  private addBlock(proof: number, previous_hash: string, transactions: Transaction[]): Block {
    const block: Block = {
      index: this.chain.length,
      proof: proof,
      previous_hash: previous_hash,
      timestamp: Date.now(),
      transactions
    }
    this.chain.push(block)
    return block
  }

  protected async decryptMessage(message: string, senderPublicKey: string): Promise<any> {
    const sharedSecret = this.ecdh.computeSecret(senderPublicKey, "hex")
    return decryptData(message, sharedSecret).then((message: string) => {
      return message
    })
  }

  // APIs
  public async attemptToSubmitProof(proof: number, from: string): Promise<Block> {
    return new Promise<Block>((resolve, reject) => {
      const lastBlock = this.chain[this.chain.length - 1]
      // see if it's a valid proof
      const validProof = isValidProof(lastBlock.proof, proof, STRING_TO_FIND)
      if (!validProof) {
        throw Error("Proof not accepted. Someone may have beaten you to it!")
      }

      this.getPendingTransactionsFromS3(TRANSACTIONS_URL).then((transactions) => {
        // reward the brave soul who found this valid proof of work
        const transaction: Transaction = {
          from: DOLLARYDOO_PRIME_MINISTER,
          to: from,
          amount: REWARD_PER_PROOF
        }

        transactions.push(transaction)

        const chainTail = JSON.stringify(this.chain[this.chain.length - 1])
        const hashedChainTail = getHash(chainTail)
        // Add a new block to the blockchain
        this.addBlock(proof, hashedChainTail, transactions)

        return this.putBlockChainInS3().then(
          () => {
            // clear out the pending transactions in the bucket
            return this.putPendingTransactionsInS3([]).then(() => {
              // return the chain tail to the original caller
              resolve(this.chain[this.chain.length - 1])
            })
          },
          (error) => {
            reject(error)
          }
        )
      })
    })
  }

  public async attemptToSubmitTransaction(
    encryptedBlob: string,
    senderPublicKey: string,
    transaction: Transaction
  ): Promise<Transaction> {
    //short circuit early
    // you can not submit a transaction `to` yourself
    // you can not submit a transaction where it isn't `from` you
    if (transaction.to === senderPublicKey || transaction.from !== senderPublicKey) {
      throw Error(INVALID_DETAILS)
    }
    if (transaction.amount <= 0) {
      throw Error(INVALID_DETAILS)
    }
    const validTransaction = await this.decryptMessage(encryptedBlob, senderPublicKey)
      .then(async (decrypted: any) => {
        const shaResult = getHash(JSON.stringify(transaction))
        if (shaResult === decrypted) {
          return this.getPendingTransactionsFromS3(TRANSACTIONS_URL).then(async (transactions) => {
            const balance = await getBalance(senderPublicKey, this.chain, transactions)
            if (balance < transaction.amount) {
              throw Error(INSUFFICIENT_FUNDS)
            }
            transactions.push(transaction)
            return this.putPendingTransactionsInS3(transactions).then(() => {
              return transaction
            })
          })
        }
      })
      .catch((error: Error) => {
        if ([FRAUDULENT_TRANSACTION, INSUFFICIENT_FUNDS, INVALID_DETAILS].indexOf(error.message) < 0) {
          throw Error(FRAUDULENT_TRANSACTION)
        } else {
          throw error
        }
      })
    return validTransaction
  }

  public async pendingTransactions(senderPublicKey: string): Promise<Transaction[]> {
    const pendingTransactions = await this.getPendingTransactionsFromS3(TRANSACTIONS_URL)
    const promise = new Promise<Transaction[]>((resolve, reject) => {
      const pending = pendingTransactions.filter((transaction: Transaction) => {
        return transaction.from === senderPublicKey
      })
      resolve(pending)
    })
    return promise
  }

  public async putBlockChainInS3(): Promise<PutObjectOutput> {
    const s3 = new S3({ region: "ap-southeast-2" })
    return s3
      .putObject({
        Bucket: "dollarydoo.potatodeveloper.com",
        Key: "chain.json",
        ContentType: "application/json",
        CacheControl: "public, max-age=30",
        Body: JSON.stringify(this.chain)
      })
      .promise()
  }

  public async getPendingTransactionsFromS3(url: string): Promise<Transaction[]> {
    return fetch(url, { method: "GET" }).then(
      (response) => {
        return response.json().then((final) => {
          return final
        })
      },
      (error) => {
        console.log(error)
        return false
      }
    )
  }

  public async putPendingTransactionsInS3(transactions: Transaction[]): Promise<PutObjectOutput> {
    const s3 = new S3({ region: "ap-southeast-2" })
    return s3
      .putObject({
        Bucket: "dollarydoo.potatodeveloper.com",
        Key: "pending_transactions.json",
        ContentType: "application/json",
        CacheControl: "no-cache, max-age=0",
        Body: JSON.stringify(transactions)
      })
      .promise()
  }
}

And here is the constants file that it references

export const STRING_TO_FIND = "123456"
export const INVALID_DETAILS = "Pobodies Nerfect! Invalid transaction details"
export const INSUFFICIENT_FUNDS = "That's not a knife that's a spoon. Also you have insufficient funds"
export const FRAUDULENT_TRANSACTION = "Fraudulent transaction detected. Discouraging the boot is a bootable offence."
export const REWARD_PER_PROOF = 5
export const DOLLARYDOO_PRIME_MINISTER = "Andy, drinking a Fosters in a dam."

export const FONT_FAMILY = "Verdana, Geneva, sans-serif"
export const API_BASE_URL = "https://dollarydooapi.potatodeveloper.com"

export interface Transaction {
  from: string
  to: string | MiningMember[]
  amount: number
}

export interface MiningMember {
  publicKey: string
  percentOfEffort: number
}

export interface Block {
  index: number
  proof: number
  previous_hash: string
  timestamp: number
  transactions: Array<Transaction>
}

And finally the business logic file

import { Block, Transaction } from "./constants"
import { getHash } from "./crypto_helpers"
import { MiningMember, DOLLARYDOO_PRIME_MINISTER } from "./constants"

const transactionElements = (
  publicKey: string,
  chain: Block[],
  pendingTransactions?: Transaction[],
  fromPersonOnly?: string
) => {
  const transactionsForPublicKey: Array<Transaction> = []

  // Gather all the transactions for a certain publicKey
  chain.forEach((block: Block, index: number, all: Block[]) => {
    const trans = block.transactions.filter((transaction: Transaction, index: number, all: Transaction[]) => {
      if (fromPersonOnly) {
        return transaction.from === fromPersonOnly
      }
      return transaction.to === publicKey || transaction.from === publicKey
    })
    transactionsForPublicKey.push(...trans)
  })

  // How much have they been GIVEN so far
  const additions = transactionsForPublicKey.reduce((total, transaction, index, array) => {
    if (transaction.to === publicKey) {
      return total + transaction.amount
    }
    return total
  }, 0)

  // How much have they GIVEN AWAY so far
  const deductions = transactionsForPublicKey.reduce((total, transaction, index, array) => {
    if (transaction.from === publicKey) {
      return total + transaction.amount
    }
    return total
  }, 0)

  // How much have the promised to GIVE AWAY when the next block is created
  let pendingDeductions = 0
  if (pendingTransactions) {
    pendingDeductions = pendingTransactions.reduce((total, transaction, index, array) => {
      if (transaction.from === publicKey) {
        return total + transaction.amount
      }
      return total
    }, 0)
  }
  return {
    additions: additions,
    deductions: deductions,
    pendingDeductions: pendingDeductions
  }
}

export const getDollarydooProofRewardAmount = (
  publicKey: string,
  chain: Block[],
  pendingTransactions?: Transaction[]
): number => {
  const { additions, deductions, pendingDeductions } = transactionElements(
    publicKey,
    chain,
    pendingTransactions,
    DOLLARYDOO_PRIME_MINISTER
  )
  return additions - deductions - pendingDeductions
}

export const getBalanceSync = (publicKey: string, chain: Block[], pendingTransactions?: Transaction[]): number => {
  const { additions, deductions, pendingDeductions } = transactionElements(publicKey, chain, pendingTransactions)
  return additions - deductions - pendingDeductions
}

export const getBalance = async (
  publicKey: string,
  chain: Block[],
  pendingTransactions?: Transaction[]
): Promise<number> => {
  const promise = new Promise<number>((resolve, reject) => {
    const { additions, deductions, pendingDeductions } = transactionElements(publicKey, chain, pendingTransactions)
    resolve(additions - deductions - pendingDeductions)
  })

  return promise
}

export const proportionalRewardForMember = (member: MiningMember, rewardPerProof: number): number => {
  return Number((rewardPerProof * (member.percentOfEffort / 100)).toFixed(3))
}

export const isValidProof = (lastKnownProof: number, proof: number, stringToFind: string): boolean => {
  const concatProofs = lastKnownProof.toString() + proof.toString()
  const guess = getHash(concatProofs)
  const guessLength = guess.length
  const halfWay = parseInt((guessLength / 2).toFixed(0))
  const shouldBeInFirstHalf = lastKnownProof % 2 === 0
  const firstHalf = guess.substring(0, halfWay)
  const secondHalf = guess.substring(halfWay, guessLength)
  if (shouldBeInFirstHalf) {
    return firstHalf.indexOf(stringToFind) >= 0
  } else {
    return secondHalf.indexOf(stringToFind) >= 0
  }
}

Aside from my unit tests, there wasn’t much more to it. This is essentially all the core code for the whole lot. Now I needed to expose this functionality to my lambda functions. I had several operations that I wanted to expose via a restful API.

chain.json

https://dollarydoo.potatodeveloper.com/chain.json

This is a file in S3 that is publicly readable. This is my blockchain.

pending_transactions.json

https://dollarydoo.potatodeveloper.com/pending_transactions.json

This is a file in S3 that is publicly readable. This is where all the pending transactions sit. When someone submits a transaction, it gets put here. When someone submits a proof of work, these transactions are stored in that block, and the file is wiped (and the process starts again).

The Functions

GET /publickey

import { Context } from "aws-lambda"
import { responseBuilder } from "../core/response-builder"
import { Blockchain } from "../core/blockchain"
import { LambdaResponse } from "../core/types"

export async function handler(event: unknown, context: Context): Promise<LambdaResponse> {
  const chain = new Blockchain()
  return responseBuilder(200, { publicKey: chain.publicKey }, { "Cache-Control": "max-age=30, Public" })
}

This function returns The Server’s public key. This is fundamental for clients to be able to interact with my blockchain. The Server’s public key is used by all clients to create an Elliptic Curve Public / Private key shared secret. See the first Dollarydoo post for a refresher.

As you can tell by the constructor code in the Blockchain class, the public key is available immediately after an instance is created. The public key is always the same, because the private key is SET to the same value every time. In my Potato Developer system I hard coded the seed data to create a private key. I would never hard code a secret like this into a real app. I will also not give Jeff Bezos my personal money to store a secret in his Secrets Manager for my Potato Developer side project. So here we are. The public key is going to be the same, every time. If it was different each time I created a new instance of the Blockchain class then I would be sunk, the whole system would not work.

The Nodejs crypto API I used is here: Nodejs Crypto docs.

POST /solve

import { Context } from "aws-lambda"
import { responseBuilder } from "../core/response-builder"
import { LambdaResponse } from "../core/types"
import { Blockchain } from "../core/blockchain"
import { BLOCKCHAIN_URL } from "../core/constants"

interface Event {
  body: string
}

interface Submitted {
  from: string
  proof: number
}

const blockChain = new Blockchain()
let chainLoaded = false

export async function handler(event: Event, context: Context): Promise<LambdaResponse> {
  if (!chainLoaded) {
    await blockChain.loadChain(BLOCKCHAIN_URL).then(() => {
      chainLoaded = true
    })
  }

  const submittedData: Submitted = JSON.parse(event.body)

  return blockChain.attemptToSubmitProof(submittedData.proof, submittedData.from).then(
    (block) => {
      return responseBuilder(200, block)
    },
    (error) => {
      return responseBuilder(500, error.message)
    }
  )
}

This is where someone can submit their Proof of Work and get given Dollarydoos as a reward by The Server. Full details about what makes up a valid proof of work will be given in another post. Hang in there, just for now imagine that it’s some number that nobody could predict, given all of the public information available in the blockchain.

POST /transact

import { Context } from "aws-lambda"
import { responseBuilder } from "../core/response-builder"
import { LambdaResponse } from "../core/types"
import { Blockchain } from "../core/blockchain"
import { BLOCKCHAIN_URL } from "../core/constants"
import { Transaction } from "../../../common/constants"

interface Event {
  body: string
}

interface Submitted {
  from: string
  to: string
  amount: number
  sender: string
  encryptedBlob: string
}

const blockChain = new Blockchain()
let chainLoaded = false

export async function handler(event: Event, context: Context): Promise<LambdaResponse> {
  if (!chainLoaded) {
    await blockChain.loadChain(BLOCKCHAIN_URL).then(() => {
      chainLoaded = true
    })
  }

  const submittedData: Submitted = JSON.parse(event.body)
  const transaction: Transaction = {
    from: submittedData.from,
    to: submittedData.to,
    amount: submittedData.amount
  }
  return blockChain.attemptToSubmitTransaction(submittedData.encryptedBlob, submittedData.sender, transaction).then(
    (transaction) => {
      return responseBuilder(200, transaction)
    },
    (error) => {
      return responseBuilder(500, { error: error.message })
    }
  )
}

This is where a person who has previously been given Dollarydoos before can give Dollarydoos to someone else. The function takes in a proposed transaction, and an encryptedBlock object that I’ll give detail about further on. It’s the way that the server knows that the sender is true, valid, is the rightful owner of the from address.

GET /pendingtransactions

import { Context } from "aws-lambda"
import { responseBuilder } from "../core/response-builder"
import { LambdaResponse } from "../core/types"
import { Blockchain } from "../core/blockchain"
import { BLOCKCHAIN_URL } from "../core/constants"

interface Event {
  queryStringParameters: {
    publickey: string
  }
}

const blockChain = new Blockchain()
let chainLoaded = false

export async function handler(event: Event, context: Context): Promise<LambdaResponse> {
  if (!chainLoaded) {
    await blockChain.loadChain(BLOCKCHAIN_URL).then(() => {
      chainLoaded = true
    })
  }

  let publickey
  if (event.queryStringParameters) {
    publickey = event.queryStringParameters.publickey
  }

  if (publickey) {
    const transactions = await blockChain.pendingTransactions(publickey)
    return responseBuilder(200, transactions)
  } else {
    return responseBuilder(400, "Missing 'publickey' query string")
  }
}

The pending transactions gives a list of the pending transactions for a given public key. I.e. for a certain public key xyz, how many dollarydoos have they already promised to someone else. The api returns an array of Transaction objects, and it’s up to the caller of this API to then determine what to do with this data.

Response Builder

//response-builder.ts
import { LambdaResponse, Headers } from "./types"

export function responseBuilder(code = 200, body: any = {}, headers: Headers = {}): Promise<LambdaResponse> {
  headers = {
    ...headers,
    "Access-Control-Allow-Origin": "https://dollarydoo.potatodeveloper.com"
  }
  return Promise.resolve({
    statusCode: code,
    body: `${JSON.stringify(body)}`,
    headers
  })
}

Back ends cool and all, but we need a way to interact with them

No system is complete without a user interface.

Up next - The Front End.