title: Education App Design Ideas subtitle: 27 September 2022 description: Some design snips for a learning app image: content/blog/2022/27-09/image1.png published: true

General App Overview

Content Creation and Editing Experience

Alternate Themes

title: Read Metadata from Images using Rust subtitle: 25 August 2022 description: Using Rust to parse EXIF metadata from image files published: true

The complete Rust code discussed in this post can be found in the exiflib GitHub repo

Introduction

Image files, such as JPEG, PNG, and RAW formats from digital cameras and software, contain metadata about the image. This metadata can contain information ranging from the make and model of the camera used to the specific shooting conditions under which a picture was taken

Reading this data depends on the image format used. This post looks at specifically reading metadata from images that use the Exchangeable Image File Format (EXIF) for storing metadata

The Rust Programming Language

The Rust programming language is used to read and process the image files. Rust is a general-purpose programming language with an emphasis on performance and type safety

While this post doesn't cover the specifics of programming in Rust, any code samples are accompanied by a description of what the code does but it's useful to have a basic understanding of programming for understanding exactly what the code is doing

It's also worth noting that this post covers a lot of bit-level processing of image files, to get a basic understanding of binary data works take a look at the previous post on Understanding Binary File Formats

The Exchangeable Image File Format (EXIF)

The Exchangeable Image File Format (EXIF) is based on the Tag Image File Format (TIFF) specification for storing metadata. This data is organised into Image File Directories (IFDs) within an image file

The EXIF section in an image file is structured as follows:

Reading EXIF Data

Reading EXIF data is done by reading the bytes in the file. The following examples will use a JPEG from a Fujifilm X-T200 as a reference, though the same concepts can be applied to understanding data from any file format that stores metadata using the EXIF structure

Under the Hood of an Image File

Below is a snippet of the Hex data for a JPEG file alongside the bytes decoded as text:

Hex Data                                           Decoded Text                       Approximate EXIF Subsections

FF D8 FF E1 57 FE 45 78 69 66 00 00 49 49 2A 00    . . . . W . E x i f . . I I * .    Header, Byte Order, Magic Number
08 00 00 00 0C 00 0F 01 02 00 09 00 00 00 9E 00    . . . . . . . . . . . . . . . .    Data Count, Data Start Location
00 00 10 01 02 00 07 00 00 00 A8 00 00 00 12 01    . . . . . . . . . . . . . . . .    |
03 00 01 00 00 00 01 00 00 00 1A 01 05 00 01 00    . . . . . . . . . . . . . . . .    |
00 00 B0 00 00 00 1B 01 05 00 01 00 00 00 B8 00    . . . . . . . . . . . . . . . .    | Data Entries
00 00 28 01 03 00 01 00 00 00 02 00 00 00 31 01    . . ( . . . . . . . . . . . 1 .    | 
02 00 1E 00 00 00 C0 00 00 00 32 01 02 00 14 00    . . . . . . . . . . 2 . . . . .    |_ 
00 00 DE 00 00 00 13 02 03 00 01 00 00 00 02 00    . . . . . . . . . . . . . . . .    | 
00 00 98 82 02 00 05 00 00 00 F2 00 00 00 69 87    . . . . . . . . . . . . . . i .    |
04 00 01 00 00 00 14 01 00 00 A5 C4 07 00 1C 00    . . . . . . . . . . . . . . . .    | Additional Data Section
00 00 F8 00 00 00 EC 29 00 00 46 55 4A 49 46 49    . . . . . . . ) . . F U J I F I    |
4C 4D 00 00 58 2D 54 32 30 30 00 00 48 00 00 00    L M . . X - T 2 0 0 . . H . . .    |
01 00 00 00 48 00 00 00 01 00 00 00 44 69 67 69    . . . . H . . . . . . . D i g i    |

The above snippet shows the hex data, in here the EXIF marker can be found on the first line - 45 78 69 66 00 00 which decodes to EXIF\0\, followed by the byte order 49 49 - II which means that the byte order for the file is Little Endian - which means that the smallest value in a sequence is the first byte - this can be used to decode 2A 00 to 42 if the byte order was Big endian the bytes representing 42 would be flipped

The byte order section is the most important thing to note on this first line as it tells an application how to read the data in the IFD as well as it is what any byte offsets should be calculated relative to

Additionally, the data entries section and the additional data section are broadly marked off, to understand where data is located in this file

Reading a File as Bytes

Rust provides a method for reading a file in the standard library is fs::read which can be used by providing it with a path to the file to read, the code for this looks like so:

let file = fs::read("./sample.jpg").unwrap();

The .unwrap() at the end tells rust to either get the file data or exit the program with an error if it could not read the file

The result of this is a Vec[u8] which means a vector (or list) of bytes - the bytes in the list are represented as integer values between 0 and 255, these are equivalent to the hex values in the snippet above

The file is what is used to read the bytes from and will be the data source for reading the EXIF data

Finding the EXIF Starting Point

To find the starting point of the EXIF data we can scan through the file until we find the Exif\0\0 pattern, a function can be defined for searching for a pattern in a list of bytes:

pub fn get_sequence_range(bytes: &[u8], pattern: &[u8]) -> Option<Range<usize>> {
    let start = bytes
        .windows(pattern.len())
        .position(|window| window == pattern)?;

    let end = start + pattern.len();

    Some(start..end)
}

The function above called get_sequence_range searches the bytes for a pattern. This uses the windows function in rust which creates a bunch of smaller lists and finds the position where the window, which is a section of bytes that's the same length as the search pattern and checks if the value is equal to the pattern

If the pattern can be found, then the function will return a range (basically, a start and end point) that goes from the start of the found pattern until the end of the pattern, which is simply the start value plus the length of the pattern

The Option indicates that the function returns either a value if it finds one (denoted by Some) or will return nothing if no value is found, denoted by None. The above function uses the shorthand for the None case which is done by placing a ? at the end of the check for the pattern - which will cause the function to end early if it could not find the pattern

The above function for finding the starting point can be used by passing it the file's bytes like so:

const EXIF_MARKER: &[u8] = "Exif\0\0".as_bytes();

let exif_range = get_sequence_range(file, EXIF_MARKER)?;

Note that in the above function the EXIF_MARKER is defined as the Exif\0\0 text converted to bytes, this is passed as the search pattern to the get_sequence_range function. This gets the EXIF header location which is used to find the Byte order (Endian) marker

Getting the Byte Order

Once we know the location of the EXIF marker, the byte order values go from the 6th and 7th byte after the start of the marker. Since this is done using a range, this means that the range goes from 6 to 8, since the end value is not included in the range, this can be see defined below:

const ENDIAN_RANGE_FROM_EXIF_MARKER: Range<usize> = 6..8;

The bytes for the endian value can be found relative to the exif_range like so:

let start = exif_range.start + ENDIAN_RANGE_FROM_EXIF_MARKER.start;

let bytes = file.get(start..)?;
let endian_bytes = bytes.get(0..2)?;
let endian = get_endian(endian_bytes)?;

The above makes use of a get_endian function to get the byte order which can be defined as follows:

fn get_endian(endian_bytes: &[u8]) -> Option<Endian> {
    let endian = parsing::full_bytes_string(endian_bytes)?;

    match endian.as_str() {
        "MM" => Some(Endian::Big),
        "II" => Some(Endian::Little),
        _ => None,
    }
}

The above function takes the bytes which start at the endian location and converts them to a string (text) value

These values are then compared using a match. If it is II or MM the function returns Big Endian (Endian::Big) or Little Endian (Endian::Little) respectively. If no matching value is found, then None is returned instead

The code above also finds the bytes, which defines the file's bytes but trims off all the bytes that are before the endian marker - this is important since any data in the IFD needs to be read relative to the this location

Following the Endian bytes are 2 bytes which specify the Magic Number (42) as mentioned above - this can also be checked to verify the byte order of the file but is not covered in this post

Getting the IFD Data Start Location and Count

Immediately after the Magic Number is four bytes that specify where the IFD data starts, in the snippet above, these bytes are 08 00 00 00 which convert to the value of 8, this informs us that the IFD data starts from 8 bytes from the Endian marker

By following the offset value, the number of entries in the IFD can be found at the 8 bytes from the Endian marker, in this case, bytes 0C 00 which convert to the value of 12, which indicates that there are 12 entries in the IFD

The code applying the above logic can be seen below:

let ifd0_offset = get_ifd_offset(&endian, ifd)? as usize;
let ifd0_entry_offset = ifd0_offset + 2;

let ifd0_count = u16::from_offset_endian_bytes(&endian, ifd, ifd0_offset)?;

The function which gets the ifd0_offset does the lookup of bytes from the range 4 to 8, relative to the Endian marker

Reading Entries in the IFD

As a reference example, the bytes for the first entry in the IFD above will be used to understand the data and how it's stored

After the bytes indicate the count, the next section consists of the entries. Each entry consists of 12 bytes and is structured like so:

• The Tag is an identifier that specifies what the value of the entry represents
• The Data format states how the data should be read
• The Component Length states how many bytes the data for the entry consists of
• The data can either be the actual data, or a value that gives the offset to the data, depending on the Component Lenght

Tag ID

Reading the Tag is done by parsing the first two bytes of an entry - This converts the value into a 16-bit unsigned integer (a positive integer)

The TagID is read like so:

let tag = u16::from_endian_bytes(endian, entry)?;

The value of the tag is a 16-bit unsigned integer, but it's more commonly represented as Hex value in the tag lookup tables, a lookup table for these can be found at the EXIF Tool Tag Names Doc

The value of the tag above 0F 01 can be converted to hex for the Little Endian notation resulting in 0x010F, the lookup table states that this tag identifies the Make property in the Exif data

Data Format

The data stored in an entry can be of 12 different formats, each of these associated with a format value - the format value can be read by reading from byte index 2 in the entry, like so:

let format_value = u16::from_offset_endian_bytes(endian, entry, 2)?;

The format value is a 16-bit unsigned integer, the same as the Tag, though this is used as an integer value and not hex. Each integer value maps to a specific format type, as seen in the below table:

The above table is implemented in code by first defining a type that states all the possible format types:

pub enum TagFormat {
    UnsignedByte,
    AsciiString,
    UnsignedShort,
    UnsignedLong,
    UnsignedRational,
    SignedByte,
    Undefined,
    SignedShort,
    SignedLong,
    SignedRational,
    SingleFloat,
    DoubleFloat,
}

Each type of value can also be described in terms of the data type it stores as follows:

pub enum ExifValue<'a> {
    UnsignedByte(u8),
    AsciiString(String),
    UnsignedShort(u16),
    UnsignedLong(u32),
    UnsignedRational(u32, u32),
    SignedByte(i8),
    Undefined(&'a [u8]),
    SignedShort(i16),
    SignedLong(i32),
    SignedRational(i32, i32),
    SingleFloat(f32),
    DoubleFloat(f64),
}

Thereafter, a function to go from the given Format Value to the type of the tag being used:

fn get_tag_format(value: &u16) -> Option<TagFormat> {
    match value {
        1 => Some(TagFormat::UnsignedByte),
        2 => Some(TagFormat::AsciiString),
        3 => Some(TagFormat::UnsignedShort),
        4 => Some(TagFormat::UnsignedLong),
        5 => Some(TagFormat::UnsignedRational),
        6 => Some(TagFormat::SignedByte),
        7 => Some(TagFormat::Undefined),
        8 => Some(TagFormat::SignedShort),
        9 => Some(TagFormat::SignedLong),
        10 => Some(TagFormat::SignedRational),
        11 => Some(TagFormat::SingleFloat),
        12 => Some(TagFormat::DoubleFloat),
        _ => None,
    }
}

As done previously, if the correct value can't be found, the function returns None

So, adding to the above code, the code for reading the tag value is:

let format_value = u16::from_offset_endian_bytes(endian, entry, 2)?;
let format = get_tag_format(&format_value)?;

Component Length

The Component length specifies the number of components for the tag format being read - for most tag formats this will be 1, however, for specific values like AsciiString or Undefined, this may be different in which case it specifies the length of the string or how many bytes are required respectively

The value for the component length can be found by reading the relevant bytes in the entry and converting them to a 32-bit unsigned integer, starting from byte index 4, like so:

let component_length = u32::from_offset_endian_bytes(endian, entry, 4)?;

Data

Once the component length is known, getting the total length of the data to be read is done by multiplying the component length by the bytes per component - since different components need different amounts of data

A function that gets the bytes per component for a given tag format can be seen below:

fn get_bytes_per_component(format: &TagFormat) -> u32 {
    match format {
        TagFormat::UnsignedByte => 1,
        TagFormat::AsciiString => 1,
        TagFormat::UnsignedShort => 2,
        TagFormat::UnsignedLong => 4,
        TagFormat::UnsignedRational => 8,
        TagFormat::SignedByte => 1,
        TagFormat::Undefined => 1,
        TagFormat::SignedShort => 2,
        TagFormat::SignedLong => 4,
        TagFormat::SignedRational => 8,
        TagFormat::SingleFloat => 4,
        TagFormat::DoubleFloat => 8,
    }
}

This is based on the table shown previously on component formats

Next, the total length can be defined as the component length multiplied by the bytes per component which can be seen in the code below:

let component_length = u32::from_offset_endian_bytes(endian, entry, 4)?;

let bytes_per_component = get_bytes_per_component(&format);

let total_length = component_length * bytes_per_component;

The data can be read from the data bytes, which start at index 8 of the entry

let data = entry.get(8..12)?;

The data value must be the raw bytes because depending on the resulting length it needs to be processed differently

If the total_length is less than or equal to 4, the data can be read directly from the data bytes, this can be done using a function that converts a TagFormat and data to the relevant value as defined in the table:

fn parse_tag_value<'a>(
    format: &TagFormat,
    endian: &'a Endian,
    bytes: &'a [u8],
) -> Option<ExifValue<'a>> {
    match format {
        TagFormat::UnsignedByte => parsing::bytes_to_unsigned_byte(endian, bytes),
        TagFormat::AsciiString => parsing::bytes_to_ascii_string(bytes),
        TagFormat::UnsignedShort => parsing::bytes_to_unsigned_short(endian, bytes),
        TagFormat::UnsignedLong => parsing::bytes_to_unsigned_long(endian, bytes),
        TagFormat::UnsignedRational => parsing::bytes_to_unsigned_rational(endian, bytes),
        TagFormat::SignedByte => parsing::bytes_to_signed_byte(endian, bytes),
        TagFormat::Undefined => parsing::bytes_to_undefined(bytes),
        TagFormat::SignedShort => parsing::bytes_to_signed_short(endian, bytes),
        TagFormat::SignedLong => parsing::bytes_to_signed_long(endian, bytes),
        TagFormat::SignedRational => parsing::bytes_to_signed_rational(endian, bytes),
        TagFormat::SingleFloat => parsing::bytes_to_single_float(endian, bytes),
        TagFormat::DoubleFloat => parsing::bytes_to_double_float(endian, bytes),
    }
}

And the data value can be obtained using the function above like so:

let value = parse_tag_value(&format, endian, data)

However, if the total_length is greater than 4, the data value needs to be read as an offset from the IFD which is then converted, again, using the parse_tag_value function above

// the value needs to be checked at the offset and used from there
let offset = u32::from_endian_bytes(endian, data)?;

let start = (offset) as usize;
let end = start + (length) as usize;

let range = start..end;

let value_bytes = bytes.get(range)?;

let result = parse_tag_value(&format, endian, value_bytes)

Putting all the above together, reading the tag above will give:

Reading Additional Entries

Once a single entry can be read - reading additional entries follows the same pattern. Since the bytes per entry is fixed - always 12 - and the number of entries is known from the IFD count, each entry can be iterated over by going 12 bytes at a time and reading their data individually. A more detailed implementation of this as well as the rest of the code can be found on the exiflib GitHub repo

Conclusion

This post provides a basic outline on reading EXIF data from an image, as well as covers the byte structure for reading EXIF entries from an image file. There's a lot more to reading EXIF data from images, but at a high level the parsing covered here should form a basic grounding in how reading this data works

For further reference and inspiration take a look a the reference list at the end of this post as well as the exiflib GitHub repo mentioned previously

References

Implementation details and guidance for reading metadata from:

• EXIF Tool Tag Names
• EXIF Viewer
• Fujifilm EXIF Viewer
• COMPSCI 365/590F - Bit Twiddling File Formats, Parsing EXIF
• Description of Exif file format (MIT Media)
• Exif Explanation
• Exif Specification
• TIFF Specification

Some reference implementations and libraries:

• rawloader docs
• libopenraw
• rust image

title: Binary Data and File Formats subtitle: 20 August 2022 description: An introduction to bits, bytes, and binary file formats published: true

Introduction

Computers work with data stored in binary formats. Reading and interpreting binary data is an important part of understanding file formats so their data can be read and interpreted

Bits and Bytes, and Binary

Binary files store data using bits

A bit can be either a 1 or 0. When working with bit-data, it's useful to group them into sets of 8, known as bytes

A byte is a sequence of 8-bits which can contain a value ranging from 0 to 255 - these are referred to as the decimal representation

Bytes consist of 8-bit, with each position representing a power of 2 from 2^0 to 2^7, as seen below:

0 0 0 0 0 0 0 0
| | | | | | | |  
| | | | | | | |__ 2^0 - 0 or 1
| | | | | | |____ 2^1 - 0 or 2
| | | | | |______ 2^2 - 0 or 4
| | | | |________ 2^3 - 0 or 8
| | | |__________ 2^4 - 0 or 16
| | |____________ 2^5 - 0 or 32
| |______________ 2^6 - 0 or 64
|________________ 2^7 - 0 or 128

total:                  0 to 255

The byte above represents the value for 0, this is because all the bits have a value of 0

Using the above explanation, the number 1 is represented using the following:

0 0 0 0 0 0 0 1
| | | | | | | |  
| | | | | | | |__ 2^0 - 1
| | | | | | |____ 2^1 - 0
| | | | | |______ 2^2 - 0
| | | | |________ 2^3 - 0
| | | |__________ 2^4 - 0
| | |____________ 2^5 - 0
| |______________ 2^6 - 0
|________________ 2^7 - 0

total:                  1

Where the position for 2^0 is the only bit with a value (1)

Similarly, 2 is represented as:

0 0 0 0 0 0 2 0
| | | | | | | |  
| | | | | | | |__ 2^0 - 0
| | | | | | |____ 2^1 - 2
| | | | | |______ 2^2 - 0
| | | | |________ 2^3 - 0
| | | |__________ 2^4 - 0
| | |____________ 2^5 - 0
| |______________ 2^6 - 0
|________________ 2^7 - 0

total:                  2

Where the bit for 2^1 has a value

Or the number 5 with bits 2^0 and 2^2 having a value:

0 0 0 0 0 1 0 1
| | | | | | | |  
| | | | | | | |__ 2^0 - 1
| | | | | | |____ 2^1 - 0
| | | | | |______ 2^2 - 4
| | | | |________ 2^3 - 0
| | | |__________ 2^4 - 0
| | |____________ 2^5 - 0
| |______________ 2^6 - 0
|________________ 2^7 - 0

total:                  5

Which is calculated by adding 2^0 + 2^2 = 1 + 4 = 5

A larger number, like 234 is:

1 1 1 0 1 0 1 0
| | | | | | | |  
| | | | | | | |__ 2^0 - 0
| | | | | | |____ 2^1 - 2
| | | | | |______ 2^2 - 0
| | | | |________ 2^3 - 8
| | | |__________ 2^4 - 0
| | |____________ 2^5 - 32
| |______________ 2^6 - 64
|________________ 2^7 - 128

total:                  234

The calculation for the above value is:

2^1 + 2^3 + 2^5 + 2^6 + 2^7 = total = 234

When substituting the powers of 2:

2 + 8 + 32 + 64 + 128 = total = 234

The numbers discussed above are all 1-byte (8-bit) numbers, which have a range between 0 and 255, adding bits to the value will allow the representation of bigger numbers, for example, a 2-byte (16-bit) number can have a value from 0 to 65,535

Hexadecimal (Hex)

In the above example, numbers are represented in binary format (e.g. 000000020), or decimal format (e.g. 2)

When looking at binary data, it can be a bit easier to navigate around by representing data in hexadecimal (hex) format - which represents every 4 bits as a value ranging from 0-15, so, similar to the byte example above, but instead using 4-bits:

0 0 0 0
| | | |  
| | | |__ 2^0 - 0 or 1
| | |____ 2^1 - 0 or 2
| |______ 2^2 - 0 or 4
|________ 2^3 - 0 or 8

Using what's already been discussed, the number 12 can be represented in bits as 1100,

Hex numbers additionally convert each of these values into a value from 0-9 or A-F, as seen in the following table:

Using the binary representation, a byte can be represented using 2 Hex values which are taken by using the first 4-bits as the first hex value, and the second 4-bits as the second hex value

For example, the value for 234, represented as bits: 11101010 can be split into 2 sets of 4-bits 1110 1010, using the table above, this becomes EA in hex

Binary Files

Binary files encode data using bits, when viewing them, it's convenient to view the data in them using bytes represented as hex values as was shown above

Binary files usually require some knowledge of how their data is structured to correctly interpret the information. This is usually described in the specification for the file format

Text Files

Plain text files are usually in the UTF-8 or UTF-16 format - this means that they use 8-bits or 16-bits to represent each character, but there are lots of other formats that a text file can use

UTF-8 data can be read by converting the binary data to text data using a table which maps the byte/hex value to the character -for the UTF-8 format, the character "A" is encoded in hex as 41 and "z" is 7a

The Hexadecimal representation for a text file that contains the following UTF-8 data:

Hello World!

Would be stored as a binary file which contains:

48 65 6C 6C 6F 20 57 6F 72 6C 64 21

Putting the hex below the text content, the hex to character mapping can be seen:

H  e  l  l  o     W  o  r  l  d  !
48 65 6C 6C 6F 20 57 6F 72 6C 64 21

Binary files can be viewed in a hex editor to see the raw binary data, but interpreting these files depends on the format used and will differ between file formats

Conclusion

Computers store data using bits. Bits can be structured into sets of 8-bits, called a byte

Data can be represented using bits, bytes, decimal values, or hexadecimal values

Files store data using binary. Binary data can be represented as decimal or hex depending and can be viewed as whichever is appropriate

References

• UTF-8 Character Table
• HexEdit
• Bits, Bytes, and Binary Summary

title: Using React.memo for Controlling Component Rendering subtitle: 16 August 2022 description: Using the react top-level API for debouncing and selectively rendering a component for better performance published: true

React Top Level API

The React library contains some functions at it's top level scope. Amongst these are the built-in hooks (like useState, useCallback, etc.) as well as some other functions for manipulating React Elements directly - which I've covered in a previous post on The React Top Level API

Component Rendering

By default, React will trigger a component render whenever there is a change to its state or props. React.memo allows us to take control of the props triggered render by giving us a way to look into the prop-change process

React.memo is a higher order component (HOC) that allows us to wrap a component and control whether or not it is updated/re-rendered by defining a function that tells react whether or not it's props are different - and effectively whether this should trigger a new render

Doing the above is useful for complex components that don't necessarily need to be rendered every time their props are changed

API Definition

The React Docs give us the following example for the React.memo HOC:

const MyComponent = (props) => {
  /* render using props */
}

const areEqual = (prevProps, nextProps) => {
  /*
  return true if passing nextProps to render would return
  the same result as passing prevProps to render,
  otherwise return false
  */
}

const MyComponentMemo = React.memo(MyComponent, areEqual);

The component MyComponent will be rendered whenever props are changed, however, using React.memo lets us define a function called areEqual that we can can use to tell React.memo whether or not the new props would render a different result to the old props

We can then use MyComponentMemo in place of MyComponent to take control of when the component is rendered

Rendering On a Specific Type of Change

Say we have the specific component TimeDisplay which shows the time that's being passed into it from App:

import './App.css'
import React, { useState, useEffect } from 'react'

interface TimeDisplayProps {
  time: number
}

const TimeDisplay: React.FC<TimeDisplayProps> = ({ time }) => {
  const display = new Date(time).toString()

  return <h1>{display}</h1>
}

export default function App() {
  const [time, setTime] = useState(Date.now())

  useEffect(() => {
    const handle = setInterval(() => {
      setTime(Date.now())
    }, 100)

    return () => {
      clearInterval(handle)
    }
  }, [])

  return (
    <main>
      <TimeDisplay time={time} />
    </main>
  )
}

The TimeDisplay component in our case only displays time to the second, so any millisecond-level changes don't matter to the component and so we can save on those renders by checking if the difference in time is similar to the previous render's time

Let's assume for our purpose that it's acceptable for the time to be delayed by about 5 seconds, we then can define a function called areTimesWithinOneSecond which compares the next render's props with the previous and returns if they are within 5 seconds of each other:

const areTimesWithinFiveSeconds = (prev: TimeDisplayProps, next: TimeDisplayProps): boolean => {
  const diff = next.time - prev.time

  return diff < 5000
}

We can use the above function in a React.memo to define a version of the TimeDisplay component which will prevent unnecessary renders:

const TimeDisplayMemo = React.memo(TimeDisplay, areTimesWithinFiveSeconds)

And it can then be used as a drop-in replacement for the TimeDisplay component:

export default function App() {
  const [time, setTime] = useState(Date.now())

  useEffect(() => {
    const handle = setInterval(() => {
      setTime(Date.now())
    }, 100)

    return () => {
      clearInterval(handle)
    }
  }, [])

  return (
    <main>
      <TimeDisplayMemo time={time} />
    </main>
  )
}

Conclusion

From the above implementation we can see that it's possible to delay rendering of a component using React.memo if the component doesn't need to be re-rendered, hence improving performance by decreasing the number of renders react needs to carry out

REPL

The REPL with the example above can seen below:

References

• The React Docs

title: Dev Tools Update subtitle: 03 August 2022 description: Software development tools and languages I'm using at the moment published: true

Programming Languages

I've primarily switched over to Typescript as my language for building applications, but I've also been learning Rust for the purpose of building some low-level libraries

For Typescript, something I've been finding to be really informative is the Type Challenges Repository which has a lot of challenges along with solutions for modeling and solving complex problems using Typescript's type system

For Rust, the Rust Book along with the No Boilerplate YouTube Channel have been really informative

Frameworks

The current version of my website is built using 11ty which is a static site generator which uses Javascript and templates for generating HTML from a variety of data sources - in my case just JSON and Markdown files

A lot of the professional work I do revolves around the React and React-Native ecosystem, but I've made some changes to my stack for personal projects due to the overhead that comes with using React

For now I'm using 11ty for my website as well as Hugo for some smaller projects

I've moved on from Gatsby for my site due to the slow build times as a static site builder - and the unnecessary complexity of graphql for this purpose along with the overall burden of node modules that comes with it

For other projects I'd like to give the Remix Framework a shot for full stack apps. Remix is a full stack framework for building React applications with a pretty cool looking data fetching and project structure

I've also been playing around with Tauri for building desktop applications. Tauri uses a Rust backend with a bring-your-own client approach and pretty much uses any client side javascript framework - I'm looking towards Svelte for this since it keeps things a lot more vanilla which is a good break from working with the other major frameworks

Code Editor

I'm back to VSCode after a short stint with NeoVim.

My reasons for leaving NeoVim were mostly due to the amount of setup when switching devices as well as the issues I experienced on Windows where configuration management was a bit inconsistent as well as the dependency management for some plugins became a lot of admin. Even preconfigured solutions like LunarVim and AstroNvim didn't work well due to lots of compat issues on Windows

As far as VSCode is concerned, some things I've been using are:

• The GitHub Online Editor
• The GitHub Colorblind Theme (GitHub.github-vscode-theme)
• Code Spell Checker Extension (streetsidesoftware.code-spell-checker)
• GitLens Extension (eamodio.gitlens)
• Hex Editor (ms-vscode.hexeditor)
• Rust Analyzer Extension (rust-lang.rust-analyzer)
• Fix Extension (withfig.fig)

Terminal

Some terminal tools I've been enjoying recently are:

• Hyper - An alternative terminal that's so far been less sluggish and more customizable than the Windows Terminal application
• Fig - Provides autocomplete for commands and integrates with loads of CLI tools to provide detailed suggestions and documentations
• NuShell - A shell that works with the idea of command output as data that can be manipulated and transformed which adds a lot of cool data processing functionality on top of your normal shell comands

The database is defined as a class which has an in-memory store data and a persist method that allows for persisting the database to a file as well as content when an instance is created from an existing JSON file

The code for the database can be seen below:

import { existsSync, readFileSync, writeFileSync } from "fs";

export class Database<TData> {
  public data: TData;

  constructor(private readonly dbPath: string, initial: TData) {
    this.data = this.load(initial);
  }

  public update = (data: Partial<TData>) => this.data = {...this.data, ...data}

  public commit = () => this.persist(this.data);

  private persist = (data: TData) =>
    writeFileSync(this.dbPath, JSON.stringify(data));

  private read = (): TData =>
    JSON.parse(readFileSync(this.dbPath).toString()) as TData;

  private load = (initial: TData): TData => {
    if (!existsSync(this.dbPath)) {
      this.persist(initial);
    }

    const current = this.read();

    return {
      ...initial,
      ...current,
    };
  };
}

Usage

The database can be used by creating an instance and modifying the data in it by using the update method, and can be written to a file using the commit method:

import {Database}from './db'

interface User {
  name: string;
  age: number;
}

interface DB {
  users: User[]
}

const initial: DB = {
  users: []
}

const db = new Database<DB>('./data.json', initial)

db.update({
  users: [{
    name: 'Test',
    age: 20,
  }]
})

db.commit()

Functional Example

The above code is found in the below REPL as a runnable example:

Overview

Something that may come up in practice is a use for a type that allows us to enforce that an object has only the common properties for a given object

For example, given the two objects below:

type AThing = {
  name: string;
  age: number;
  email: string;
  phone: number;
};

type BThing = {
  businessName: string[];
  email: string;
  phone: string;
};

I would like a type that contains only the things that these objects have in common, namely phone and email

A Type for Common Object Keys

This isn't something that typescript has out-of-the-box, however it can be implemented by using some type juggling

First, we define a type called CommonKeys which gets us all the keys which are common in the two objects

type CommonKeys<T, R = {}> = R extends T ? keyof T & CommonKeys<Exclude<T, R>> : keyof T;

The CommonKeys type makes use of a condition to check if R which is the recursive type extends T which is the input type. Based on this, we cut down T one type at a time until there is no more object that can extend R, then for an input type T which is the same as R (an empty object) the result of CommonKeys<{}> will be never since {} has no keys, and will end the recursion

Applying this to the above types, we get:

type ABCommonKeys = CommonKeys<AThing | BThing>;
// type ABCommonKeys = "email" | "phone"

And as a sanity check, we can also apply this to {}:

type Basic = CommonKeys<{}>;
// type Basic = never

A Type for Common Object

Next, we can use the CommonKeys type defined above to create a Common type which wne used with the intersection will result in a type that has all the keys common in all types from the intersection

type Common<T> = Pick<T, CommonKeys<T>>;

We can apply this now to a type of AThing | BThing like so:

type ABCommonObject = Common<AThing | BThing>;
// type ABCommonObject = {
//   email: string;
//   phone: string | number;
// }

And we can see that we have the desired result which is an object with the properties that are common between both input object types

Final Solution

We can put the code from above together into the final solution which is just the two above types:

/** Gets the keys common to any type/union of `T` */
type CommonKeys<T, R = {}> = R extends T ? keyof T & CommonKeys<Exclude<T, R>> : keyof T;

/** Gets an object type containing all keys that exist within a type/union `T` */
type Common<T> = Pick<T, CommonKeys<T>>;

Using typeof

We can use the typeof keyword in javascript to find out whether what type an object is. This is useful if we have an object that can take on different structures, for example

type Data = string[]
type GetData = Data | (() => Data)

In the above example, we have a type called GetData which can be either some data or a function to get data. Using this, we can can create a function which fetches data like so:

const fetchData = (getData: GetData): Data => {
  if (typeof getData === 'function') {
    return getData()
  }

  return getData
}

Using in

Javascript also has the in operator which can be used to infer types by us checking a property of an object

type SimpleData = { 
  name:string;
}

type ComplexData = {
  name: {
    first: string;
    last: string;
  }
  isComplex: true;  
}

type AnyData = SimpleData | ComplexData

We can then use the in operator to check the existence of a property of an object by using it along with a key that we expect to be in one object but not another

const getComplexName = (data: AnyData): string => {
  // isComplex is the name of the key that we expect in `ComplexData` but not `SimpleData`
  if ('isComplex' in data) {
    return [data.name.first, data.name.last].join(' ')
  }

  return data.name
}

Using is

We can use the typescript is keyword to specify that the return of a boolean means that a variable satisfies a specific condition

For example, we can create a function that basically does what the in operator in the above function does:

const isComplex = (data: AnyData): data is ComplexData => {
  return (data as ComplexData).isComplex
}

This can be used in place of the in check in the above example like so:

const getComplexName2 = (data: AnyData): string => {
  // isComplex is the name of the key that we expect in `ComplexData` but not `SimpleData`
  if (isComplex(data)) {
    return [data.name.first, data.name.last].join(' ')
  }

  return data.name
}

References

• My TS Notes
• MDN The in operator
• MDN The typeof operator
• TS Docs on Narrowing

An localhost HTTP proxy is useful for debugging and can be easily defined using Node.js by installing http-proxy

yarn add http-proxy

And then adding the following to an index.js file:

index.js

const httpProxy = require('http-proxy')

const target = 'http://my-target-website.com:1234'

httpProxy.createProxyServer({ target }).listen(8080)

Which will create a server that listens on 8080 and will proxy requests to the target

For a reference on the React Top-Level API you can take a look at the React Docs

Introduction

React allows us to do lots of different things using concepts like composition and higher order components. Most of the time these methods are good enough for us to do what we want, however there are some cases where these methods can prove to be insufficient such as when building complex library components or components that need to allow for dynamic composition or do runtime modification of things like child component props, etc.

For the above purposes we can make use of the React Top-Level API. For the purpose of this writeup I'll be making use of the parts of this API that allow us to modify a component's children and modify their props as well as how they're rendered

Our Goal

For the purpose of this doc I'll be using the React API to get to do the following:

1. Show the count of children (Items) passed to a component (Wrapper)
2. Render each child in a sub-wrapper ItemWrapper
3. Modify the props of the children by adding a position prop

When this is done, we want to render a component that results in the following markup:

<Wrapper>
  <Count />
  <ItemWrapper>
    <Item name="" position=""/>
  </ItemWrapper>
  <ItemWrapper>
    <Item name="" position=""/>
  </ItemWrapper>
  <ItemWrapper>
    <Item name="" position=""/>
  </ItemWrapper>
</Wrapper>

But a consumer can be used like:

<Wrapper>
  <Item name="">
  <Item name="">
  <Item name="">
</Wrapper>

Using React.Children to work with a component's children

The React.Children API (see docs) provides us with some utilities for traversing the children passed to a component

Before we can do any of the following, we need to define the structure of an item. Our Item component is defined as follows:

interface ItemProps { name: string; position?: number; }

const Item: React.FC<ItemProps> = ({ name, position }) => 
  <div>{name}, {position}</div>

Use React.Children.count to get the count

The React.Children.count function counts the number of child nodes passed to a React component, we can use it like so:

const count = React.Children.count(children)

For our example, let's start off by creating a Count component that simply takes a count prop and displays some text:

interface CountProps { count: number; }

const Count: React.FC<CountProps> = ({ count }) => <p>Total: {count}</p>

Next, we can define our Wrapper which will take children and pass the count to our Count component:

const Wrapper: React.FC = ({ children }) => {
  const count = React.Children.count(children)

  return <div>
    <Count count={count}  />
  </div>
}

Use React.Children.map to wrap each child

Next, the React.Children.map function allows us to map over the children of an element and do stuff with it, for example:

const items = React.Children.map(children, (child) => {
  return <ItemWrapper>{child}</ItemWrapper>
})

Based on the above, we can define an ItemWrapper as so:

const ItemWrapper: React.FC = ({ children }) => 
  <li style={{
    backgroundColor: 'lightgrey',
    padding: '10px',
    margin: '20px'
  }} >{children}</li>

And we can update the Wrapper to make use of React.children.map:

const Wrapper: React.FC = ({ children }) => {
  const count = React.Children.count(children)

  const items = React.Children.map(children, (child) => {
    return <ItemWrapper>{child}</ItemWrapper>
  })

  return <div>
    <Count count={count}  />
    <ul>{items}</ul>
  </div>
}

Use React.cloneElement to change child props

Lastly, we want to append a position prop to the Item. To do this we can make use of the React.cloneElement function which allows us to clone an element and modify the props of it. Using this function looks like so:

const childProps = child.props
const newProps = {...child.props, position: index}

const newChild = React.cloneElement(child, newProps)

Integrating this into the React.Children.map function above will result in our Wrapper looking like so:

const Wrapper: React.FC = ({ children }) => {
  const count = React.Children.count(children)

  const items = React.Children.map(children, (child, index) => {
    const childProps = child.props
    const newProps = {...child.props, position: index}
    
    const newChild = React.cloneElement(child, newProps)
    
    return <ItemWrapper>{newChild}</ItemWrapper>
  })

  return <div>
    <Count count={count}  />
    <ul>{items}</ul>
  </div>
}

Use React.isValidElement

We've completed most of what's needed, however if for some reason our child is not a valid react element our component may still crash. To get around this we can use the React.isValidElement function

We can update our map function above to return null if the element is not value:

const items = React.Children.map(children, (child, index) => {
  if (!React.isValidElement(child)) return null

  const childProps = child.props
  const newProps = {...child.props, position: index}
  
  const newChild = React.cloneElement(child, newProps)
  
  return <ItemWrapper>{newChild}</ItemWrapper>
})

Which results in our Wrapper now being:

const Wrapper: React.FC = ({ children }) => {
  const count = React.Children.count(children)
  const items = React.Children.map(children, (child, index) => {
    if (!React.isValidElement(child)) return null
    
    const childProps = child.props
    const newProps = {...child.props, position: index}
    
    const newChild = React.cloneElement(child, newProps)
    
    return <ItemWrapper>{newChild}</ItemWrapper>
  })

  return <div>
    <Count count={count}  />
    <ul>{items}</ul>
  </div>
}

The Result

Lastly, we'll render the above using the App component, the API for the above components should be composable as we outlined initially. The App component will now look like so:

const App: React.FC = () => {
  return (
    <Wrapper>
      <Item name="Apple" />
      <Item name="Banana" />
      <Item name="Chocolate" />
    </Wrapper>
  )
}

And the rendered HTML:

<div>
  <p>Total: 3</p>
  <ul>
    <li style="background-color: lightgrey; padding: 10px; margin: 20px;">
      <div>Apple, 0</div>
    </li>
    <li style="background-color: lightgrey; padding: 10px; margin: 20px;">
      <div>Banana, 1</div>
    </li>
    <li style="background-color: lightgrey; padding: 10px; margin: 20px;">
      <div>Chocolate, 2</div>
    </li>
  </ul>
</div>

And lastly, if you'd like to interact with the code from this sample you can see it in this Repl

Data Visualization with D3 and React

React is a library for building reactive user interfaces using JavaScript (or Typescript) and D3 (short for Data-Driven Documents) is a set of libraries for working with visualizations based on data

Before getting started, I would recommend familiarity with SVG, React, and D3

Some good references for SVG are on the MDN SVG Docs

A good place to start for React would be the React Docs or my React Notes

And lastly, the D3 Docs

Getting Stared

To follow along, you will need to install Node.js and be comfortable using the terminal

I'm going to be using a React App with TypeScript initialized with Vite as follows:

yarn create vite

And then selecting the react-ts option when prompted. Next, install d3 from the project root with:

yarn add d3
yarn add --dev @types/d3

Now that we've got a basic project setup, we can start talking about D3

Scales (d3-scale)

d3-scale Documentation

Broadly, scales allow us to map from one set of values to another set of values,

Scales in D3 are a set of tools which map a dimension of data to a visual variable. They help us go from something like count in our data to something like width in our rendered SVG

We can create scales for a sample dataset like so:

type Datum = {
  name: string
  count: number
}

export const data: Datum[] = [
  { name: "🍊", count: 21 },
  { name: "🍇", count: 13 },
  { name: "🍏", count: 8 },
  { name: "🍌", count: 5 },
  { name: "🍐", count: 3 },
  { name: "🍋", count: 2 },
  { name: "🍎", count: 1 },
  { name: "🍉", count: 1 },
]

Also, a common thing to do when working with scales is to define margins around out image, this is done simply as an object like so:

const margin = {
  top: 20,
  right: 20,
  bottom: 20,
  left: 35,
};

This just helps us simplify some position/layout things down the line

Scales work by taking a value from the domain (data space) and returning a value from range (visual space):

const width = 600;
const height = 400;

const x = d3
  .scaleLinear()
  .domain([0, 10])    // values of the data space
  .range([0, width])  // values of the visual space

const position = x(3) // position = scale(value)

Additionally, there's also the invert method which goes the other way - from range to domain

const position = x(3)      // position === 30
const value = x.invert(30) // value === 3

The invert method is useful for things like calculating a value from a mouse position

D3 has different Scale types:

• Continuous (Linear, Power, Log, Identity, Time, Radial)
• Sequential
• Diverging
• Quantize
• Quantile
• Threshold
• Ordinal (Band, Point)

Continuous Scales

These scales map continuous data to other continuous data

D3 has a few different continuous scale types:

• Linear
• Power
• Log
• Identity
• Radial
• Time
• Sequential Color

For my purposes at the moment I'm going to be looking at the methods for Linear and Sequential Color scales, but the documentation explains all of the above very thoroughly and is worth a read for additional information on their usage

Linear

We can use a linear scale in the fruit example for mapping count to an x width:

const maxX = d3.max(data, (d) => d.count) as number;

const x = d3
  .scaleLinear<number>()
  .domain([0, maxX])
  .range([margin.left, width - margin.right]);

If we don't want the custom domain to range interpolation we can create a custom interpolator. An interpolator is a function that takes a value from the domain and returns the resulting range value

D3 has a few different interpolators included for tasks such as interpolating colors or rounding values

We can create a custom color domain to interpolate over and use the interpolateHsl or interpolateRgb functions:

const color = d3
  .scaleLinear<string>()
  .domain([0, maxX])
  .range(["pink", "lightgreen"])
  .interpolate(d3.interpolateHsl);

Sequential Color

If for some reason we want to use the pre-included color scales

The scaleSequential scale is a method that allows us to map to a color range using an interpolator.

D3 has a few different interpolators we can use with this function like d3.interpolatePurples, d3.interpolateRainbow or d3.interpolateCool among others which look quite nice

We can create a color scale using the d3.interpolatePurples which will map the data to a scale of purples:

const color = d3
  .scaleSequential()
  .domain([0, maxX])
  .interpolator(d3.interpolatePurples);

These can be used instead of the scaleLinear with interpolateHsl for example above but to provide a pre-calibrated color scale

Ordinal Scales

Ordinal scales have a discrete domain and range and are used for the mapping of discrete data. These are a good fit for mapping a scale with categorical data. D3 offers us the following scales:

• Band Scale
• Point Scale

Band Scale

A Band Scale is a type of Ordinal Scale where the output range is continuous and numeric

We can create a mapping for where each of our labels should be positioned with scaleBand:

const names = data.map((d) => d.name);

const y = d3
  .scaleBand()
  .domain(names)
  .range([margin.top, height - margin.bottom])
  .padding(0.1);

The domain can be any size array, unlike in the case of continuous scales where the are usually start and end values

Building a Bar Graph

When creating visuals with D3 there are a few different ways we can output to SVG data. D3 provides us with some methods for creating shapes and elements programmatically via a builder pattern - similar to how we create scales.

However, there are also cases where we would want to define out SVG elements manually, such as when working with React so that the react renderer can handle the rendering of the SVG elements and we can manage our DOM structure in a way that's a bit more representative of the way we work in React

The SVG Root

Every SVG image has to have an svg root element. To help ensure that this root scales correctly we also use it with a viewBox attribute which specifies which portion of the SVG is visible since the contents can go outside of the bounds of the View Box and we may not want to display this overflow content by default

Using the definitions for margin, width and height from before we can get the viewBox for the SVG we're trying to render like so:

const viewBox = `0 ${margin.top} ${width} ${height - margin.top}`;

And then, using that value in the svg element:

return (
  <svg viewBox={viewBox}>
    {/* we will render the graph in here */}
  </svg>
)

At this point we don't really have anything in the SVG, next up we'll do the following:

1. Add Bars to the SVG
2. Add Y Labels to the SVG
3. Add X Labels to the SVG

Bars

We can create Bars using the following:

const bars = data.map((d) => (
  <rect
    key={y(d.name)}
    fill={color(d.count)}
    y={y(d.name)}
    x={x(0)}
    width={x(d.count) - x(0)}
    height={y.bandwidth()}
  />
));

We make use of the x and y functions which help us get the positions for the rect as well as y.bandWidth() and x(d.count) to height and width for the element

We can then add that into the SVG using:

return (
  <svg viewBox={viewBox}>
    <g>{bars}</g>
  </svg>
);

At this point, the resulting SVG will look like this:

Y Labels

Next, using similar concepts as above, we can add the Y Labels:

const yLabels = data.map((d) => (
  <text key={y(d.name)} y={y(d.name)} x={0} dy="0.35em">
    {d.name}
  </text>
));

Next, we can add this into the SVG, and also wrapping the element in a g with a some basic text alignment and translation for positioning it correctly:

return (
  <svg viewBox={viewBox}>
    <g
      fill="steelblue"
      textAnchor="end"
      transform={`translate(${margin.left - 5}, ${y.bandwidth() / 2})`}
    >
      {yLabels}
    </g>
    <g>{bars}</g>
  </svg>
);

The state of the SVG at this point is:

X Labels

Next, we can add the X Labels over each rect using:

const xLabels = data.map((d) => (
  <text key={y(d.name)} y={y(d.name)} x={x(d.count)} dy="0.35em">
    {d.count}
  </text>
));

And the resulting code looks like this:

return (
  <svg viewBox={viewBox}>
    <g
      fill="steelblue"
      textAnchor="end"
      transform={`translate(${margin.left - 5}, ${y.bandwidth() / 2})`}
    >
      {yLabels}
    </g>
    <g>{bars}</g>
    <g
      fill="white"
      textAnchor="end"
      transform={`translate(-6, ${y.bandwidth() / 2})`}
    >
      {xLabels}
    </g>
  </svg>
);

And the final SVG:

Final Result

The code for the entire file/graph can be seen below:

import React from "react";
import * as d3 from "d3";
import { data } from "../data/fruit";

const width = 600;
const height = 400;

const margin = {
  top: 20,
  right: 20,
  bottom: 20,
  left: 35,
};

const maxX = d3.max(data, (d) => d.count) as number;

const x = d3
  .scaleLinear<number>()
  .domain([0, maxX])
  .range([margin.left, width - margin.right])
  .interpolate(d3.interpolateRound);

const names = data.map((d) => d.name);

const y = d3
  .scaleBand()
  .domain(names)
  .range([margin.top, height - margin.bottom])
  .padding(0.1)
  .round(true);

const color = d3
  .scaleSequential()
  .domain([0, maxX])
  .interpolator(d3.interpolateCool);

export const Fruit: React.FC = ({}) => {
  const viewBox = `0 ${margin.top} ${width} ${height - margin.top}`;

  const yLabels = data.map((d) => (
    <text key={y(d.name)} y={y(d.name)} x={0} dy="0.35em">
      {d.name}
    </text>
  ));

  const bars = data.map((d) => (
    <rect
      key={y(d.name)}
      fill={color(d.count)}
      y={y(d.name)}
      x={x(0)}
      width={x(d.count) - x(0)}
      height={y.bandwidth()}
    />
  ));

  const xLabels = data.map((d) => (
    <text key={y(d.name)} y={y(d.name)} x={x(d.count)} dy="0.35em">
      {d.count}
    </text>
  ));

  return (
    <svg viewBox={viewBox}>
      <g
        fill="steelblue"
        textAnchor="end"
        transform={`translate(${margin.left - 5}, ${y.bandwidth() / 2})`}
      >
        {yLabels}
      </g>
      <g>{bars}</g>
      <g
        fill="white"
        textAnchor="end"
        transform={`translate(-6, ${y.bandwidth() / 2})`}
      >
        {xLabels}
      </g>
    </svg>
  );
};

Ticks and Grid Lines

Note that D3 includes a d3-axis package but that doesn't quite work given that we're manually creating the SVG using React and not D3's string-based rendering

We may want to add Ticks and Grid Lines on the X-Axis, we can do this using the scale's ticks method like so:

const xGrid = x.ticks().map((t) => (
  <g key={t}>
    <line
      stroke="lightgrey"
      x1={x(t)}
      y1={margin.top}
      x2={x(t)}
      y2={height - margin.bottom}
    />
    <text fill="darkgrey" textAnchor="middle" x={x(t)} y={height}>
      {t}
    </text>
  </g>
));

And then render this in the svg as:

return (
<svg viewBox={viewBox}>
  <g>{xGrid}</g>
  { /* previous graph content */ }
</svg>
);

The result will look like this:

Building a Line Graph

We can apply all the same as in the Bar Graph before to draw a Line Graph. The example I'll be using consists of a Datum as follows:

export type Datum = {
  date: Date;
  temp: number;
};

Given that the X-Axis is a DateTime we will need to do some additional conversions as well as formatting

Working with Domains

In the context of this graph it would also be useful to have an automatically calculated domain instead of a hardcoded one as in the previous example

We can use the d3.extent function to calculate a domain:

const dateDomain = d3.extent(data, (d) => d.date) as [Date, Date];
const tempDomain = d3.extent(data, (d) => d.temp).reverse() as [number, number];

We can then use this domain definitions in a scale:

const tempScale = d3
  .scaleLinear<number>()
  .domain(tempDomain)
  .range([margin.top, height - margin.bottom])
  .interpolate(d3.interpolateRound);

const dateScale = d3
  .scaleTime()
  .domain(dateDomain)
  .range([margin.left, width - margin.right]);

Create a Line

The d3.line function is useful for creating a d attribute for an SVG path element which defines the line segments

The line function requires x and y mappings. The line for the graph path can be seen as follows:

const line = d3
  .line<Datum>()
  .x((d) => dateScale(d.date))
  .y((d) => tempScale(d.temp))(data) as string;

We also include the Datum type in the above to scope down the type of data allowed in the resulting function

Formatting

D3 includes functions for formatting DateTimes. We can create a formatter for a DateTime as follows:

const formatter = d3.timeFormat("%Y-%m")

We can then use the formatter like so:

formatter(dateTime)

Grid Lines

We can define the X Axis and grid lines similar to how we did it previously:

const xGrid = dateTicks.map((t) => (
  <g key={t.toString()}>
    <line
      stroke="lightgrey"
      x1={dateScale(t)}
      y1={margin.top}
      x2={dateScale(t)}
      y2={height - margin.bottom}
      strokeDasharray={4}
    />
    <text fill="darkgrey" textAnchor="middle" x={dateScale(t)} y={height}>
      {formatter(t)}
    </text>
  </g>
));

And the Y Axis grid lines:

const yGrid = tempTicks.map((t) => (
  <g key={t.toString()}>
    <line
      stroke="lightgrey"
      y1={tempScale(t)}
      x1={margin.left}
      y2={tempScale(t)}
      x2={width - margin.right}
      strokeDasharray={4}
    />
    <text
      fill="darkgrey"
      textAnchor="end"
      y={tempScale(t)}
      x={margin.left - 5}
    >
      {t}
    </text>
  </g>
));

Final result

Using all the values that have been defined above, we can create the overall graph and grid lines like so:

return (
  <svg viewBox={viewBox}>
    <g>{xGrid}</g>
    <g>{yGrid}</g>
    <path d={line} stroke="steelblue" fill="none" />
  </svg>
);

The final code can be seen below:

import React from "react";
import * as d3 from "d3";
import { data, Datum } from "../data/temperature";

const width = 600;
const height = 400;

const margin = {
  top: 20,
  right: 50,
  bottom: 20,
  left: 50,
};

const tempDomain = d3.extent(data, (d) => d.temp).reverse() as [number, number];

const tempScale = d3
  .scaleLinear<number>()
  .domain(tempDomain)
  .range([margin.top, height - margin.bottom])
  .interpolate(d3.interpolateRound);

const tempTicks = tempScale.ticks();

const dateDomain = d3.extent(data, (d) => d.date) as [Date, Date];

const dateScale = d3
  .scaleTime()
  .domain(dateDomain)
  .range([margin.left, width - margin.right]);

const dateTicks = dateScale.ticks(5).concat(dateScale.domain());

const line = d3
  .line<Datum>()
  .x((d) => dateScale(d.date))
  .y((d) => tempScale(d.temp))(data) as string;

const formatter = d3.timeFormat("%Y-%m");

export const Temperature: React.FC = ({}) => {
  const viewBox = `0 0 ${width} ${height}`;

  const xGrid = dateTicks.map((t) => (
    <g key={t.toString()}>
      <line
        stroke="lightgrey"
        x1={dateScale(t)}
        y1={margin.top}
        x2={dateScale(t)}
        y2={height - margin.bottom}
        strokeDasharray={4}
      />
      <text fill="darkgrey" textAnchor="middle" x={dateScale(t)} y={height}>
        {formatter(t)}
      </text>
    </g>
  ));

  const yGrid = tempTicks.map((t) => (
    <g key={t.toString()}>
      <line
        stroke="lightgrey"
        y1={tempScale(t)}
        x1={margin.left}
        y2={tempScale(t)}
        x2={width - margin.right}
        strokeDasharray={4}
      />
      <text
        fill="darkgrey"
        textAnchor="end"
        y={tempScale(t)}
        x={margin.left - 5}
      >
        {t}
      </text>
    </g>
  ));

  return (
    <svg viewBox={viewBox}>
      <g>{xGrid}</g>
      <g>{yGrid}</g>
      <g>
        <path d={line} stroke="steelblue" fill="none" />
      </g>
    </svg>
  );
};

And the resulting SVG will then look something like this:

I often find myself writing a function to JSON.stringify some data to log in either a pretty or flat structure

It's just more of a convenience method, and it's pretty much the same as doing console.log(JSON.stringify(data)) and looks like this:

const _ = (data: any, pretty: boolean = false) => {
  return console.log(JSON.stringify(data, null, pretty ? 2 : 0));
};

And then, when I need to log something:

_(myData)

Or, if I want to pretty print the JSON

_(myData, true)

Something i often find myself needing is a way to create a range in javascript, similar to what python has

Here's a basic implementation of something that works like that:

const range = (start, end, count, includeEnd = false) => {
  const space = (end - start) / (count - includeEnd)
  return new Array(count).fill(0).map((_,i) => start + (i * space))
}

The above will output something like:

const withoutEnd = range(20, 21, 5)
// withoutEnd === [ 20, 20.2, 20.4, 20.6, 20.8 ]

const withEnd = range(20, 21, 5, true)
// withEnd === [ 20, 20.25, 20.5, 20.75, 21 ]

Template literal types provide us a way to combine string types or enums in Typescript.

In the below example, we can make use of a string type called Action and an enum called Resource to define a type which is a combination of an action combined with a resource

type Action = 'UPDATE' | 'CREATE' | 'DELETE'

enum Resource {
  Person = 'Person',
  Product = 'Product',
  Sale = 'Sale'
}

// the `Lowercase` type concerts all string type options to lowercase
type ResourceAction = `${Resource}.${Action}`

Such that the ResourceAction type is now defined as:

type ResourceAction = 
  "Person.UPDATE" 
  | "Person.CREATE" 
  | "Person.DELETE" 
  | "Product.UPDATE" 
  | "Product.CREATE" 
  | "Product.DELETE" 
  | "Sale.UPDATE" 
  | "Sale.CREATE" 
  | "Sale.DELETE"

Now, if you're like me - you probably want your types to be consistent in some way. For this purpose, we can use the Lowercase type as follows:

type ResourceActionLowercase = Lowercase<`${Resource}.${Action}`>

Which results in the following types:

type ResourceActionLowercase = 
  "person.update" 
  | "person.create" 
  | "person.delete" 
  | "product.update" 
  | "product.create" 
  | "product.delete" 
  | "sale.update" 
  | "sale.create" 
  | "sale.delete"

Prior to doing any of the below you will require your ~/.aws/credentials file to be configured with the credentials for your AWS account

Serverless Stack Framework

SST Framework is a framework built on top of CDK for working with Lambdas and other CDK constructs

It provides easy CDK setups and a streamlined debug and deploy process and even has integration with the VSCode debugger to debug stacks on AWS

Init Project

To init a new project use the following command:

npx create-serverless-stack@latest my-sst-app --language typescript

Which will create a Serverless Stack applocation using TypeScript

Run the App

You can run the created project in using the config defined in the sst.json file:

{
  "name": "my-sst-app",
  "stage": "dev",
  "region": "us-east-1",
  "lint": true,
  "typeCheck": true
}

Using the following commands command will build then deploy a dev stack and allow you to interact with it via AWS/browser/Postman/etc.

npm run start

Additionally, running using the above command will also start the application with hot reloading enabled so when you save files the corresponding AWS resources will be redeployed so you can continue testing

The Files

The application is structured like a relatively normal Lambda/CDK app with lib which contains the following CDK code:

Stack

lib/index.ts

import MyStack from "./MyStack";
import * as sst from "@serverless-stack/resources";

export default function main(app: sst.App): void {
  // Set default runtime for all functions
  app.setDefaultFunctionProps({
  runtime: "nodejs12.x"
  });

  new MyStack(app, "my-stack");

  // Add more stacks
}

lib/MyStack.ts

import * as sst from "@serverless-stack/resources";

export default class MyStack extends sst.Stack {
  constructor(scope: sst.App, id: string, props?: sst.StackProps) {
  super(scope, id, props);

  // Create the HTTP API
  const api = new sst.Api(this, "Api", {
    routes: {
    "GET /": "src/lambda.handler"    },
  });

  // Show API endpoint in output
  this.addOutputs({
    "ApiEndpoint": api.httpApi.apiEndpoint,
  });
  }
}

And src which contains the lambda code:

src/lambda.ts

import { APIGatewayProxyEventV2, APIGatewayProxyHandlerV2 } from "aws-lambda";

export const handler: APIGatewayProxyHandlerV2 = async (
  event: APIGatewayProxyEventV2
) => {
  return {
  statusCode: 200,
  headers: { "Content-Type": "text/plain" },
  body: `Hello, World! Your request was received at ${event.requestContext.time}.`,
  };
};

Add a new Endpoint

Using the defined constructs it's really easy for us to add an additional endpoint:

src/hello.ts

import { APIGatewayProxyEventV2, APIGatewayProxyHandlerV2 } from "aws-lambda";

export const handler: APIGatewayProxyHandlerV2 = async (
  event: APIGatewayProxyEventV2
) => {
  const response = {
    data: 'Hello, World! This is another lambda but with JSON'
  }

  return {
  statusCode: 200,
  headers: { "Content-Type": "application/json" },
  body: JSON.stringify(response),
  };
};

And then in the stack we just update the routes:

lib/MyStack.ts

const api = new sst.Api(this, "Api", {
  routes: {
  "GET /": "src/lambda.handler",
  "GET /hello": "src/hello.handler" // new endpoint handler
  },
});

So that the full stack looks like this:

lib/MyStack.ts

import * as sst from "@serverless-stack/resources";

export default class MyStack extends sst.Stack {
  constructor(scope: sst.App, id: string, props?: sst.StackProps) {
  super(scope, id, props);

  // Create the HTTP API
  const api = new sst.Api(this, "Api", {
    routes: {
    "GET /": "src/lambda.handler",
    "GET /hello": "src/hello.handler"
    },
  });

  // Show API endpoint in output
  this.addOutputs({
    "ApiEndpoint": api.httpApi.apiEndpoint,
  });
  }
}

VSCode Debugging

SST supports VSCode Debugging, all that's required is for you to create a .vscode/launch.json filw with the following content:

.vscode/launch.json

{
  "version": "0.2.0",
  "configurations": [
    {
      "name": "Debug SST Start",
      "type": "node",
      "request": "launch",
      "runtimeExecutable": "npm",
      "runtimeArgs": ["start"],
      "port": 9229,
      "skipFiles": ["<node_internals>/**"]
    },
    {
      "name": "Debug SST Tests",
      "type": "node",
      "request": "launch",
      "runtimeExecutable": "${workspaceRoot}/node_modules/.bin/sst",
      "args": ["test", "--runInBand", "--no-cache", "--watchAll=false"],
      "cwd": "${workspaceRoot}",
      "protocol": "inspector",
      "console": "integratedTerminal",
      "internalConsoleOptions": "neverOpen",
      "env": { "CI": "true" },
      "disableOptimisticBPs": true
    }
  ]
} 

This will then allow you to run Debug SST Start which will configure the AWS resources using the npm start command and connect the debugger to the instance so you can debug your functions locally as well as make use of the automated function deployment

Add a DB

From these docs

We can define our table using the sst.Table class:

const table = new sst.Table(this, "Notes", {
  fields: {
  userId: sst.TableFieldType.STRING,
  noteId: sst.TableFieldType.NUMBER
  },
  primaryIndex: {
  partitionKey: "userId", sortKey: "noteId"
  }
})

Next, we can add some endpoint definitions for the functions we'll create as well as access to the table name via the environment:

const api = new sst.Api(this, "Api", {
  defaultFunctionProps: {
      timeout: 60, // increase timeout so we can debug
    environment: {
      tableName: table.dynamodbTable.tableName,
    },
  },
  routes: {
  // .. other routes
    "GET  /notes": "src/notes/getAll.handler", // userId in query
    "GET  /notes/{noteId}": "src/notes/get.handler", // userId in query
    "POST /notes": "src/notes/create.handler"
  },
});

And lastly we can grant the permissions to our api to access the table

api.attachPermissions([table])

Adding the above to the MyStack.ts file results in the following:

import * as sst from "@serverless-stack/resources";

export default class MyStack extends sst.Stack {
  constructor(scope: sst.App, id: string, props?: sst.StackProps) {
  super(scope, id, props);

  const table = new sst.Table(this, "Notes", {
    fields: {
    userId: sst.TableFieldType.STRING,
    noteId: sst.TableFieldType.STRING
    },
    primaryIndex: {
    partitionKey: "userId", sortKey: "noteId"
    }
  })

  // Create the HTTP API
  const api = new sst.Api(this, "Api", {
    defaultFunctionProps: {
    timeout: 60, // increase timeout so we can debug
    environment: {
      tableName: table.dynamodbTable.tableName,
    },
    },
    routes: {
    // .. other routes
    "GET  /notes": "src/notes/getAll.handler", // userId in query
    "GET  /notes/{noteId}": "src/notes/get.handler", // userId in query
    "POST /notes": "src/notes/create.handler"
    },
  });

  api.attachPermissions([table])

  // Show API endpoint in output
  this.addOutputs({
    "ApiEndpoint": api.httpApi.apiEndpoint,
  });
  }
}

Before we go any further, we need to install some dependencies in our app, particularly uuid for generating unique id's for notes, we can install a dependency with:

npm install uuid
npm install aws-sdk

Define Common Structures

We'll also create some general helper functions for returning responses of different types, you can view the details for their files below but these just wrap the response in a status and header as well as stringify the body

src/responses/successResponse.ts

const successResponse = <T>(item: T) => {
  return {
  statusCode: 200,
  headers: { "Content-Type": "application/json" },
  body: JSON.stringify(item),
  };
};

export default successResponse;

src/responses/badResuestsResponse.ts

const badRequestResponse = (msg: string) => {
  return {
  statusCode: 400,
  headers: { "Content-Type": "text/plain" },
  body: msg,
  };
}
  

export default badRequestResponse

src/responses/internalErrorResponse.ts

const internalErrorResponse = (msg: string) => {
  console.error(msg);
  return {
    statusCode: 500,
    headers: { "Content-Type": "text/plain" },
    body: "internal error",
  };
  };

export default internalErrorResponse

And we've also got a Note type which will be the data that gets stored/retreived:

src/notes/Note.ts

type Note = {
  userId: string;
  noteId: string;
  content?: string;
  createdAt: number;
};

export default Note

Access DB

Once we've got a DB table defined as above, we can then access the table to execute different queries

We would create a DB object instance using:

const db = new DynamoDB.DocumentClient();

Create

A create is the simplest one of the database functions for us to implement, this uses the db.put function with the Item to save which is of type Note:

const create = async (tableName: string, item: Note) => {
  await db.put({ TableName: tableName, Item: item }).promise();
};

Get

We can implement a getOne function by using db.get and providing the full Key consisting of the userId and noteId

const getOne = async (tableName: string, noteId: string, userId: string) => {
  const result =  await db.get({
    TableName: tableName,
    Key: {
      userId: userId,
      noteId: noteId
    }
  }).promise();

  return result.Item
};

GetAll

We can implement a getByUserId function which will make use of db.query and use the ExpressionAttributeValues to populate the KeyConditionExpression as seen below:

const getByUserId = async (tableName: string, userId: string) => {
  const result =  await db.query({
    TableName: tableName,
    KeyConditionExpression: "userId = :userId",
    ExpressionAttributeValues: {
      ":userId": userId,
    },
  }).promise();

  return result.Items
};

Define Lambdas

Now that we know how to write data to Dynamo, we can implement the following files for the endpoints we defined above:

Create

src/notes/create.ts

import { APIGatewayProxyEventV2, APIGatewayProxyHandlerV2 } from "aws-lambda";
import { DynamoDB } from "aws-sdk";
import { v1 } from "uuid";
import internalErrorResponse from "../responses/internalErrorResponse";
import successResponse from "../responses/successResponse";
import badRequestResponse from "../responses/badRequestResponse"
import Note from "./Note";

const db = new DynamoDB.DocumentClient();

const toItem = (data: string, content: string): Note => {
  return {
    userId: data,
    noteId: v1(),
    content: content,
    createdAt: Date.now(),
  };
};

const parseBody = (event: APIGatewayProxyEventV2) => {
  const data = JSON.parse(event.body || "{}");

  return {
    userId: data.userId,
    content: data.content,
  };
};

const isValid = (data: Partial<Note>) =>
  typeof data.userId !== "undefined" && typeof data.content !== "undefined";

const create = async (tableName: string, item: Note) => {
  await db.put({ TableName: tableName, Item: item }).promise();
};

export const handler: APIGatewayProxyHandlerV2 = async (
  event: APIGatewayProxyEventV2
) => {
  if (typeof process.env.tableName === "undefined")
    return internalErrorResponse("tableName is undefined");

  const tableName = process.env.tableName;
  const data = parseBody(event);

  if (!isValid(data)) return badRequestResponse("userId and content are required");

  const item = toItem(data.userId, data.content);
  await create(tableName, item);

  return successResponse(item)
};

Get

src/notes/get.ts

import { APIGatewayProxyEventV2, APIGatewayProxyHandlerV2 } from "aws-lambda";
import { DynamoDB } from "aws-sdk";
import badRequestResponse from "../responses/badRequestResponse";
import internalErrorResponse from "../responses/internalErrorResponse";
import successResponse from "../responses/successResponse";

type RequestParams = {
  noteId?: string;
  userId?: string
};

const db = new DynamoDB.DocumentClient();

const parseBody = (event: APIGatewayProxyEventV2): RequestParams => {
  const pathData = event.pathParameters;
  const queryData = event.queryStringParameters;

  return {
    noteId: pathData?.noteId,
    userId: queryData?.userId
  };
};

const isValid = (data: RequestParams) => typeof data.noteId !== "undefined" && typeof data.userId !== 'undefined'

const getOne = async (tableName: string, noteId: string, userId: string) => {
  const result =  await db.get({
    TableName: tableName,
    Key: {
      userId: userId,
      noteId: noteId
    }
  }).promise();

  return result.Item
};

export const handler: APIGatewayProxyHandlerV2 = async (
  event: APIGatewayProxyEventV2
) => {
  const data = parseBody(event);

  if (typeof process.env.tableName === "undefined")
    return internalErrorResponse("tableName is undefined");

  const tableName = process.env.tableName;

  if (!isValid(data)) return badRequestResponse("noteId is required in path, userId is required in query");

  const items = await getOne(tableName, data.noteId as string, data.userId as string);

  return successResponse(items)
};
import { APIGatewayProxyEventV2, APIGatewayProxyHandlerV2 } from "aws-lambda";
import { DynamoDB } from "aws-sdk";
import badRequestResponse from "../responses/badRequestResponse";
import internalErrorResponse from "../responses/internalErrorResponse";
import successResponse from "../responses/successResponse";

type RequestParams = {
  noteId?: string;
  userId?: string
};

const db = new DynamoDB.DocumentClient();

const parseBody = (event: APIGatewayProxyEventV2): RequestParams => {
  const pathData = event.pathParameters;
  const queryData = event.queryStringParameters;

  return {
    noteId: pathData?.noteId,
    userId: queryData?.userId
  };
};

const isValid = (data: RequestParams) => typeof data.noteId !== "undefined" && typeof data.userId !== 'undefined'

const getOne = async (tableName: string, noteId: string, userId: string) => {
  const result =  await db.get({
    TableName: tableName,
    Key: {
      userId: userId,
      noteId: noteId
    }
  }).promise();

  return result.Item
};

export const handler: APIGatewayProxyHandlerV2 = async (
  event: APIGatewayProxyEventV2
) => {
  const data = parseBody(event);

  if (typeof process.env.tableName === "undefined")
    return internalErrorResponse("tableName is undefined");

  const tableName = process.env.tableName;

  if (!isValid(data)) return badRequestResponse("noteId is required in path, userId is required in query");

  const items = await getOne(tableName, data.noteId as string, data.userId as string);

  return successResponse(items)
};

GetAll

src/notes/getAll.ts

import { APIGatewayProxyEventV2, APIGatewayProxyHandlerV2 } from "aws-lambda";
import { DynamoDB } from "aws-sdk";
import badRequestResponse from "../responses/badRequestResponse";
import internalErrorResponse from "../responses/internalErrorResponse";
import successResponse from "../responses/successResponse";

type PathParams = {
  userId?: string;
};

const db = new DynamoDB.DocumentClient();

const parseBody = (event: APIGatewayProxyEventV2): PathParams => {
  const data = event.queryStringParameters;

  return {
    userId: data?.userId,
  };
};

const isValid = (data: PathParams) => typeof data.userId !== "undefined";

const getByUserId = async (tableName: string, userId: string) => {
  const result =  await db.query({
    TableName: tableName,
    KeyConditionExpression: "userId = :userId",
    ExpressionAttributeValues: {
      ":userId": userId,
    },
  }).promise();

  return result.Items
};

export const handler: APIGatewayProxyHandlerV2 = async (
  event: APIGatewayProxyEventV2
) => {
  const data = parseBody(event);

  if (typeof process.env.tableName === "undefined")
    return internalErrorResponse("tableName is undefined");

  const tableName = process.env.tableName;

  if (!isValid(data)) return badRequestResponse("userId is required in query");

  const items = await getByUserId(tableName, data.userId as string);

  return successResponse(items)
};

Testing

Once we've got all the above completed, we can actually test our endpoints and create and read back data

create:

POST https://AWS_ENDPOINT_HERE/notes

{
  "userId": "USER_ID",
  "content": "Hello world"
}

Which responds with:

200

{
  "content": "Hello world",
  "createdAt": 1619177078298,
  "noteId": "NOTE_ID_UUID",
  "userId": "USER_ID"
}

get:

GET https://AWS_ENDPOINT_HERE/notes/NOTE_ID_UUID?userId=USER_ID

200

{
  "content": "Hello world",
  "createdAt": 1619177078298,
  "noteId": "NOTE_ID_UUID",
  "userId": "USER_ID"
}

getAll

GET htttps://AWS_ENDPOINT_HERE/notes?userId=USER_ID

200

[
  {
    "content": "Hello world",
    "createdAt": 1619177078298,
    "noteId": "NOTE_ID_UUID",
    "userId": "USER_ID"
  }
]

Creating Notes Using a Queue

When working with microservices a common pattern is to use a message queue for any operations that can happen in an asynchronous fashion, we can create an SQS queue which we can use to stage messages and then separately save them at a rate that we're able to process them

In order to make this kind of logic we're going to break up our create data flow - a the moment it's this:

lambda -> dynamo
return <-

We're going to turn it into this:

lambda1 -> sqs
 return <-

          sqs -> lambda2 -> dynamo

This kind of pattern becomes especially useful if we're doing a lot more stuff with the data other than just the single DB operation and also allows us to retry things like saving to the DB if we have errors, etc.

A more complex data flow could look something like this (not what we're implementing):

lambda1 -> sqs
 return <-

           sqs -> lambda2 -> dynamo // save to db
               -> lambda3 -> s3     // generate a report
           sqs <-

           sqs -> lambda4           // send an email

Create Queue

SST provides us with the sst.Queue class that we can use for this purpose

To create a Queue you can use the following in stack:

const queue = new sst.Queue(this, "NotesQueue", {
  consumer: "src/consumers/createNote.handler",
});

queue.attachPermissions([table]);
queue.consumerFunction?.addEnvironment(
  "tableName",
  table.dynamodbTable.tableName
);

The above code does the following:

1. Create a queue
2. Give the queue permission to access the table
3. Add the tableName environment variable to the queue's consumerFunction

We will also need to grant permissions to the API to access the queue so that our create handler is able to add messages to the queue

api.attachPermissions([table, queue]);

Which means our Stack now looks like this:

lib/MyStack.ts

import * as sst from "@serverless-stack/resources";

export default class MyStack extends sst.Stack {
  constructor(scope: sst.App, id: string, props?: sst.StackProps) {
    super(scope, id, props);

    const table = new sst.Table(this, "Notes", {
      fields: {
        userId: sst.TableFieldType.STRING,
        noteId: sst.TableFieldType.STRING,
      },
      primaryIndex: {
        partitionKey: "userId",
        sortKey: "noteId",
      },
    });

    const queue = new sst.Queue(this, "NotesQueue", {
      consumer: "src/consumers/createNote.handler",
    });

    queue.attachPermissions([table]);
    queue.consumerFunction?.addEnvironment(
      "tableName",
      table.dynamodbTable.tableName
    );

    // Create the HTTP API
    const api = new sst.Api(this, "Api", {
      defaultFunctionProps: {
        timeout: 60, // increase timeout so we can debug
        environment: {
          tableName: table.dynamodbTable.tableName,
          queueUrl: queue.sqsQueue.queueUrl,
        },
      },
      routes: {
        "GET  /": "src/lambda.handler",
        "GET  /hello": "src/hello.handler",
        "GET  /notes": "src/notes/getAll.handler",
        "POST /notes": "src/notes/create.handler",
        "GET  /notes/{noteId}": "src/notes/get.handler",
      },
    });

    api.attachPermissions([table, queue]);

    // Show API endpoint in output
    this.addOutputs({
      ApiEndpoint: api.httpApi.apiEndpoint,
    });
  }
}

Update the Create Handler

Since we plan to create notes via a queue we will update our create function in the handler to create a new message in the queue, this is done using the SQS class from aws-sdk:

src/notes/create.ts

import { SQS } from "aws-sdk";

const queue = new SQS();

Once we've got our instance, the create function is done by means of the queue.sendMessage function:

src/notes/create.ts

const create = async (queueUrl: string, item: Note) => {
  return await queue
    .sendMessage({
      QueueUrl: queueUrl,
      DelaySeconds: 0,
      MessageBody: JSON.stringify(item),
    })
    .promise();
};

Lastly, our handler remains mostly the same with the exception of some additional validation to check that we have the queue connection information in the environment:

src/notes/create.ts

export const handler: APIGatewayProxyHandlerV2 = async (
  event: APIGatewayProxyEventV2
) => {
  // pre-save validation
  if (typeof process.env.queueUrl === "undefined")
    return internalErrorResponse("queueUrl is undefined");

  const queueUrl = process.env.queueUrl;

  const data = parseBody(event);

  if (!isValid(data))
    return badRequestResponse("userId and content are required");

  // save process
  const item = toItem(data.userId, data.content);
  const creatresult = await create(queueUrl, item);

  if (!creatresult.MessageId) internalErrorResponse("MessageId is undefined");

  return successResponse(item);
};

Implementing the above into the create handler means that our create.ts file now looks like this:

src/notes/create.ts

import { APIGatewayProxyEventV2, APIGatewayProxyHandlerV2 } from "aws-lambda";
import { v1 } from "uuid";
import internalErrorResponse from "../responses/internalErrorResponse";
import successResponse from "../responses/successResponse";
import badRequestResponse from "../responses/badRequestResponse";
import Note from "./Note";
import { SQS } from "aws-sdk";

const queue = new SQS();

// helper functions start

const toItem = (data: string, content: string): Note => {
  return {
    userId: data,
    noteId: v1(),
    content: content,
    createdAt: Date.now(),
  };
};

const parseBody = (event: APIGatewayProxyEventV2) => {
  const data = JSON.parse(event.body || "{}");

  return {
    userId: data.userId,
    content: data.content,
  };
};

const isValid = (data: Partial<Note>) =>
  typeof data.userId !== "undefined" && typeof data.content !== "undefined";

// helper functions end

const create = async (queueUrl: string, item: Note) => {
  return await queue
    .sendMessage({
      QueueUrl: queueUrl,
      DelaySeconds: 0,
      MessageBody: JSON.stringify(item),
    })
    .promise();
};

export const handler: APIGatewayProxyHandlerV2 = async (
  event: APIGatewayProxyEventV2
) => {
  // pre-save validation
  if (typeof process.env.queueUrl === "undefined")
    return internalErrorResponse("queueUrl is undefined");

  const queueUrl = process.env.queueUrl;

  const data = parseBody(event);

  if (!isValid(data))
    return badRequestResponse("userId and content are required");

  // save process
  const item = toItem(data.userId, data.content);
  const creatresult = await create(queueUrl, item);

  if (!creatresult.MessageId) internalErrorResponse("MessageId is undefined");

  return successResponse(item);
};

Add Queue-Based Create Handler

Now that we've updated our logic to save the notes into the queue, we need to add the logic for the src/consumers/createNote.handler consumer function as we specified above, this handler will be sent an SQSEvent and will make use of the DynamoDB Table we gave it permissions to use

First, we take the create function that was previously on the create.ts file for saving to the DB:

src/consumers/createNote.ts

import { DynamoDB } from "aws-sdk";

const db = new DynamoDB.DocumentClient();

const create = async (tableName: string, item: Note) => {
  const createResult = await db
    .put({ TableName: tableName, Item: item })
    .promise();
  if (!createResult) throw new Error("create failed");

  return createResult;
};

We'll also need a function for parsing the SQSRecord object into a Note:

src/consumers/createNote.ts

const parseBody = (record: SQSRecord): Note => {
  const { noteId, userId, content, createdAt } = JSON.parse(
    record.body
  ) as Note;

  // do this to ensure we only extract information we need
  return {
    noteId,
    userId,
    content,
    createdAt,
  };
};

And finally we consume the above through the handler, you can see in the below code that we are iterating over the event.Records object, this is because the SQSEvent adds each new event into this array, the reason for this is because we can also specify batching into our Queue so that the handler is only triggered after n events instead of each time, and though this isn't happening in our case, we still should handle this for our handler:

src/consumers/createNote.ts

export const handler: SQSHandler = async (event) => {
  // pre-save environment check
  if (typeof process.env.tableName === "undefined")
    throw new Error("tableName is undefined");

  const tableName = process.env.tableName;

  for (let i = 0; i < event.Records.length; i++) {
    const r = event.Records[i];
    const item = parseBody(r);
    console.log(item);

    const result = await create(tableName, item);
    console.log(result);
  }
};

Putting all the above together our createNote.ts file now has the following code:

import { SQSHandler, SQSRecord } from "aws-lambda";
import Note from "../notes/Note";
import { DynamoDB } from "aws-sdk";

const db = new DynamoDB.DocumentClient();

const create = async (tableName: string, item: Note) => {
  const createResult = await db
    .put({ TableName: tableName, Item: item })
    .promise();
  if (!createResult) throw new Error("create failed");

  return createResult;
};

const parseBody = (record: SQSRecord): Note => {
  const { noteId, userId, content, createdAt } = JSON.parse(
    record.body
  ) as Note;

  // do this to ensure we only extract information we need
  return {
    noteId,
    userId,
    content,
    createdAt,
  };
};

export const handler: SQSHandler = async (event) => {
  if (typeof process.env.tableName === "undefined")
    throw new Error("tableName is undefined");

  const tableName = process.env.tableName;

  for (let i = 0; i < event.Records.length; i++) {
    const r = event.Records[i];
    const item = parseBody(r);
    console.log(item);

    const result = await create(tableName, item);
    console.log(result);
  }
};

This completes the implementation of the asynchronous saving mechanism for notes. As far as a consumer of our API is concerned, nothing has changed and they will still be able to use the API exactly as we had in the Testing section above

Deploy

Thus far, we've just been running our API in debug mode via the npm run start command, while useful for testing this adds a lot of code to make debugging possible, and isn't something we'd want in our final deployed code

Deploying using sst is still very easy, all we need to do is run the npm run deploy command and this will update our lambda to use a production build of the code instead:

npm run deploy

Teardown

Lastly, the sst CLI also provides us with a function to teardown our start/deploy code. So once you're done playing around you can use this to teardown all your deployed services:

npm run remove

Note that running the remove command will not delete the DB tables, you will need to do this manually

When creating a multi-module project for Python, especially when coming from another language ecosystem, you may encounter issues importing code from other files, let's take a look at this

The Modules

In our project, we've got the following files:

package1
  |- module1.py
package2
  |- namespace2
       |- module2.py
main.py

Let's say each of our modules export a function, here's the code for them:

package1/module1.py

def hello_p1_m1():
  return "hello"

package2/subpackage1/module1.py

def hello_p2_s1_m1():
  return "hello"

The Main

Now that we've got our files with their functions separated, we try using them from the main.py file:

main.py

import package1.module1 as p1m1
import package2.namespace1.module1 as p2s1m1

print(p1m1.hello_p1_m1())
print(p2n1m1.hello_p2_s1_m1())

The Error

Now, we try to run this using python:

python main.py

And we get the following error:

Traceback (most recent call last):
  File "main.py", line 1, in <module>
    import package1.module1 as p1m1
ImportError: No module named package1.module1

The Solution

Now, this can be pretty annoying, but the solution is very simple. Python identifies a module as a single file, and the assumption that leads to the issue above is that we assume a package has the same convention of just being a directory

However, a packgage requires an __init__.py file to be defined so that they are recognized correctly

So, we need to add two files:

• package1/__init__.py
• package2/subpackage1/__init__.py

Additionally, these files are completely empty, and only serve as information. Note that we don't need to include a file in package2 directly as this directory does not contain any modules within it so is not really a package in itself and is simply a wrapper subpackage1

This post will take a look at the process of setting up a new F# library and two methods of configuring XUnit to test your project's code

Create a Project

Before we can start testing we need a project that we can run tests on

First, we're going to create a folder that we can work in:

mkdir MyProject
cd MyProject

We can use the following command to create a new project in our MyProject directory:

dotnet new classlib -lang=f# -o MyProject.Lib

The project that we created will contain the following Library.ts file, this is the file that we'll write tests for. First, we want to update the hello function so that it returns a formatted string:

MyProject.Lib/Library.fs

namespace MyProject.Lib

module Say =
    let hello name =
        sprintf "Hello %s" name

Adding Tests

Depending on our preferred project structure we can either:

1. Add tests in a separate project
2. Add test files alongside lib files

Method 1: Create Tests in Separate Project

The standard method of .NET unit testing with XUnit is to make use of separate Project and Test solutions, so a normal test setup would look something like:

MyProject.Lib
MyProject.Lib.Tests

To add an new XUnit test project you can run:

dotnet new xunit -lang=f# -o MyProject.Tests

Then, so we're able to test the code from MyProject.Lib, we need to add a reference to it from the Test project we just created:

dotnet add MyProject.Tests reference MyProject.Lib

Then we can create a file in our test project called LibraryTests.fs which will contain our test code which we will cover in the last section, as well as adding a reference to this file in the MyProject.Tests.fsproj

MyProject.Tests.fsproj

...
  <ItemGroup>
    <Compile Include="Tests.fs" />
    <!-- Add the next line -->
    <Compile Include="LibraryTests.fs" /> 
    <Compile Include="Program.fs" />
  </ItemGroup>
...

Method 2: Create Tests Alongside Lib Files

The second method I'm going to discuss is keeping our test.fs files alongside the code that the file is testing. The structure of our project is something more like this:

MyProject.Lib
|- Library.fs
|- Library.test.fs

Overall I find this more manageable is the way I keep my test code in other languages and frameworks as well

To implement this method we need to add some dependencies to our project

cd MyProject.Lib
dotnet add package Microsoft.NET.Test.Sdk
dotnet add package xunit
dotnet add package xunit.runner.visualstudio

Then we can create a file in our project called Library.test.fs which will contain our test code which we will cover next, as well as a reference to this file in the MyProject.Lib.fsproj

MyProject.Lib.fsproj

...
  <ItemGroup>
    <Compile Include="Library.fs" />
    <!-- Add the next line -->
    <Compile Include="Library.test.fs" />
  </ItemGroup>
...

It's important to note that this file must be added below the Library.fs file as it will reference it for tests to run

Test Files

More detailed information on XUnit can be found in Unit Testing notes

Since we've configured XUnit it may be useful to understand how these tests work in the context of F#. XUnit tests are organized into modules. Regardless of which of the two methods above you're using the test files work the same

Generally, a test file will contain:

1. A top-level module definition
2. open statements to import XUnit
3. Test functions annotated with Fact or Theory

XUnit tests can be broken into 2 types:

1. Single-case tests without input parameters inputs are labelled Fact
2. Multi-case tests which make use of input parameters are labelled Theory and use InlineData

Fact

Let's add the following content into our test file into one that tests the hello function from our Lib code with the input "Name"

LibraryTests.fs/Library.test.fs

module LibraryTests

open Xunit
open MyProject.Lib.Say

[<Fact>]
let ``Say.hello -> "Hello name" `` () =
    let name = "name"
    let expected = "Hello name"
    
    let result = hello name

    Assert.Equal(expected, result)

F# allows us to name our functions using special characters provided they're enclosed in backticks as seen above. Naming test functions this way allows them to be more discriptive than more traditional variable names

Additionally, there's the normal XUnit test setup which includes calling our test function with some input and asserting something about it using Assert.Equal from XUnit

Theory

We can add a Theory to test our function with multiple different inputs:

LibraryTests.fs/Library.test.fs

[<Theory>]
[<InlineData("name", "Hello name")>]
[<InlineData("World", "Hello World")>]
let ``Say.hello -> concantenated string`` (name:string, expected: string) =
    let result = hello name

    Assert.Equal(expected, result)

In the above we add the InlineData attribute which allows us to provide inputs to our test, as well as specifying a name and expected argument for our function. The test framework will then call our test using the arguments as specified in InlineData

When we're done our test file should have the following content:

module LibraryTests

open Xunit
open MyProject.Lib.Say

[<Fact>]
let ``Say.hello -> "Hello name" `` () =
    let name = "name"
    let expected = "Hello name"
    
    let result = hello name

    Assert.Equal(expected, result)

[<Theory>]
[<InlineData("name", "Hello name")>]
[<InlineData("World", "Hello World")>]
let ``Say.hello -> concantenated string`` (name:string, expected: string) =
    let result = hello name

    Assert.Equal(expected, result)

Running Tests

In order to run tests we can use the dotnet-cli. Depending on the method used you can run your test from the project's root directory using the following command:

• Method 1 - dotnet test MyProject.Tests
• Method 2 - dotnet test MyProject.Lib

Alternatively, tests can also be run from your IDE or Visual Studio Code with the Ionide and .NET Core Test Explorer extensions installed

Additional Resources

If you'd like a deeper look into F# or XUnit here are some of my other posts which cover those:

• Introduction to F# Web APIs
• Introduction to F#
• Entity Framework with F#
• Introduction to Unit Testing
• Testing Private Members

Nabeel Valley

Circuit Diagram

This script will make use of the GPIO Pins to flicker an LED

The circuit diagram is below:

I've used a $330\Omega$ resistor for the resistor connected in series $R_LED$, however the resistance for a given LED can be calculated with this equation (From Circuit Specialists):

$R_{LED} = \frac{V_{source} - V_{LED}}{I_{LED}}$

It's important to connect the LED in the correct direction on the circuit

Code

Below is a simple python script which will handle turning the GPIO pins on and off using the RPi.GPIO library

import RPi.GPIO as GPIO
import time

pin = 21
dur = 1

GPIO.setmode(GPIO.BCM)
GPIO.setwarnings(False)
GPIO.setup(pin, GPIO.OUT)

while True:
  GPIO.output(pin, GPIO.HIGH)
  print "LED ON"
  time.sleep(dur)

  GPIO.output(pin, GPIO.LOW)
  print "LED OFF"
  time.sleep(dur)

When working with markdown it can often be useful to be able to style elements using custom CSS

We can accomplish this by including a style tag. An example to illustrate this is changing the way a table renders in a specific document by changing the style of table rows to be striped, you would include the following CSS:

<style>
tr:nth-child(even) {
  background-color: #b2b2b2;
  color: #f4f4f4;
}
</style>

If the platform you're displaying the HTML in already has the styles included then you may need to add !important to override in the CSS:

<style>
tr:nth-child(even) {
  background-color: #b2b2b2!important;
  color: #f4f4f4!important;
}
</style>

my-sample-doc.md

# Custom CSS in Markdown Example

This is a document where tables are rendered with every second table row with a specific background and foreground colour 

<style>
tr:nth-child(even) {
  background-color: #b2b2b2!important;
  color: #f4f4f4!important;
}
</style>

The above CSS will lead to the following table being rendered with alternating row colours:

| Col 1 | Col 2 | Col 3 |
| ----- | ----- | ----- |
| A     | B     | C     |
| 1     | 2     | 3     |
| A1    | B2    | C3    |

Note that depending on your markdown renderer the embeded style tags may be removed, you'll have to look at your specific renderer/converter/platform to see whether this works for you

With the converter I'm using, showdown.js, this is how the table above looks (with !important included to override my current table styles):

When using React it can sometimes be useful to render a standard HTML element given the element name dynamically as a prop

React allows us to do this provided we store the element name in a variable that starts with a capital letter, as JSX requires this to render a custom element

We can do something like this by defining a generic element in React which just takes in the name of the tag in addition to it's usual props and children, something like this:

const GenericElement = ({ tagName:Tag, children, ...innerProps}) => 
  <Tag {...innerProps}> 
    {children}
  </Tag>

We can then render this element by calling the GenericElement with the tagName prop and then any children we'd like:

<GenericElement tagName="h1">I am an h1</GenericElement>

Here's a short example using Replit for reference, all the important stuff is in the src/App.jsx file:

A short lil guide to setting up Home Assistant on a RaspberryPi using WiFi instead of Ethernet

1. Download Balena Etcher and install from here
2. Go to the HomeAssistant website and copy the relevant release URL from here
3. Connect your RaspberryPi's SD card to your computer
4. Open Balena Etcher and select Flash from URL and select the RaspberryPi's SD to flash to
5. Once flashing is complete, open the RaspberryPi's SD in File Explorer, and create the following file:

RASPBERRYPIDRIVE/CONFIG/network/my-network

!cat .\my-network
[connection]
id=my-network
uuid=72111c67-4a5d-4d5c-925e-f8ee26efb3c3
type=802-11-wireless
[802-11-wireless]
ssid=YOUR_WIFI_NETWORK_NAME
#hidden=true

[802-11-wireless-security]
auth-alg=open
key-mgmt=wpa-psk
psk=YOUR_WIFI_PASSWORD

[ipv4]
method=auto

[ipv6]
addr-gen-mode=stable-privacy
method=auto

Be sure to use LF line endings, additional information on this setup can be found on GitHub

When working with the FSharp.Data.JsonProvider type provider you may encounter a need to serialize a JsonValue array

Let's take a look at an example where this may be necessary:

First, we'll define the type of our data using a type provider, in this case it's connected to the following:

data/api-response.json

{
  "data": [
      {
          "id": "id",
          "caption": "caption",
          "media_type": "IMAGE",
          "media_url": "url_to_image"
      }
  ],
  "paging": {
      "cursors": {
          "before": "hash",
          "after": "hash"
      },
      "next": "full_request_url"
  }
}

And the F# file that's using this as the basis for the type definition is as follows:

open FSharp.Data

type ApiResponse = JsonProvider<"./data/api-response.json", SampleIsList=true>

We can also create some sample data using the ApiResponse.Parse method that's now defined on our type thanks to the JsonProvider

// get some sample data
let sampleData = ApiResponse.Parse("""{
  "data": [
      {
          "id": "id",
          "caption": "caption",
          "media_type": "IMAGE",
          "media_url": "url_to_image"
      }
  ],
  "paging": {
      "cursors": {
          "before": "hash",
          "after": "hash"
      },
      "next": "full_request_url"
  }
}""")

If you were to use the sampleData object and try to parse it to JSON you could directly call the sampleData.JsonValue.ToString() method. As designed, this immediately returns the JSON representation of the object

However, when we take a look at the sampleData.Data property, we will notice this is a Datum array, the problem here is that the array type doesn't have JsonValue property

However, if we look at how JsonValue is defined we will see the following:

union JsonValue =
  | String of string
  | Number of decimal
  | Float of float
  | Record of properties : (string * JsonValue) array
  | Array of elements : JsonValue array
  | Boolean of bool
  | Null

Based on this, we can see that an Array of elements can be seen as a JsonValue array, using this information, we can make use of the JsonValue.Array constructor and the JsonValue property of each of the elements of the Data property to convert the JsonValue array to a JsonValue

let jsonValue =
    // get the property we want
    sampleData.Data 
    // extract the JsonValue property from each element
    |> Array.map (fun p -> p.JsonValue)
    // pass into the JsonValue.Array constructor
    |> JsonValue.Array

let json = jsonValue.ToString()

Taken from this StackOverflow answer

Another git process seems to be running in this repository, e.g.
an edirot opened by 'git commit'. Please make sure all processes
are terminated then try again. If it still fails, a git process
may have crashed in this repository earlier:
remove the file manually to continue

Sometimes when using Jenkins a Git process may crash or timeout resulting in an error message like the above one. This can result in issues when running later Git steps or running other Git steps in the repository in the Jenkins workspace

The simplest way I've found to fix this is to simply delete the .git/index.lock or .git/shallow.lock file from the workspace that's giving the issue and run again

In this post I'm going to cover the following:

Attribute Types and Execution Order

There are a few different attribute types that we can handle on a WebAPI that provide us with the ability to wrap some functionality around our endpoints, below are some of the common attributes that we can implement and the order in which they execute (StackOverflow)

1. Authorization - IAuthorizationFilter
2. Action - IActionFilter
3. Result - IResultFilter
4. Exception - IExceptionFilter

IActionFilter

The IActionFilter executes before and after a method is executed and contains two different methods for doing this, namely the OnActionExecuting and OnActionExecuted methods respectively. A basic implemtation of IActionFilter would look like this:

namespace CSharpAttributes.Attributes
{
  public class LogStatusAttribute : Attribute, IActionFilter
  {
    public LogStatusAttribute()
    {
      Console.WriteLine("Attribute Initialized");
    }

    public void OnActionExecuting(ActionExecutingContext context)
    {
      Console.WriteLine("OnActionExecuting");
    }

    public void OnActionExecuted(ActionExecutedContext context)
    {
      Console.WriteLine("OnActionExecuted");
    }
  }
}

This can then be implemented on a controller method with a [LogStatus] attribute:

[LogStatus]
[HttpGet]
public IEnumerable<WeatherForecast> Get()
{
  Console.WriteLine("Executing Get");
  return data;
}

The order of logging which we see will be as follows:

1. Attribute Initialized when the controller is instantiated
2. OnActionExecuting when the controller is called
3. Executing Get when the controller is executed
4. OnActionExecuted when the controller is done executing

IAuthorizationFilter

The IAuthorizationFilter executes as the first filter on a controller's method call

namespace CSharpAttributes.Attributes
{
  public class CustomAuthorizeAttribute : Attribute, IAuthorizationFilter
  {
    public CustomAuthorizeAttribute()
    {
      Console.WriteLine("Attribute Initialized");
    }

    public void OnAuthorization(AuthorizationFilterContext context)
    {
      Console.WriteLine("OnAuthorization");
    }
  }
}

This can then be implemented on a controller method with a [CustomAuthorize] attribute:

[CustomAuthorize]
[HttpGet]
public IEnumerable<WeatherForecast> Get()
{
  Console.WriteLine("Executing Get");
  return data;
}

The order of logging which we see will be as follows:

1. Attribute Initialized when the controller is instantiated
2. OnAuthorization when the controller is called
3. Executing Get when the controller is executed

Modify Response Data

An attribute's context parameter gives us ways by which we can access the HttpContext as well as set the result of a method call so that it can be handled down the line. For example, we can implement our CustomAuthorize attribute with the following:

public void OnAuthorization(AuthorizationFilterContext context)
{
  if (!context.HttpContext.Request.Headers.ContainsKey("X-Custom-Auth"))
  {
    context.Result = new UnauthorizedResult();
  }

  Console.WriteLine("Attribute Called");
}

This will mean that if we set the context.Result in our method then the controller will not be executed and the endpoint will return the UnauthorizedResult early. You can also see that we're able to access things like the HttpContext which makes it easy for us to view the request/response data and do things based on that

Attribute on a Class

Note that it's also possible to apply the above to each method in a class by adding the attribute at the top of the class declaration:

[ApiController]
[LogStatus]
[Route("[controller]")]
public class WeatherForecastController : ControllerBase
{
  ...

Attributes with Input Parameters

We are also able to create attributes that enable the consumer to modify their behaviour by taking input parameters to the constructor, we can update our LogStatus attribute to do something like add a prefix before all logs:

namespace CSharpAttributes.Attributes
{
  public class LogStatusAttribute : Attribute, IActionFilter
  {
    private readonly string _prefix;

    public LogStatusAttribute(string prefix = "")
    {
      _prefix = prefix;
      Console.WriteLine("Attribute Initialized");
    }

    public void OnActionExecuted(ActionExecutedContext context)
    {
      Console.WriteLine(_prefix + "OnActionExecuted");
    }

    public void OnActionExecuting(ActionExecutingContext context)
    {
      Console.WriteLine(_prefix + "OnActionExecuting");
    }
  }
}

Then, applying to our controller method like so:

[LogStatus("WeatherController-Get:")]
[HttpGet]
public IEnumerable<WeatherForecast> Get()
{
  Console.WriteLine("Executing Get");
  return data;
}

So the new output will look like so:

1. Attribute Initialized when the controller is instantiated
2. WeatherForecast-Get:OnActionExecuting when the controller is called
3. Executing Get when the controller is executed
4. WeatherForecast-Get:OnActionExecuted when the controller is done executing

Attribute Setting at Class and Method Level

Since an attribute can be implemented at a class and method level it's useful for us to be able to implement it at a class and the override the behaviour or add behaviour for a specific method

We can do this by setting the attribute inheritence to false

Updating out LogStatusAttribute we can add the AttributeUsage Attribute as follows:

namespace CSharpAttributes.Attributes
{
  [AttributeUsage(AttributeTargets.All, Inherited = false)]
  public class LogStatusAttribute : Attribute, IActionFilter
  {
    ...

This means that we can independently apply the attribute at class and method levels, so now our controller can look something like this:

namespace CSharpAttributes.Controllers
{
  [ApiController]
  [Route("[controller]")]
  [LogStatus("WeatherForecast:")]
  public class WeatherForecastController : ControllerBase
  {
    [LogStatus("Get:")]
    [HttpGet]
    public IEnumerable<WeatherForecast> Get()
    {
      Console.WriteLine("Executing Get");
      return data;
    }
  }
}

Which will output the logs as follows:

1. Attribute Initialized when the controller is instantiated
2. WeatherForecast:OnActionExecuting when the class is called
3. Get:OnActionExecuting when the controller is called
4. Executing Get when the controller is executed
5. Get:OnActionExecuted when the controller is done executing
6. WeatherForecast:OnActionExecuted when the class is done executing

Sometimes it's useful to have an endpoint that you can use to debug data that's being POSTed to an application

You can make use of the following express.js app to log your application's POST requests:

const express = require("express")
const app = express()

// parse json
app.use(express.json())  

// GET endpoint to check uptime
app.get('/', (req, res) => {
  res.json({ data: 'hello' })
})

// POST endpointthat logs request body
app.post('/', (req, res) => {
  console.log(req.body)
  res.json(req.body)
})

// listen for requests
const listener = app.listen(process.env.PORT, () => {
  console.log("listening on port " + listener.address().port)
})

const name = "Nabeel"

console.log(`Hello ${name}`)

const name = "Nabeel"

pm.environment.set('name', name)

console.log(pm.environment.get('name'))

pm.sendRequest('https://randomuser.me/api/', (err, res) => {
  const apiResponse = res.json()

  // do more stuff
})

pm.sendRequest('https://randomuser.me/api/', (err, res) => {
  const apiResponse = res.json()

  const user = apiResponse.results[0]
  
  const email = user.email
  const firstName = user.name.first
  const lastName = user.name.last

  pm.environment.set('email', email)
  pm.environment.set('firstName', firstName)
  pm.environment.set('lastName', lastName)
})

{
  "email": "{{email}}",
  "firstName": "{{firstName}}",
  "lastName": "{{lastName}}"
}

{
  "experimentalNetworkStubbing": true
}

cy.route2('POST', '/do-stuff').as('data')

cy.wait('@data').then((data) => {
  // do stuff using the data object
  console.log(data)
})

cy.wait('@data').then((data) => {
  console.log(data)

  // parse the response body
  const res = JSON.parse(data.response.body)

  // assertions on the response
  assert.isTrue(res.success)
  assert.isNotNull(res.message)
})

dotnet fsi

let printer s =
    printfn s
;;

printer "Hello World";;

dotnet fsi ./myscript.fsx

(Invoke-WebRequest -Uri https://raw.githubusercontent.com/python-poetry/poetry/master/get-poetry.py -UseBasicParsing).Content | python -

curl -sSL https://raw.githubusercontent.com/python-poetry/poetry/master/get-poetry.py | python -

poetry new my-project

poetry init

poetry add flask

poetry run python app.py

poetry run flask run

poetry shell

dotnet dev-certs https --clean
dotnet dev-certs https --trust

document
  .querySelectorAll(selector)
  .forEach((el) => (el.style.border = 'solid 2px red'))

document
  .querySelectorAll('[id]')
  .forEach((el) => (el.style.border = 'solid 2px red'))

https://my.website.com/#:~:text=When%20linking%20to%20a%20webpage

https://my.website.com/#:~:text=when%20linking,HTML%20of%20a%20page

const processDir = process.cwd()

const fileDir = __dirname

const { join } = require('path')

const path1 = join(basePath, './downdir/myfile.txt')
const path2 = join(basePath, '../updir')

const { resolve } = require('path')

const absPath = resolve('./downdir/myfile.txt')

const absPath = join(process.cwd(), './downdir/myfile.txt')

const { join } = require('path')

const absPath = join(__dirname, './downdir/myfile.txt')

const input = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]

const output = [
  [1, 4, 7, 10],
  [2, 5, 8],
  [3, 6, 9],
]

const transform = (arr, segments) =>
  [...new Array(segments)].map((_, outerPos) =>
    input.filter((el, innerPos) => innerPos % segments === outerPos)
  )

const rows = 3
const transformedData = transform(input, rows)

docker run -d -p 27017:27017

docker run -d -p 27017:27017

version: '3.3'

services:
  # mongonode0:27017
  mongo0:
    image: mongo
    hostname: mongo0
    container_name: mongo0
    restart: always
    ports:
      - '37000:27017'
    volumes:
      - ./mongo0-data:/data/db
    environment:
      MONGO_INITDB_ROOT_USERNAME: root
      MONGO_INITDB_ROOT_PASSWORD: password

npm init adonis-ts-app my-api

cd my-api

npm start

PORT=3333
HOST=0.0.0.0
NODE_ENV=development
APP_KEY=mE8zN8V_7PzazKfv9_ds-8CGjVRLA2wo

ℹ  info      cleaning up build directory build
ℹ  info      copy .env,ace build
☐  pending   compiling typescript source files
✔  success   built successfully
ℹ  info      copy .adonisrc.json build
…  watch     watching file system for changes
ℹ  info      starting http server
✔  create    ace-manifest.json
[1595755199986] INFO  (my-app/1531 on 47f057d8722c): started server on 0.0.0.0:3333

{
  "hello": "world"
}

import Route from '@ioc:Adonis/Core/Route'

Route.get('/', async () => {
  return { hello: 'world' }
})

node ace make:controller User

export default class UsersController {}

export default class UsersController {
  public async get() {
    return [
      {
        id: 1,
        name: 'Bob Smith',
        email: 'bob@smithmail.com',
      },
    ]
  }
}

Route.get('users', 'UsersController.get')