API with NestJS #110. Managing JSON data with PostgreSQL and Prisma
source link: https://wanago.io/2023/05/29/api-nestjs-prisma-json/
Go to the source link to view the article. You can view the picture content, updated content and better typesetting reading experience. If the link is broken, please click the button below to view the snapshot at that time.
API with NestJS #110. Managing JSON data with PostgreSQL and Prisma
Relational databases such as PostgreSQL are great for storing structured data. This approach has many advantages, but it can lack flexibility. On the other hand, databases such as MongoDB that store JSON-like documents might give you more flexibility than you would like. Fortunately, PostgreSQL offers support for storing and querying loosely-structured JSON data. This article explores the features and benefits of using the JSON and JSONB columns.
Using the json column type
The first column type to look into is JSON. It stores the provided data as is in a text format.
CREATE TABLE products ( id int GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY, name text NOT NULL, properties json |
When we insert data, PostgreSQL ensures that it is a valid JSON. If not, it throws an error.
INSERT INTO products ( name, properties VALUES ( 'Introduction to Algorithms', '{ "publicationYear": 1990, "authors": ["Thomas H. Cormen", "Charles E. Leiserson", "Ronald L. Rivest", "Clifford Stein"] }' |
Above, we’ve added our first record to the products column. Since it is a book, it has the publication year and a list of authors. Thanks to using a JSON column, we don’t need to add the publication_year and authors columns to our table. Since the properties column is flexible, we can use it to store any product.
INSERT INTO products ( name, properties VALUES ( 'A8', '{ "brand": "Audi", "engine": { "numberOfCylinders": 6, "fuel": "petrol" } }' |
When working with PostgreSQL, we can make parts of our data flexible while putting constraints on other columns. For example, we could add the category_id column to group our products.
If we would only have two types of products, it would make sense to create separate books and cars tables. However, it would get complicated if we had tens of different product types.
Defining a json column with Prisma
To add the json column with Prisma, we need the @db.Json annotation.
productSchema.prisma
model Product { id Int @id @default(autoincrement()) name String properties Json? @db.Json |
Creating the above schema and generating a migration yields the following result:
migrations/20230523001250_products/migration.sql
-- CreateTable CREATE TABLE "Product" ( "id" SERIAL NOT NULL, "name" TEXT NOT NULL, "properties" JSON, CONSTRAINT "Product_pkey" PRIMARY KEY ("id") |
It’s crucial to acknowledge that when inserting values into a json column, Prisma expects us to provide the data matching the InputJsonValue interface. When we look under the hood of Prisma, we can see that it includes any valid JSON value.
export type InputJsonValue = string | number | boolean | InputJsonObject | InputJsonArray |
Since we want properties to be an object, we need to enforce the InputJsonObject interface built into Prisma.
createProduct.dto.ts
import { Prisma } from '@prisma/client'; import { IsString, IsNotEmpty, IsOptional } from 'class-validator'; import IsJsonObject from '../../utils/isJsonObject'; export class CreateProductDto { @IsString() @IsNotEmpty() name: string; @IsJsonObject() @IsOptional() properties?: Prisma.InputJsonObject; export default CreateProductDto; |
To achieve the above with the class-validator library, we need to create a custom decorator that checks if the provided value is a valid object.
isJsonObject.ts
import { registerDecorator, ValidationArguments } from 'class-validator'; function IsJsonObject() { return function (object: object, propertyName: string) { registerDecorator({ name: 'isJsonObject', target: object.constructor, propertyName: propertyName, validator: { validate(value: unknown) { return ( typeof value === 'object' && value !== null && !Array.isArray(value) defaultMessage(validationArguments?: ValidationArguments): string { return `${validationArguments.property} must be a valid object`; export default IsJsonObject; |
Thanks to all of the above, we can now insert JSON data through an API created with NestJS.
Handling null values
Our properties column is nullable and does not need to contain any value. Unfortunately, this can cause confusion because the JSON format can store a null value.
JSON.stringify(null); // 'null' |
To differentiate between null as a lack of value in the database and as a null value stored in the json column, Prisma introduced the Prisma.JsonNull and Prisma.DbNull constants.
Let’s assume that whenever the user sends null as the value for our properties column, they want to remove the data from the column altogether. To achieve that, we need Prisma.DbNull.
async updateProduct(id: number, product: UpdateProductDto) { return await this.prismaService.product.update({ data: { ...product, id: undefined, properties: product.properties ?? Prisma.DbNull, where: { } catch (error) { error instanceof PrismaClientKnownRequestError && error.code === PrismaError.RecordDoesNotExist throw new NotFoundException(); throw error; |
Manipulating the JSON data
PostgreSQL is equipped with quite a few different operators that help us query JSON data. One of the most important ones is ->>, which allows us to get a value of a particular property as text.
SELECT * FROM "Product" WHERE (properties->>'publicationYear')::int < 2000 |
We can achieve the same thing with Prisma thanks to the path option.
this.prismaService.product.findMany({ where: { properties: { path: ['publicationYear'], lt: 2000, |
We can access nested properties by providing the path array with multiple strings such as ['user', 'address', 'street']
Prisma offers a wide variety of filters besides the lt filter, which means “less than”.
export type JsonNullableFilterBase = { equals?: InputJsonValue | JsonNullValueFilter path?: string[] string_contains?: string string_starts_with?: string string_ends_with?: string array_contains?: InputJsonValue | null array_starts_with?: InputJsonValue | null array_ends_with?: InputJsonValue | null lt?: InputJsonValue lte?: InputJsonValue gt?: InputJsonValue gte?: InputJsonValue not?: InputJsonValue | JsonNullValueFilter |
The jsonb column
The json column type offers fast insertion because the data we store in the database is stored as text. But, unfortunately, PostgreSQL has to parse the value whenever we perform operations on it, and it causes a performance hit.
To deal with the above issue, PostgreSQL introduced the jsonb column type, which stands for “JSON Binary”. Whenever we store data in the jsonb column, PostgreSQL parses our JSON into a binary format. While this can cause the INSERT operations to be slightly slower, it significantly improves the querying performance. It also doesn’t preserve whitespaces and the order of properties.
Besides the performance improvements the jsonb column provides, it also gives us more operators we can use. It also provides us with additional indexing capabilities.
CREATE TABLE products ( id int GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY, name text NOT NULL, properties json |
To create a jsonb column with Prisma, we can use the @db.JsonB annotation.
productSchema.prisma
model Product { id Int @id @default(autoincrement()) name String properties Json? @db.JsonB |
Since jsonb is the default column used for storing JSON in Prisma we can omit using the @db.JsonB annotation.
Creating the above schema and generating the migration gives us the following code:
migrations/20230524030331_products/migration.sql
CREATE TABLE "Product" ( "id" SERIAL NOT NULL, "name" TEXT NOT NULL, "properties" JSONB, CONSTRAINT "Product_pkey" PRIMARY KEY ("id") |
Using the jsonb column gives us all of the functionalities of the json type and more.
Summary
In this article, we’ve gone through storing JSON in a PostgreSQL database through raw SQL and Prisma. We also compared the json and jsonb column types. While the above approach gives us a lot of flexibility, it forces us to give up some of the benefits of relational databases. Therefore, we should avoid overusing it and think twice before choosing it as our solution.
Series Navigation<< API with NestJS #109. Arrays with PostgreSQL and PrismaRecommend
About Joyk
Aggregate valuable and interesting links.
Joyk means Joy of geeK