@pothos/plugin-prisma-utils

[hayes]

Hey! I've been working a bit on the @pothos/plugin-prisma-utils plugin lately. The goal of this plugin is to make building code-generators for prisma types easier.

Building and maintaining a code-generator for full prisma crud operations has been outside of the scope of what I want to maintain for Pothos core, and am very excited to see projects like this that add these kinds of features as 3rd party plugins.

I'd be curious to hear if you think any of the new helpers in that plugin would be valuable for this project.

If you are interested, I'd love to collaborate on figuring out how to make the prisma-utils package as useful for 3rd party generators like yours as possible. I know there are probably still a lot of missing pieces to make this really useful, but hopefully this direction can help build more powerful generators in the future.

[quintal-william]

Hi @hayes, thanks for reaching out!

Your ideas for the plugin already look great! I'll definitely play around with it sometime this week to see if I can come up with some more detailed feedback. My first thoughts are:

When we first designed our code generator, extensibility was a big priority (use cases: rename fields, add auth layer, add complexity, etc). This is why you often see our definitions split into two parts, one to accept all defaults, and one that is the fully-extendable raw object:

In order to do this, we had to write some utility types based on the Pothos types that supported this API while maintaining type-safety. It would be nice if the plugin could have some built-in solution for this, because some of these types are good enough for now, but far from perfect for more advanced use cases due to my limited understanding of the Pothos type system.

Thanks again for reaching out! I hope we can have a good discussion about the future of your plugin in this issue thread. I know that @Cauen has some different views on this project, I hope he can shine some light on his thoughts 😄

[Cauen]

Hi @hayes
I'm glad to see you here willing to help-us make this lib even better, your work is amazing.

I think William was very accurate
Our goal is to bring speed at the same time as customization power.

This means:

1. Inputs (Automatic + Customizable)
2. Objects (Automatic + Customizable)
3. Resolvers: Queries/Mutations (Automatic + Customizable)

The important point is that the generated parts needs to be decoupled from the customized parts. So when there is a new generation, all customizations need to be maintained (and typesafe).

---

Improving Inputs with Prisma Utils Plugin

If I understand correctly, this plugins currently helps create Input Types

I also saw that you implemented 2 types of generators. A static one (like ours) and a dynamic one (which I didn't know was possible :O)

Static Generator:

I don't know what advantages your static generator (using the prisma helpers) would have over ours (maybe typescript checking performance?). If there's an advantage, we can update to how you did.

I believe it would be a good addition here, if we used it containing some "native" way to extend the inputs (Its not directly related to prisma, so maybe a custom plugin or method to Pothos core)... (Our current way is a bit "ugly")

Maybe something like this:

// schema/__generated__/inputs.ts
export const PostCreate: InputObjectRef<Prisma.PostCreateInput> = builder.prismaCreate(
  'Post',
  {
    name: 'PostCreate',
    fields: () => ({
      id: 'Int',
      createdAt: 'DateTime',
      updatedAt: 'DateTime',
      title: 'String',
      ...
    }),
  },
);

// schema/Post/inputs.ts
export const PostCreateCustom = builder.extendInput(
  PostCreate,
  ({ name, description, fields }) => ({
    name: `${name}Custom`,
    description: `${description} Custom`,
    fields: ({
      ...fields,
      customField: 'Int',
    }),
  }),
);

Dynamic Generator

Here I see a huge potential

It's really beautiful that we can generate the inputs just when we use them. And with a few code changes, we can make it easily customizable for derived inputs.

Using this with the Crud Generator can generate a much leaner schema in many cases. What do you think? @saphewilliam

Our crud generator

I don't know if it meets the scope of your intention, but here's where your experience could help us...

As William said, keeping our objects/resolvers customizable needs these utility functions to be maintained separately, which bring 2 problems:

1. Difficulty to maintain (since imported types/generics are complex)
2. Unexpected errors in cases of missmatch with official prisma plugin version.

If this was built-in, we could free users to update prisma and pothos plugins freely.

[hayes]

Thanks for all this context, this is exactly what I've been looking for.

When we first designed our code generator, extensibility was a big priority (use cases: rename fields, add auth layer, add complexity, etc). This is why you often see our definitions split into two parts, one to accept all defaults, and one that is the fully-extendable raw object:

This makes a lot of sense, and is not something I have thought much about supporting yet. Something that makes this really tricky is the potential for circular references. It's really hard to create references to types that may not end up being defined

Making a few assumptions here:

• Omitting types is part of the customization, and this is done after the generator runs
• You want to support nested inputs
• This is sticking with dynamic generation

There is an internal hook builder.configStore.onFieldUse that could potentially be used as a way to lazily define types for refs only if they actually get used (this is something I'd need to explore further, not sure how possible this is).

Overall, I really like the idea of generating objects that contain all the potential fields, and then letting people decide which fields they want to actually include when they manually define the type (or rename the fields, etc).

I think there is a way to potentially do something where types get used by a field are lazily added to the schema only if the type was not manually added first.

eg:

// generated
const CreateUserFields = {
  name: t.field(...),
  name: t.field(...),
  profile: t.field({ type: ProfileRef, ...})
}

const CreateProfileFields = {
  bio t.field(...),
}

// user code
builder.prismaCreate('User', {
  fields: t => ({
    ...CreateUserFields
  })
})
// profile automatically with all fields because it's used

// user code without profile
builder.prismaCreate('User', {
  fields: t => ({
    ...CreateUserFields,
   profile: undefined
  })
})
// CreateProfile not created

// user code with profile definition
builder.prismaCreate('User', {
  fields: t => ({
    ...CreateUserFields,
   profile: undefined
  })
})
// profule not automatically created because it was customized/defined manually
builder.prismaCreate('User', {
  fields: t => ({
    ...CreateProfileFields,
  })
})

In order to do this, we had to write some utility types based on the Pothos types that supported this API while maintaining type-safety. It would be nice if the plugin could have some built-in solution for this, because some of these types are good enough for now, but far from perfect for more advanced use cases due to my limited understanding of the Pothos type system.

I think there are probably some utilities that could make sense to export either from core or as part of a plugin. Specifically, making it easy to define objects that contain a set of fields for a specific type (without actually adding them to the schema).

Utilities for options objects for all the different builder methods might get a bit complex, but could be worth exploring as well. I definitely want to make maintaining these kinds of tools easier. Type-safety and type comparability across versions is really hard to ensure with types that are as complex as what pothos requires, so I definitely understand this pain.

The important point is that the generated parts needs to be decoupled from the customized parts. So when there is a new generation, all customizations need to be maintained (and typesafe).

This is huge, love this emphasis. I think this is part of why I've always been so hesitant to add any kind of codegen to pothos core. Ensuring that things don't change under your feet when you regenerate, and that everything stays type-safe is really important.

If I understand correctly, this plugins currently helps create Input Types

Yup, currently the main operations Create, Update, OrderBy, Where, WhereUnique, and there are some additional helpers to make sure the nested features of those operations work (filters for scalars and lists, and definitions for all the relation mutation options which get pretty complex)

I don't know what advantages your static generator (using the prisma helpers) would have over ours (maybe typescript checking performance?). If there's an advantage, we can update to how you did.

A big piece of what the plugin helpers do is normalizing nulls/undefined. GraphQL allows both nulls and undefined to be sent for nullable fields, which means that you need to coerce all values in your input types to strip out the nulls that prisma doesn't expect.

The other part is that is simplifies the inputs a lot. Filters you just list operations, and the plugin will generate fields of the right type. For other fields, you can specify just the type, and the plugin takes care of making sure things are wrapped as lists or nullable as needed.

The last thing the plugin does is ensure type-safety of the inputs by making sure the field types you define are actually compatible with the prisma types the input are for. This could be either a positive or negative depending on how you look at it. You get a little less flexibility, but the extra type-safety ensure better compatibility when passing things directly to prisma.

I want to re-iterate that the generators I wrote are hacky demos to show what generated types could look like, and the goal is 100% to support 3rd party generators like yours, and not for anyone to use the generators I wrote.

I believe it would be a good addition here, if we used it containing some "native" way to extend the inputs (Its not directly related to prisma, so maybe a custom plugin or method to Pothos core)... (Our current way is a bit "ugly")

This is interesting. You can extend most types in pothos, but inputs have always required a single/complete definition to ensure type-safety.

I think potentially extending input types could be safe, because the new ref should only have additional fields. If you wanted to strip out fields in the extends, that would introduce type-safety issues, but could be useful for libraries/plugins.

Would the same use-case be addressed by something like I described above where types can be lazy-defined?

It's really beautiful that we can generate the inputs just when we use them. And with a few code changes, we can make it easily customizable for derived inputs.

I think there is a lot of potential here. The big downside is that there is no "ejection" so you can't just decide that you want to stop generating and copy-past the code, but this approach is entirely flexible and could also work for full crud API.

Customization may be a little tricky, but should be possible, I'd be happy to help out on making this work of that is a direction you are interested in going in.

I don't know if it meets the scope of your intention, but here's where your experience could help us...

Not out of scope, I think so far things have been limited to inputs for that plugin because thats the hardest part to do manually right now. The utils are meant for people writing generators, but I also wanted to make it easier to manually build those input types without codegen (for smaller schemas at least).

But longer term, I think any types of utils that simplify creating output types are also on the table.

[hayes]

Spent some time thinking about what could be exported from Pothos that could make things easier for you regarding generated types.

One thing that might work is to create helpers for lazy defining types. The API I was playing around with looks something like this:

// Exported by a pothos package
type ExtendableOptions<T extends {}, Fields extends FieldMap> = {
  [K in keyof T]?: K extends 'fields'
    ? T[K] extends (...args: any[]) => unknown
      ? (t: Parameters<T[K]>[0], fields: ReturnType<T[K]>) => Fields
      : never
    : T[K];
};

export function lazyObject<
  Types extends SchemaTypes,
  Interfaces extends InterfaceParam<Types>[],
  Param extends ObjectParam<Types>,
  Options extends ObjectTypeOptions<Types, Param, ParentShape<Types, Param>, Interfaces>,
>(builder: PothosSchemaTypes.SchemaBuilder<Types>, param: Param, { fields, ...options }: Options) {
  return <T extends ExtendableOptions<Options, Fields>, Fields extends FieldMap>({
    fields: newFields,
    ...extension
  }: T): ObjectRef<OutputShape<Types, Param>, ParentShape<Types, Param>> => {
    const fieldsFn: Options['fields'] = (t) => {
      const originalFields = fields?.(t) ?? {};

      return newFields ? newFields(t, originalFields as never) : originalFields;
    };

    const merged = {
      ...options,
      ...extension,
      fields: fieldsFn as never,
    };

    return builder.objectType(param, merged as never);
  };
}

// Builder is setup somewhere else
const builder = new SchemaBuilder<{
  Objects: {
    User: {
      id: string;
      name: string;
      email: string;
    };
  };
}>({});

// Generated code, accepts all options for the relevant object
const defineUser = lazyObject(builder, 'User', {
  fields: (t) => ({
    id: t.exposeID('id'),
    name: t.exposeString('name'),
  }),
});

// User code (or invoked by another generated method)
export const User = defineUser({
  description: 'A user',
  // fields works just like normal, but is passed the fields from the original definition
  fields: (t, fields) => ({
    email: t.exposeString('email'),
    id: t.exposeID('id'),
    nameField: fields.name,
  }),
});

I would basically export a bunch of "lazy" method for defining types.

• lazyObject
• lazyInterface
• lazyUnion
• lazyEnum
• lazyInputObject
• lazyNode
• lazyPrismaObject
• lazyPrismaNode

One big thing that isn't quite figured out here is how you would define fields that reference these lazy types. Would these be roughly the direction for the helpers you were imagining that would require you to have fewer complex types in you generators while still allowing full customization?

[quintal-william]

[Edit: accidentally submitted]

I think your idea nicely extracts our generator design decision into a more generic utility function! Especially by incorporating the builder into the function call instead of hard coded in the definition like we did, and by using that given builder to hide complexity from the generator user. I like it! Some thoughts:

• It would be even better if the fields could also be lazily defined as such, something you mention could be difficult to accomplish.
• Since our generator also supports extending the generated queries and mutations, you could also add lazyQuery and lazyMutation to the list while you're at it.
• I think, if possible, I would allow the generator consumer to access the full generated user object as a top-level anonymous function. This would allow the user to modify any bit of the generated code, it would standardize the define* function calls and allow the user to keep using the familiar Pothos builder API to extend the generated code (see example below).

Assuming these changes, then the API for our generator would move from the example in our docs to the following:

// ./src/graphql/User/object.ts

import { defineUser, defineUserSessionsField } from '@/graphql/__generated__/User';

// Pass no parameters to accept all default generated object code
export const DefaultUser = defineUser();

// Or modify it as you wish
export const ModifiedUser = defineUser((generatedUser) => ({
  // Don't implicily merge options but allow the user to explicitly specify which options to include
  ...generatedUser,

  // Use generated properties to create new ones
  description: "[DOCS] " + generatedUser.description,

  fields: (t) => {
    // Type-safely omit and rename fields
    const { password: _password, email: emailAddress, ...fields } = generatedUser.fields(t);

    return {
      ...fields,
      // Renamed field
      emailAddress,
      // Edit and extend field
      sessions: defineUserSessionsField((generatedField) => {
        ...generatedField,
        args: { ...generatedField.args, customArg: t.arg({ type: 'String', required: false }) },
        authScopes: { admin: true },
      }),
      // Add custom field
      customField: t.field({ type: 'String', resolve: () => 'Hello world!' }),
    };
  },
}));

Where the generated code would go from the current state to something like this:

// ./src/graphql/__generated__/User/object.base.ts

import { lazyObject, lazyExposeId } from '@pothos/plugin-prisma-utils';
import { builder } from '@graphql/builder';

export const defineUser = lazyObject(builder, 'User', {
  fields: () => ({
    id: defineUserIdField(),
    name: defineUserNameField(),
    email: defineUserEmailField(),
    password: defineUserPasswordField(),
    // `lazyRelation` knows to call t.relation under the hood
    sessions: defineUserSesssionsField(),
  }),
});

export const defineUserIdField = lazyExpose(builder, 'User', 'ID', { // or `lazyExposeID`
  description: "This is the user ID",
  nullable: false,
});

export const defineUserNameField = lazyExpose(builder, 'User', 'String', { // or `lazyExposeString`
  description: undefined,
  nullable: false,
});

export const defineUserSesssionsField = lazyRelation(builder, 'User', 'Session', {
  description: undefined,
  nullable: false,
  args: undefined,
  query: undefined,
});

I would think this is my ideal API, I don't know how you would implement these lazyRelation and lazyExpose functions though (are you able to extract t from just the builder?). I think this API would reduce complexity for both the generator builder and the generator user while keeping the exact same customization possibilities.

As a separate thought: if you don't intend for people to use your "hacky" code generation examples, maybe in the future when this generator implements the new features of @pothos/plugin-prisma-utils you could cite this project as a "proper" example of a code generator.

[hayes]

It would be even better if the fields could also be lazily defined as such, something you mention could be difficult to accomplish.

Makes sense, I'll play around with some options for this, and see if there is a good way to incorporate it cleanly. Ideally you wouldn't have to have the fields separate, I'd love it if you could just customize fields as part of the same API

export const ModifiedUser = defineUser((generatedUser) => ({
  // Don't implicily merge options but allow the user to explicitly specify which options to include
  ...generatedUser,

  // Use generated properties to create new ones
  description: "[DOCS] " + generatedUser.description,

  fields: (t) => {
    // Type-safely omit and rename fields
    const { password: _password, email: emailAddress, ...fields } = generatedUser.fields(t);

    return {
      ...fields,
      // Renamed field
      emailAddress,
      // Edit and extend field
      sessions: fields.sessions.define((t, fieldOptions) => {
        ...fieldOptions,
        args: {
           ...fieldOptions.args,
           // customize existing args
           id: fieldOptions.id.define({
             description: 'id arg!',
           }),
           // add new args
           customArg: t.arg({ type: 'String', required: false }) },
        authScopes: { admin: true },
      }),
      // Add custom field
      customField: t.field({ type: 'String', resolve: () => 'Hello world!' }),
    };
  },
}));

I'm not sure how hard/feasable an API like that would be. I'd really like to have everything consolidated into a single API where possible. I think creating these kinds of extendable field definitions would need to be something that can be used on it's own.

I may even just bake that into core, because customizing fields after definitions from various plugins would probably be helpful.

Since our generator also supports extending the generated queries and mutations, you could also add lazyQuery and lazyMutation to the list while you're at it.

Absolutely, root types should definitely be included.

I think, if possible, I would allow the generator consumer to access the full generated user object as a top-level anonymous function. This would allow the user to modify any bit of the generated code, it would standardize the define* function calls and allow the user to keep using the familiar Pothos builder API to extend the generated code (see example below).

Yes, love that idea, and should be pretty straight forward to do.

[Cauen]

@hayes Glad you embraced this cause

Clearing

(Hayes: About objects) I think there is a way to potentially do something where types get used by a field are lazily added to the schema only if the type was not manually added first.
eg...

(Hayes: About inputs extensability) Would the same use-case be addressed by something like I described above where types can be lazy-defined?

I don't know if this was clear at some point, but I saw that you forked the project and as your examples are clearer now, I think you get the idea of what I'm going to say...
But as a matter of conscience...

We generate inputs and objects in differently.

1. Inputs are generated and added (by using builder.inputType())
2. Objects are defined, but not added. (It needs to be executed, one by one, importing the generated base files, or by running the generated autocrud function)

That said, extending the inputs solves this use case: #27

(Hayes: About dynamic generator) I think there is a lot of potential here. The big downside is that there is no "ejection" so you can't just decide that you want to stop generating and copy-past the code, but this approach is entirely flexible and could also work for full crud API.
Customization may be a little tricky, but should be possible, I'd be happy to help out on making this work of that is a direction you are interested in going in.

I think this disadvantage would not hinder potential users of this generator. Anyway, I believe that in a future version, we could use it as a custom configuration.

(Hayes: About helpers) But longer term, I think any types of utils that simplify creating output types are also on the table.

🎉 Thanks for that

Official Helpers

Once again, I make William's words mine.

Note: Our generator automatically handle integration with the official prisma-plugin.

• We use builder.prismaObject to execute generated objects.
• We use t.prismaField for prisma createOne, findFirst ..., and t.field for count, deleteMany and updateMany.

So, maybe we need:

• lazyObject as a new Pothos lazy concept exported from core.
• lazyPrismaObject as lazy prisma integration exported from prisma-plugin. (We would use this here)
• lazyQuery, lazyMutation from core as concept.
• lazyPrismaQuery, lazyPrismaMutation from prisma-plugin (We would use this here) .

If this increases the complexity, instead of the helpers being more complete like the ones you proposed... Just being the same as the ones that already exist today...

Just some examples:

Current API for customizing Objects

import { builder } from "../builder";
import { ContentPageObject } from "../__generated__/ContentPage";
import { parseShortcodeHtml } from "./utils/shortcode/render";

/**
 * Adds a `computed version` of field
 * Here, you can:
 *  - Remove fields
 *  - Rename fields
 *  - Replace fields
 *  - Update description that are generated from schema.prisma doc comments
 *  - Update findUnique function from official pothos plugin `prisma-plugin`
 *  - Apply pothos plugins
 */
export const ContentPage = builder.prismaObject("ContentPage", {
  ...ContentPageObject,
  fields: (t) => {
    const { ...fields } = ContentPageObject.fields(t);

    return {
      ...fields,
      renderedHtml: t.string({
        resolve: async ({ HTML }) =>
          HTML ? parseShortcodeHtml({ html: HTML }) : "",
      }),
    };
  },
});

Current API for customizing Queries/Mutations

import { configs } from "@/configs";
import { builder } from "@/schema/builder";
import { findManyShopOrderQueryObject } from "@/schema/__generated__/ShopOrder";

/**
 * Adds user access level for orders
 * Here, you can:
 *  - Protect route
 *  - Change return type
 *  - Change nullability
 *  - Insert/Remove args
 *  - Apply pothos plugins
 */
export const findManyShopOrder = builder.queryFields((t) => {
  const findMany = findManyShopOrderQueryObject(t);
  return {
    findManyShopOrder: t.prismaField({
      ...findMany,
      resolve: async (query, _root, args, context, _info) => {
        const user = context.user;
        if (!user) throw new Error("Não autorizado");
        if (configs.isDevelopment || user.role === "ADMIN")
          return findMany.resolve(query, _root, args, context, _info);

        return findMany.resolve(
          query,
          _root,
          {
            where: {
              ...args.where,
              idUser: user.id,
            },
          },
          context,
          _info
        );
      },
    }),
  };
});

Proposed API for Objects (William)

// ./src/graphql/User/object.ts

import { defineUser, defineUserSessionsField } from '@/graphql/__generated__/User';

// Pass no parameters to accept all default generated object code
export const DefaultUser = defineUser();

// Or modify it as you wish
export const ModifiedUser = defineUser((generatedUser) => ({
  // Don't implicily merge options but allow the user to explicitly specify which options to include
  ...generatedUser,

  // Use generated properties to create new ones
  description: "[DOCS] " + generatedUser.description,

  fields: (t) => {
    // Type-safely omit and rename fields
    const { password: _password, email: emailAddress, ...fields } = generatedUser.fields(t);

    return {
      ...fields,
      // Renamed field
      emailAddress,
      // Edit and extend field
      sessions: defineUserSessionsField((generatedField) => {
        ...generatedField,
        args: { ...generatedField.args, customArg: t.arg({ type: 'String', required: false }) },
        authScopes: { admin: true },
      }),
      // Add custom field
      customField: t.field({ type: 'String', resolve: () => 'Hello world!' }),
    };
  },
}));

Proposed API for Queries/Mutations (Mine)

// ./src/graphql/User/queries/findMany.ts
import { defineFindManyUserQuery } from "@/graphql/__generated__/User";

// Pass no parameters to accept all default generated object code
export const findManyUserAuto = defineFindManyUserQuery();

// Or modify it as you wish
export const findManyUserCustom = defineFindManyUserQuery((generatedQuery, t) => ({
  ...generatedQuery,
  args: {
    ...generatedQuery.args,
    customArg: t.arg({ type: "String", required: false }),
  },
  resolve: async (query, root, args, context, info) => {
    const { customArg } = args;
    console.log(customArg);
    return generatedQuery.resolve(query, root, args, context, info);
  },

  // Add an custom extension
  authScopes: { admin: true },
}));

[hayes]

I think this all makes sense so far.

On my end, I think what I am going to focus on over the next few weeks is moving towards a 4.0 release of Pothos. The biggest feature that I haven't been able to build without a breaking change is the ability to build good chaining APIs on type and field refs.

The original use case was validation like:

builder.inputType(...).withZodValidation(sodSchema) => Input type<ShapeFromZod> where you could basically chain things off of refs to create modified versions of the refs with additional behavior that would also change the type information.

Storing info about the builder on the refs is the main change, and would allow potentially having a .extend or .define or something similar on any object or field ref, and would potentially make a lot of the APIs we talked about above more compostable without having to account for each plugin specifically.

I think some of the 4.0 changes I have in mind will lay a good groundwork for implementing the ideas we discussed above.

The other thing I was looking at was potentially defining objects and refs that are not implicitly added to the builder for more modular schemas which may also be relevant. Lots of details to figure out there, so not sure how applicable this is yet.

[quintal-william]

Good luck with working on all that @hayes, it sounds very promising! We'll keep a close eye on development and you'll know where to find us if you need any more feedback 😄

[Cauen]

Hello @hayes
Yesterday I noticed the release of version 4.0 3 days ago
I believe it has been a long journey, so congratulations 🚀

Did you manage to achieve what you expected with this new version? :D

[hayes]

I probably got about 80% of what I wanted to do into the 4.0 release, but some of the things I wanted to do ended up being too complicated to do in a way that wouldn't require much larger migrations.

The core things relevant here I think all made it in, although there might be some new options or minor changes required in Pothos to enable them (for example, builders still track all the refs they create, but it should now be easy to add an option so that they aren't automatically included).

There are also a lot of things that should be possible now that haven't been thoroughly tested. For example Refs from different builders should be usable together, but there may be some edge cases that require minor fixes to make it a smooth developer experience. The goal was to get 4.0 out with the core breaking changes in a way that is relatively easy to migrate, and unblock more incremental non breaking changes to add new capabilities

The short answer is yes, 4.0 should have whats needed to build the things we talked about above, but there might be minor changes or options I need to add for certain edge cases