Using the protocol for other languages (in particular python)

[Nova38]

I find twoslash enabled doc to be really helpful when reading typescript projects that use them in their docs. I have been working on a few personal projects that use shiki for syntax highlighting some python files. I saw that there was a twoslash protocol that was created (#17 (comment)) to allow for things other than the typescript language server to generate the info that twoslash needs to render it. I was wondering what the steps would be to get the data generated into twoslash for rendering in to shiki?

Also if i wanted to include the docstrings for the nodes would I need to convert them into the jsdoc equivalents or what format do the strings of the types bellow need to be?

twoslash/packages/twoslash-protocol/src/types/nodes.ts

Lines 15 to 25 in fcc9074

	export interface NodeHover extends NodeBase {
	type: 'hover'
	/** The string content of the node this represents (mainly for debugging) */
	target: string
	/** The base LSP response (the type) */
	text: string
	/** Attached JSDoc info */
	docs?: string
	/** JSDoc tags */
	tags?: [name: string, text: string | undefined][]
	}

[antfu]

How much is the difference between Python's doc strings and jsodc? You do not have to convert them into jsdocs, it's just a data field and it's up to the renderer to parse the string to DOM. So it's possible that you provide your own patched renderer or so to render directly with docstring.

[Nova38]

Python has several different docstring standards so it is a little less unified. However their plugins for sphinx that complice them into structured data that is more standardized that I could pass to a patched renderer like you suggested

[bismitpanda]

Is there anything for python to use the twoslash protocol?

[Nova38]

Not yet I haven't had a chance to work on it yet.

I have to ideas on how to go about implementing it though. One option is to extract the needed information for the typing, hover, etc... from a language server and then sterilize it into the twoslash protocol format.

Microsoft has a tool in python for extraction info from an Lsp that could be useful. This approach would potentially be scalable out to other languages using their respective lsp's.
https://github.com/microsoft/multilspy

The other option is to use a library like giffe (https://mkdocstrings.github.io/griffe/) that provides a way to anylis python code and extract types, doc strings, etc.... Then either write a python script to produce it or transform the json directly.n

[Nova38]

It also looks like there is a second protocol that exist on top of lsp that is ment for sterilizing some of the Lsp data to be called on by a much more limited environment. I haven't gotten far into the docs on it yet, but it might be handy for this use case.

https://microsoft.github.io/language-server-protocol/overviews/lsif/overview/

[julien-blanchon]

@Nova38 here is a working draft
https://github.com/julien-blanchon/twoslash-python/tree/main