[caikitnlp-169] Add LoRA configuration support for fine-tuning module

[ibm-peach-fish]

• Add LoRA configuration support in the toolkit functionality created in Refactor peft module to take out common peft config functionality #163

• Add LoRA configuration and training from text-generation module (train).

• Expose parameter required for LoRA configuration via .train function

• Add support for saving LoRA models with "merged weights". This is to be done in .train function itself, that way, the model that we configure to __init__ function will look like any other transformers model.

• Unit tests cover new/changed code

• Examples build against new/changed code

• README updated here

• Example for training lora added in example script

[ibm-peach-fish]

For the readme update:

From @gkumbhat on slack

Me: could i get more specification on what readme.md you want updated, I'm having a hard time following the link on 169
G: just this section: https://github.com/caikit/caikit-nlp/blob/main/README.md#:~:text=Salient%20Feature(s)-,Text%20Generation,-1.%20PeftPromptTuning
Particularly “Salient Feature(s)” part
Me: The link in the issue looked like that too / I think you mean the text generation row of the table in the introduction section / and from the links i think you're pointing to the PeftPromptTuning, but all of that work was dismissed, the LoRA stuff lives in the text genertaion model / and hence i am lost with what you want me to put where
G: oh yes / I was thinking it would be a mention after Fine-tuning Both modules above provide optimized inference capability using Text Generation Inference Server / that way its in point 2 for TextGeneration but we add a line stating this module can also do LoRA fine-tuning

Sometimes when I don't understand things, I'll just walk away for a bit then I 'get it' when I came back- often it works. This time it didn't. So I need more guidance on what to put where in the readme.md, but ott this PR is ready for review

[chakrn]

cc: @Ssukriti

Would be great to get your eyes on this

[gkumbhat]

Thanks for adding support for LoRA. Upon review a couple of things needs to be changed:

1. The input to train function needs to be a data model or built-in python data types. This is so that we can automatically then leverage those data types and type hints to generate a server spec servable via caikit.runtime. Under the hood, these parameter to .train function gets converted to a proto spec automatically and gets exposed to our gRPC and REST server. Currently it it accepting LoRAConfig, which won't work with auto generation of server and thus won't get exposed in API.
2. The lora vector and lora based PEFT model gets generated after the model is already trained. So it looks like the LoRA vectors / weights are actually not getting modified / trained.
3. Once we move the LoRAConfig parameters to data model type object,, we would also need to be filter and set "reasonable defaults" for those configuration parameters.
4. It looks like the lora config related changes in peft_config.py aren't actually getting used.
5. Upon looking at the implementation and considering generally how Lora is being referenced as part of "prompt tuning", we decided it would probably be better to move this to prompt tuning module and add a flag to save prompt vectors via merged weights or not. There will certainly be some things we would need to figure out to correctly hook it up for inferencing and its routing configuration. But that probably aligns more with how people are using it.

[ibm-peach-fish]

Based on the requested changes, I think a complete refactor is in order- closing.

[chakrn]

I would like to keep this open while Sukriti reviews

[Ssukriti]

@gkumbhat since I do not have write access to caikit-nlp, could you create a branch in main repo and merge this PR to that branch ? I can then continue to commit to that branch too, instead of having to pull from another fork

[Ssukriti]

added first PR to create data model and utilities #270

Remaining work is being paused till we have more direction on priorities. We can close this PR , and continue to refer to it for any further refactor as separate PRs