[Compute] Add llm.c request encoder kernel reference

[jacobthebanana]

[NLP] Add llm.c request encoder kernel reference

Type of Change

• New Pocket Reference
• Edit to Existing Pocket Reference
• Other (please describe):

Fixes #

Book

• fundamentals
• nlp
• cv
• rl
• fl
• responsible_ai

Description

Add LLM.C encoder forwarder references.

Checklist

• I have included appropriate contributor tags ({{#author}} or {{#authors}})
• I have added the reading time preprocessor tag under the title
• Content is concise and within 7 minutes reading time
• I have included relevant references and further reading links
• I have tested locally using mdbook watch books/<book-name> --open
• Pre-commit hooks pass without errors
• I have linked to related issues

Additional Context

CUDA Club

[nerdai]

@jacobthebanana can you kindly sync your local repo and force push the commit after?

[nerdai]

Hi @jacobthebanana,

I had to clean up your PR quite a bit, and in doing so, I hope I didn't modify the core of your contribution, namely your encoder_forward.md pocket ref.

Here's what I had to do:

1. resolve conflicts — your PR contained more than your contributed pocket ref. I suspect that you were probably branch was created off an out-of-date base. In future, before contributing your new pocket ref, please do (i) update your local main branch; (ii) create your new dev branch from this in-sync main branch

2. lint — we have linters to ensure the markdown is formatted nicely as well as doesn't contain any syntactical errors (though not all are caught by the linters). In future please do make lint and address any linting issues in your PR in order to get the checks to pass.

3. Finally, I also added your Github user instead of the placeholder VectorInstitute

[nerdai]

nit but can we change the title to "# Forward Pass Kernels of Positional Embeddings within GPT-2"

[nerdai]

in-line math in the mdbook's render of markdown math doesn't use single "$" as delimeters. Rather it uses '\( \)'. Please replace this and all instances of '$ ... $' with '\( ... \)'

[nerdai]

Thanks @jacobthebanana for the pocket ref contribution! After cleaning up your PR, I was able to make a first pass at reviewing the core content. Left a few comments

[nerdai]

Please also change this to MLA style

[_Karpathy, Andrej. "encoder_forward.cu." llm.c, GitHub, accessed 10 Apr. 2025, github.com/karpathy/llm.c/blob/master/dev/cuda/encoder_forward.cu._](https://github.com/karpathy/llm.c/blob/master/dev/cuda/encoder_forward.cu)

[Viky397]

New sentence: "The kernel uses #pragma unroll to automatically optimize this part of the code during compilation."

[Viky397]

Reads well! :)