Show details for query tokens (#1392)

docs/api-reference/advanced/configuration.mdx

@@ -209,6 +209,7 @@ Alright, let's dive into what each key means in the yaml config above:
         - `top_p` (Float): Controls the diversity of word selection. A higher value (closer to 1) makes word selection more diverse.
         - `stream` (Boolean): Controls if the response is streamed back to the user (set to false).
         - `online` (Boolean): Controls whether to use internet to get more context for answering query (set to false).
+        - `token_usage` (Boolean): Controls whether to use token usage for the querying models (set to false).
         - `prompt` (String): A prompt for the model to follow when generating responses, requires `$context` and `$query` variables.
         - `system_prompt` (String): A system prompt for the model to follow when generating responses, in this case, it's set to the style of William Shakespeare.
         - `number_documents` (Integer): Number of documents to pull from the vectordb as context, defaults to 1

docs/components/llms.mdx

@@ -840,6 +840,52 @@ answer = app.query("What is the net worth of Elon Musk today?")
 ```
 </CodeGroup>
 
+## Token Usage
+
+You can get the cost of the query by setting `token_usage` to `True` in the config file. This will return the token details: `input_tokens`, `output_tokens`, `total_cost`.
+The list of paid LLMs that support token usage are:
+- OpenAI
+- Vertex AI
+- Anthropic
+- Cohere
+- Together
+- Groq
+- Mistral AI
+- NVIDIA AI
+
+Here is an example of how to use token usage:
+<CodeGroup>
+ 
+```python main.py
+os.environ["OPENAI_API_KEY"] = "xxx"
+
+app = App.from_config(config_path="config.yaml")
+
+app.add("https://www.forbes.com/profile/elon-musk")
+
+response, token_usage = app.query("what is the net worth of Elon Musk?")
+# Elon Musk's net worth is $209.9 billion as of 6/9/24.
+# {'input_tokens': 1228, 'output_tokens': 21, 'total_cost (USD)': 0.001884}
+
+
+response, token_usage = app.chat("Which companies did Elon Musk found?")
+# Elon Musk founded six companies, including Tesla, which is an electric car maker, SpaceX, a rocket producer, and the Boring Company, a tunneling startup.
+# {'input_tokens': 1616, 'output_tokens': 34, 'total_cost (USD)': 0.002492}
+```
+  
+```yaml config.yaml
+llm:
+  provider: openai
+  config:
+    model: gpt-3.5-turbo
+    temperature: 0.5
+    max_tokens: 1000
+    token_usage: true
+```
+</CodeGroup>
+
+If a model is missing and you'd like to add it to `model_prices_and_context_window.json`, please feel free to open a PR.
+
 <br/ >
 
 <Snippet file="missing-llm-tip.mdx" />