  Welcome     iframe width  560  height  315  src  https   www youtube com embed SBZWj7qKtFA si neogiHQmE7i 8Uvz  title  YouTube video player  frameborder  0  allow  accelerometer  autoplay  clipboard write  encrypted media  gyroscope  picture in picture  web share  allowfullscreen   iframe     GroundX APIs help you rapidly build RAG  retreieval augmented generation  applications that you can trust     We ve been building the GroundX platform alongside LLM providers like OpenAI  Google and Meta for more than two years and have worked hard to build a framework that will save you many hours of pain while providing the accurate  hallucination resistant answers you need         Build Your First RAG App in 10 Minutes or Less    Whether you re new to RAG development or already deep in the practice  GroundX APIs are designed to save you time  We ve bundled all the pieces you need into one seemless platform that just works     Proprietary ETL Engine  extract  transform  load  to process your documents    Semantic Database    Dynamic Chunker    Powerful Content Search with Proprietary Re Ranker    Light Orchestrator    Enterprise security with bank level encryption    If you re not familar with all of those terms  don t worry  We ve removed the time  pain and expense of writing thousands of lines of code and integrating multiple platforms into just a few API calls  Your complex documents go in one side  LLM ready search retrievals come out the other with industry leading accuracy  Then call the LLM of your choice for completions      Some engineers have told us they are building RAG apps up to 10 times faster than with any other tools on the market          Cutting Edge Accuracy    GroundX is also designed to tackle one of the most difficult challenges of working with LLMs   their tendency to make things up or hallucinate when they don t know the right answer      To tackle hallucinations  we first have to understand what they are  Essentially  when LLMs run out of facts  they just make them up  They re like the know it all at the party that should have shut up 30 minutes ago  The problem is much worse with RAG apps where the LLMs have never seen your data before     The key here is to make sure the LLM understands your data when you send it for completion  We ve found that 90  of hallucinations are data problems NOT prompt problems  And you can t prompt your way out of a data problem       GroundX solves these issues with a proprietary ingest pipeline for your data that       Converts your documents  cleans the text and reformats it into LLM friendly data    Dynamically chunks your content at natural break points that express full ideas    Automatically creates contextual metadata for your chunks    Automatically builds a knowledge graph to maintain entity relationships across documents  Coming Soon     Under the hood  we ve built a fine tuned computer vision model for your docs and a pipeline of several LLMs to understand the layout of your documents  dynamically chunk and summarize your content and search your data  Give us your complex documents littered with tables  forms  headers  footers  index pages and other visual formatting that chokes LLMs and our pipeline returns clean  simple LLM ready data      We also built a proprietary search engine that outperforms vector similarity  provides fuller retrievals and re ranks each retrieved chunk for its adherence to the question       This all happens without writing a single line of code  Just ingest your content  search and complete  With GroundX you finally have RAG you can trust  made simple             Key Features    In addition to speed and accuracy  our core features are         Effortless Content Upload    Upload content from various sources like websites  documents  presentations  and raw text files in one centralized location with our Content Upload API       Sophisticated Automated Pre Processing    Our ingestion pipeline will automatically clean and format your content for use with LLMs  generate valuable metadata  and improve search and retrieval performance       Superior Search Capabilities    Make the most of our Search API  employing a proprietary combined vector and semantic approach for accurate and relevant search results       Secure Storage and Transmission    Data is encrypted at rest and during transmission to an LLM  Each corpus is stored in virtual isolation        Effortless Content Upload    At GroundX  we understand the complexity of managing diverse content sources  That s why we ve designed the Content Upload API  a comprehensive solution for uploading your data across multiple formats  including documents  presentations  raw text files  and even entire websites via a URL  The API ensures a secure transmission and storage of your content  simplifying your content management process with a centralized storage system       Sophisticated Automated Pre Processing    GroundX s pre processing capabilities enable you to enhance your content s utility and value  Our ingestion pipeline will   automatically   clean your data  re format your content for LLMs  and generate valuable metadata such as document  page  and section summaries  This preparation for interacting with an LLM significantly enhances the performance of search and retrieval tasks         Clean your Data    extraneous text  such as page and line numbers  page footers  and other non essential text that hinders an LLMs ability to understand your content  is automatically removed      Re Format for LLMs    common document elements such as columns  tables  and forms are re formatted for better search and retrieval performance and translated to a more appropriate style for LLM completions      Generate Metadata    document  page  and section summaries are automatically generated to better preserve the context of your content       Superior Search Capabilities    At GroundX  we ve taken a fresh approach to search that s optimized for RAG applications  The results consistently outperform vector similarity  the industry standard   How does it work     First  we re storing LLM friendly versions of your data in our database  explained above   That s vital for LLMs to be able to understand your content  Every retrieval contains the original text  automatically created metadata that summarizes the key elements of your chunks and an LLM friendly rewrite of your chunks     Then we pair a semantic search with a proprietary re ranker that scores every retrieval for adherence to the question  And finally  we merge all of those components to creat a suggested retrieval that is tuned to send to LLMs for completion      It s the first search and retrieval engine built specifically for RAG and LLM apps        Secure Storage and Transmission    The GroundX team spent more than a decade at IBM building highly trusted enterprise applications for millions of global users  We know how important security and privacy are to every enterprise and how much work goes into getting there  So we built enterprise security right into the platform     First  we believe your data is your data  You or your customers own it  not us  Your uploaded data is always encrypted at rest and in transit  We don t train LLMs on your data and the whole environment runs in a  virtual private cloud with one set of keys that you control      Today  this runs as a service we manage for you on AWS  Soon we ll have versions of our platform you can run on your own managed cloud         What You Can Build    Our customers are building all kinds of amazing LLM powered applications with GroundX APIs including customer service bots  personalized sales assistants  legal tools  advanced knowledge base search tools  health applications and more  Let your imagination run wild and be sure to tell us what you re cooking  We love to support the amazing applications built on GroundX        Getting Started    Before you dive in  you ll need an API key to authenticate your requests  To obtain your key  please  create an account  https   dashboard groundx ai auth register   Once you ve done that  you re ready to start using our APIs     In the following sections  you ll find detailed guides for each API  including how to authenticate  make requests  and what response formats to expect  Each guide is accompanied by examples to help you understand how to use the APIs effectively        Explore the APIs    In our documentation  you ll find detailed information about each API  including endpoints  request and response formats  and any required parameters  We provide code examples to assist you in integrating our APIs seamlessly into your projects     We re continuously working to improve our service based on user feedback  so we appreciate any suggestions or insights you have regarding our APIs  If you have any questions or need assistance  don t hesitate to reach out to our support team     Join us in our mission to unlock the power of knowledge  Let s make data more accessible  manageable  and valuable together  Get started with GroundX today 

Welcome

  Concepts   iframe width  560  height  315  src  https   www youtube com embed  JT yRtF5 M si Nb5jIBLfJsNZ8lwJ  title  YouTube video player  frameborder  0  allow  accelerometer  autoplay  clipboard write  encrypted media  gyroscope  picture in picture  web share  allowfullscreen   iframe     This page introduces some of the key concepts that form the foundation of GroundX s suite of APIs and services  Understanding these concepts will help you make the most of our APIs        Ingest    GroundX s Ingest API provides a powerful tool for uploading your content  supporting a variety of data types and formats  You can upload entire websites via a URL  documents  presentations  and raw text files  The API is designed to ensure secure transmission and centralized storage of your content  simplifying content management        X Ray    A document X Ray is the summarized GroundX understanding of your documents  It is the end result of the GroundX Ingest API and contains document  section  and chunk layout and metadata information  This information is stored in the GroundX search database and can be accessed using the GroundX Search API  You can also download the X Ray data and use it outside of GroundX Search in your own internal systems        Buckets    Buckets are a core feature of the GroundX APIs  aimed at the effective organization and storage of content  When documents or other forms of content are uploaded to the GroundX pipeline  they are processed and then stored in buckets  These buckets act as an organizational unit for content  making it easy to reuse and share buckets of content across different projects  The search APIs will search through the content in any buckets associated with a particular project        Projects    Projects in GroundX serve as the primary organizational unit  encompassing multiple buckets and the content they contain  When using the search APIs  you will search your buckets at the project level using the projectId  The Search APIs will scan all buckets and content associated with that specific project  In essence  while buckets organize content  projects consolidate these buckets for broader  unified access and searchability        Search    GroundX s Search API implements a proprietary combined vector and semantic approach  offering a level of search precision and relevance that is superior to traditional methods  The API allows for real time application of pre processors to better understand and match queries to relevant content        Authentication    To ensure secure access to its services  GroundX uses API Key based authentication  The API Key is provided to the users upon completion of a request form and is used in the request header  X API Key  for authentication   

Concepts

  Local Content Ingest   iframe width  560  height  315  src  https   www youtube com embed  iMVoB5paeY si h0pvy F4Qoe0 7EF  title  YouTube video player  frameborder  0  allow  accelerometer  autoplay  clipboard write  encrypted media  gyroscope  picture in picture  web share  allowfullscreen   iframe        Introduction    In this tutorial  we ll cover how to add or ingest your local files to GroundX     GroundX s true potential commences with the file ingestion process  one of its key advantages over other RAG solutions       With our proprietary ingest pipeline  your files undergo three critical processes in which GroundX          formats your content for LLM use      parses content into intelligible text chunks      and generates contextual metadata     Unlike other RAG solutions that require you to previously convert your files into plain text  Ground X is compatible with a wide variety of file formats  detects document structures  such as tables or page numbers  eliminates clutter  and re writes content so that it can be clearly understood by an LLM        Getting started        Required information  Before we begin  make sure you have the following information       The ID of the GroundX bucket in which you wish to store your file     The path of the file you want to upload from your computer     The file name you wish to give your file once it s in the GroundX bucket     Indicate the file type to get the file correctly processed      Example         code       python  bucketId   6830  ingestLocal    documents Aristotle rhetoric pdf    fileName    aristotle rhetoric pdf    fileType    pdf             javascript  const bucketId   6830   const filePath    documents Aristotle rhetoric pdf    const fileName    aristotle rhetoric pdf    const fileType    pdf                      Adding extra search data  Although not required because GroundX automatically generates contextual search data for your files  you can add extra search data to take maximum advantage of GroundX s search capabilities  help maintain document context in the search query responses  and add tags or notes indicating instructions on how to handle the search results      Example         code       python  searchData          title   rhetoric        author   Aristotle        keywords    Ethos    Pathos    Logos    Rhetorical Triangle    Persuasion                 javascript  const searchData          title   rhetoric        author   Aristotle        keywords    Ethos    Pathos    Logos    Rhetorical Triangle    Persuasion                         Set up environment  Set up your environment      Example         code       python  from groundx import Groundx    groundx   Groundx       api key GROUNDX API KEY                javascript  import   Groundx   from  groundx typescript sdk    import fs from  fs      const groundx   new Groundx        apiKey  GROUNDX API KEY                         info title  API Key      The  GROUNDX API KEY  placeholder represents your API key  We recommend storing your API key as an environment variable and accessing it from there  For this purpose  you can use libraries such as  dotenv  https   www npmjs com package dotenv  in Node js or  os  https   docs python org 3 library os html os environ  in Python     Go to the  GroundX dashboard  https   dashboard groundx ai auth login  to get your API key     Go to the  GroundX SDKs documentation  https   github com groundxai groundx sdks groundx sdks  for installation details             API request  Make the API request to ingest local documents and include the variables in the request body        code       python  response   groundx documents ingest local       body                            blob   open ingestLocal   rb                  metadata                       bucketId   bucketId                    fileName   fileName                    fileType   fileType                    searchData   searchData                                                   javascript  const response   await groundx documents ingestLocal                           blob  fs readFileSync filePath                metadata                     bucketId  bucketId                   fileName  fileName                   fileType  fileType                   searchData  searchData                                                        note  Go to  api Document ingestLocal  for endpoint details             API response    After making the request  you should receive a response with  processId  and  status   This response indicates that GroundX is uploading or ingesting your file into the indicated bucket        json          ingest               processId    23e782ac 3829 4833 965d e77b4e289885              status    queued                       Final details    Processing time depends on the size of your files  File size can be up to ten megabytes      After automatically ingesting your files and eliminating the typical complexity of other RAG solutions  GroundX has prepared your content for searchability and automated response generation for your queries   

Local Content Ingest

  Remote Hosted Content Ingest   iframe width  560  height  315  src  https   www youtube com embed KUeOKCiK2gw si vTpI YLgE iZPH7a  title  YouTube video player  frameborder  0  allow  accelerometer  autoplay  clipboard write  encrypted media  gyroscope  picture in picture  web share  allowfullscreen   iframe        Introduction    In this tutorial  we ll go over how to add or ingest remote hosted content into Ground X  This is the step where the magic begins  Ground X s ingestion pipeline is much more than just simply extracting content from your files        Through its unique ingestion pipeline  comprised of three critical stages  Ground X       Formats your content for LLM use    Parses content into intelligible text chunks    And generates contextual search data    Unlike other platforms that require content to be processed in plain text before being ingested  Ground X can automatically ingest a wide range of content types for you and also recognizes document structures  such as tables or page numbers  eliminates clutter  and rewrites content that can be clearly understood by an LLM       GroundX s fine tuned computer vision model identifies the coordinates of these objects  extracts their content  and then converts them into LLM readable formats          Getting started      Required information  To add online files to GroundX  you simply need the following information         BucketID  The ID of the GroundX bucket in which you will store your document      SourceURL  The URL of the file you want to add to your GroundX bucket       Example         code       python  bucketId   6839   sourceUrl    https   data chhs ca gov dataset hci walk bicycle xls              javascript  const bucketId   6839   const sourceUrl    https   data chhs ca gov dataset hci walk bicycle xls    const searchData          title   Time Walk Bike to Work  2001 2011        publisher   California Department of Public Health        homepage   https   catalog data gov dataset        abstract   This table contains data on the percent of population aged 16 years or older whose commute to work is 10 or more minutes day by walking or biking for California  its regions  counties  and cities towns                          Adding extra search data  Provide document context with extra search data  Although not required because Ground X automatically generates contextual data for your files  you can add extra search data to take maximum advantage of Ground X s search capabilities  help maintain document context in the search query responses  and add tags or notes indicating instructions on how to handle the search results      Example         code       python  searchData          title   Time Walk Bike to Work  2001 2011        publisher   California Department of Public Health        homepage   https   catalog data gov dataset        abstract   This table contains data on the percent of population aged 16 years or older whose commute to work is 10 or more minutes day by walking or biking for California  its regions  counties  and cities towns                 javascript  const searchData          title   Time Walk Bike to Work  2001 2011        publisher   California Department of Public Health        homepage   https   catalog data gov dataset        abstract   This table contains data on the percent of population aged 16 years or older whose commute to work is 10 or more minutes day by walking or biking for California  its regions  counties  and cities towns                         Set up environment  Set up your environment      Example         code       python  from groundx import Groundx    groundx   Groundx       api key GROUNDX API KEY                javascript  import   Groundx   from  groundx typescript sdk    import fs from  fs      const groundx   new Groundx        apiKey  GROUNDX API KEY                         info title  API Key      The  GROUNDX API KEY  placeholder represents your API key  We recommend storing your API key as an environment variable and accessing it from there  For this purpose  you can use libraries such as  dotenv  https   www npmjs com package dotenv  in Node js or  os  https   docs python org 3 library os html os environ  in Python     Go to the  GroundX dashboard  https   dashboard groundx ai auth login  to get your API key     Go to the  GroundX SDKs documentation  https   github com groundxai groundx sdks groundx sdks  for installation details               API request  Now simply make an API request to upload remote documents and include bucket ID  source URL  and extra search data in the request body      Example         code       python  ingest   groundx documents ingest remote       documents                            bucketId   bucketId                searchData   searchData                sourceUrl   ingestHosted                fileType   fileType                           print ingest body             javascript  const ingest   await groundx documents ingestRemote        documents                            bucketId  bucketId               sourceUrl  sourceUrl               searchData  searchData                           console log ingest data                      note  Go to  api Document ingestRemote  for endpoint details             API response  After making the request  you should receive a response with  processId  and  status   This response indicates that GroundX is uploading or ingesting your file into the indicated bucket       Example         json        ingest           processId    744aaf18 ff7f 459e 831c 071866dcfa2d         status    queued                       Final details  Processing time depends on the size of your files  File size can be up to ten megabytes      After automatically ingesting your files and simplifying all of its complexity for you  Ground X has prepared your content for searchability and automated response generation for your queries   

Remote Hosted Content Ingest

  X Ray  Document Understanding    Comprehensive Document Parsing for Modern Applications In this guide  we ll introduce EyeLevel s X Ray  a modern parser designed to extract high quality data from complicated real world documents  X Ray employs cutting edge parsing techniques which are specifically designed to support modern workflows like RAG  Agents  and Document Summarization  allowing developers to connect data from human centric documents to LLM powered applications     X Ray in a Nutshell You can think of X Ray as a cocktail of document understanding and advanced parsing approaches packaged together under a single API  To give you an idea  these are some of the components which X Ray employs to understand human centric documents     Bespoke document understanding models to detect key elements within documents    Advanced OCR processes which facilitate textual extraction from a variety of document representations    A repairing and reformatting pipeline that improves parse interpretability    A re contextualization system that promotes fully contextualized summarizations of parsed results   The upshot is a system which can extract complete ideas from complex documents  and represent those ideas in a way which is easy for both developers and LLMs to understand     See it for yourself X Ray s fine tuned vision model is one of the most critical components of the system  Over the last 4 years  EyeLevel has collected a comprehensive set of documents from a variety of domains which have been used to train  in our opinion  the highest quality vision model for understanding complex real world documents to date  You can use  this demo  https   dashboard eyelevel ai xray  to get an idea of how X Ray works with your documents     Medical billing receipt     imgs im1 png  Medical billing receipt    An example of X Ray identifying and extracting key elements from a real world document    Or you can get started with our APIs by following these simple steps     How to use X Ray     1  Account Setup X Ray exists as a sub component of a product called GroundX  We won t be using GroundX s core functionality in this article  but we will use GroundX to invoke X Ray and query the results  Thus  our first step is to set up a GroundX API Key   First set up an account  https   dashboard groundx ai auth register   then you can find your API key by navigating to the  API Key page  https   dashboard groundx ai apikey   GroundX has a free trial tier which you can use to experiment with X Ray   Once you re set up  install the SDK      code     python pip install groundx python sdk         typescript npm install groundx typescript sdk   save              note X Ray was added to the SDKs in versions  Python 1 3 19  and  TypeScript 1 3 24   Older versions of the SDKs do not contain X Ray support           2  Install Dependencies for this Guide  If you are using Python  you may skip this step  The TypeScript version of this guide uses third party dependencies to demonstrate the GroundX APIs   You may already have these installed on your system  If not  you will need to install the following dependencies to run the code in this guide      code     typescript npm install axios   save               3  Creating a Bucket Once you have a GroundX API key you may wish to create a bucket  Buckets can be used to organize documents into different groupings  which can be useful for certain applications  We can list all available buckets via  api Bucket list   and create a new bucket via  api Bucket create       code     python from groundx import Groundx import urllib request  json  time    authenticating groundx   Groundx      api key  XXXX       creating a new bucket bucket response   groundx buckets create    name  parsed documents bucket       storing the bucket id bucket id   bucket response body  bucket    bucketId           typescript import   Groundx   from  groundx typescript sdk   import axios from  axios   import   as fs from  fs       authenticating const groundx   new Groundx     apiKey   XXXX           creating a new bucket const bucket response   await groundx buckets create     name   parsed documents bucket           storing the bucket id const bucket id   bucket response data bucket bucketId                4  Uploading Documents Uploading documents to a GroundX bucket will automatically trigger X Ray  There are a variety of uploading options which might be useful for a variety of use cases  In this example we re uploading a document which is stored locally using  api Document ingestLocal       code     python ingest response   groundx documents ingest local       blob   open  sample pdf    rb       metadata          bucketId   bucket id       fileName    sample        fileType    pdf                  typescript const ingest response     await groundx documents ingestLocal               blob  fs readFileSync  sample pdf          metadata            bucketId  bucket id          fileName   sample           fileType   pdf                                       5  Querying Upload Status Ingesting returns a  process id   which can be used with  api Document getProcessingStatusById  to query the progress of the upload  This code checks the status of the process every 10 seconds until ingestion is done      code     python process id   ingest response body  ingest    processId    while  True       ingest   groundx documents get processing status by id          process id process id      if  ingest body  ingest    status       complete            break     if  ingest body  ingest    status       error            raise ValueError  Error Ingesting Document       print ingest body  ingest    status        time sleep 3          typescript  const process id   ingest response data ingest processId  let ingest   while  true      ingest   await groundx documents getProcessingStatusById       processId  process id          if   ingest     ingest status    ingest status    200         ingest data     ingest data ingest        console error ingest       throw Error  GroundX ingest request failed           if  ingest data ingest status      complete     ingest data ingest status      error         break         console log ingest data ingest status     await new Promise  resolve     setTimeout resolve  3000                    6  Getting X Ray Results Now that our documents are fully uploaded we can get all the documents in our bucket via  api Document lookup   We only uploaded a single document  so we can get the one and only document at index  0   and then get the URL in which the X Ray output is stored      code     python   Getting parsed documents from the bucket response   groundx documents lookup      id bucket id      Getting the X Ray parsing results for one of the documents xray url   response body  documents   0   xrayUrl   with urllib request urlopen xray url  as url      data   json loads url read   decode        print data          typescript    Getting parsed documents from the bucket const document response   await groundx documents lookup       id  bucket id         const data   await axios get document response  data  documents   0   xrayUrl          console log data  data                 7  Interpreting X Ray Results X Ray provides a rich set of results which may be useful in a variety of use cases  Here are some noteworthy outputs of X Ray      fileKeywords    A list of keywords which describe the document     fileSummary    A summary of the entire document     boundingBoxes    Key regions within the document which contain meaningful content      contentType    The type of content a certain chunk is  Textual paragraph  graphical figures  or tables      json    A reformatted representation of graphs and figures in a json format  useful for both LLM and programatic workflows      narrative    A reformatted representation of graphs and figures in a narrative format  often useful in LLM applications      sectionSummary    A contextually summarized representation of a particular section of the document   This is the full structure of an X Ray parse      json    Successful lookup response      fileType    string      One of the supported file types   as described here  https   documentation groundx ai docs file types     language    string      Language detected on the first page of your document during processing    fileKeywords    string      Auto generated comma delimited list of keywords describing your document    fileName    string      Name you gave the document when it was uploaded    fileSummary    string      Auto generated document summary    documentPages        Pages and metadata within the document              chunks        Semantic objects found on page  some semantic objects are spread across multiple pages                      boundingBoxes        Boxes containing semantic object elements                              bottomRightX   number     X coordinate for lower right corner of semantic object element                bottomRightY   number     Y coordinate for lower right corner of semantic object element                pageNumber   number     Number of page for semantic object element  starting at 1                topLeftX   number     X coordinate for upper left corner of semantic object element                topLeftY   number     Y coordinate for upper left corner of semantic object element                                       chunk   number     Unique integer ID for the semantic object            contentType        Types of elements represented within the semantic object              string      table     figure     paragraph                          json        Element text  reformatted into JSON format  for  table  and  figure  elements only              object     Auto generated JSON object describing a section of the information within the  table  or  figure                          multimodalUrl    string      Element image  for multimodal processing  table  and  figure  elements only            narrative        Element text  reformatted into narrative format  for  table  and  figure  elements only              string     Auto generated narrative description of a section of the information within the  table  or  figure                          pageNumbers        Pages where semantic object exists             number    Number of page where semantic object exists                         sectionSummary    string      Auto generated section summary for the document section containing the semantic object            suggestedText    string      Element text  reformatted for LLM completion            text    string     Element text  extracted and unprocessed                           height   number     Pixel height of page image        pageNumber   number     Number of page  starting at 1        pageUrl    string      Hosted URL for the page image        width   number    Pixel width of page image               sourceUrl    string      Hosted URL for your document      

X-Ray: Document Understanding

  Search   iframe width  560  height  315  src  https   www youtube com embed gqNO5zGXU1w si CXtwQ3ziOuTnBP7P  title  YouTube video player  frameborder  0  allow  accelerometer  autoplay  clipboard write  encrypted media  gyroscope  picture in picture  web share  allowfullscreen   iframe        Introduction  In this tutorial  we ll explore the GroundX search API     What makes GroundX search results stand out       Under the hood  the GroundX Search API goes through the following process         First  it analyzes your search query  and improves it if necessary  to carry out a semantic search that helps avoid the hallucination trap most RAG apps fall into with their vectorized searches       Next  it searches through GroundX buckets where your content and its extra search data are stored     GroundX s unique extra search data approach helps maintain your content within its original context   After finding relevant content  the Search API returns much more than just simple  unformatted chunks of raw text like most RAG systems  Instead  GroundX returns a search response bundled with intelligible text chunks along with extra search data  automatically generated documentation and section summaries  source URL  and a new  more readable  contextualized  and performant version of the text chunk ready for LLM use       Then  GroundX uses its proprietary re ranker that scores every chunk for how well it answers the original question to ensure that the most trustworthy results are always on top       And finally  it merges all of this into a simple text block that you can send to the LLM of your choice  so you get accurate  contextualized  and hallucination free responses from your LLMs when working with your content        LLM integration   Sounds complex    Luckily GroundX does all the hard stuff for you in the background  you just have to follow these simple steps     1  Make an API search request      Example         code       python  queryString    How many workers walk to work in the Bay Area      result   groundx search content       id 6839        n 10       query queryString        llmText   result body  search    text                javascript  const queryString    How many workers walk to work in the Bay Area      const result   await groundx search content        id  6839       n  10       query  queryString            const llmText   result data search text             info title  More information      The  search API request   getting started  is explained further below            1  Retrieve the  search text  property and pass it on to the LLM of your choice      Example         code       python  from openai import OpenAI  client   OpenAI       api key   OPENAI API KEY        completion   openai chat completions create     model  gpt 3 5 turbo      messages                    role    system              content      Get response from this data         s                     llmText                            role    user              content   queryString                            javascript  import OpenAI from  openai      const openai   new OpenAI       api key  OPENAI API KEY         const chatCompletion   await openai chat completions create      model   gpt 3 5 turbo      messages                   role    system           content    Respond only with                              llmText                                                      role    user               content   queryString                                      info title  LLM API      Make sure to get the API key  endpoints  and SDK from your LLM provider  For example  if using ChatGPT  go to the  OpenAI documentation  https   platform openai com docs overview            3  Get a response from the LLM using your retrieved data      Example using          code       python  print completion choices 0  message content             javascript  console log chatCompletion choices 0  message content                    The end result     retrievals that outperform traditional vector systems and boost the accuracy of LLM completions         Getting started  Let s go into the details     To make an API request you ll need to do the following         Ingest content  First  make sure you ve already uploaded to a GroundX bucket the content you want to search through        note    Go to either  Local Content Ingest  https   documentation groundx ai docs ingest local content  or  Remote Content Ingest  https   documentation groundx ai docs ingest remote content  for details on how to add your content to a GroundX bucket                Set up environment  Next  set up your environment with the GroundX SDK        code       python  from groundx import Groundx    groundx   Groundx       api key   GROUNDX API KEY                javascript  import   Groundx   from  groundx typescript sdk      const groundx   new Groundx        apiKey  GROUNDX API KEY                          Bucket or project ID  Get the ID of the project or bucket you want to search through      Example         code       python  groundxId   3839            javascript  const id   3839                   note    Go to the documentation for information on how to obtain a project or bucket ID with the corresponding endpoint      api Bucket list      api Project list               Search query string  Create a search query string      The string can be a question or the keywords you want to search for     If query strings are more than 30 words long  GroundX automatically rewrites the string with keywords using its internal LLM     GroundX processes the string to search through your content and its extra search data to retrieve the most relevant results      Example         code       python  queryString    How many workers walk to work in the Bay Area              javascript  const queryString    How many workers walk to work in the Bay Area                     Number of results  Optionally  you can also set the number of results that are returned by the search request  By default  search queries return up to 20 results      Example         code       python  n   10            javascript  const n   10                     API request  You re now ready to make the API search request   Include the ID and search query string in the request body      Example         code       python  response   groundx search content       id groundxId        query query       n n        print response body  search    text               javascript  const response   await groundx search content        id  id       query  queryString      n  n         console log response data search text                    API response  After making the request  you will receive a Search object as the response       Let s go over some of the details of the Search schema        Response sample         json          count   10        query    How many workers walk to work in the Bay Area          results                                bucketId   6839                documentId    81111b9e ff13 41ef a89a af667a4b1cac                 score   357 09656                searchData                       documentSummary    This document is a detailed report from the Healthy Communities Data and Indicators Project by UCSF and CDPH                        sectionSummary      Given a row from the provided TSV table  this row represents data about the commuting habits of workers in a specific region in California over two time periods  2005 2007 and 2008                        fullTitle    Percent of population aged 16 years or older whose commute to work is 10 or more minutes day by walking or biking                     publisher    California Department of Public Health                                 sourceUrl    https   upload groundx ai file hciwalkbicycle778narrativeandexamples3 26 14 pdf                 suggestedText    In the Bay Area between 2005 2007  70 287 out of 3 122 743 workers  or 2 25   walked for 10 or more minutes per day  This number increased to 81 737 out of 3 202 761 workers  or 2 55   in the 2008 2010 period  In Butte  1 266 out of 85 042 workers  or 1 49   walked for 10 or more minutes per day between 2005 2007  This increased to 1 678                    text     t  t t  t2005 2007 t  t t  t t  t2008 2010 n t  tWorkers who t  t t  t t  tWorkers who n t  twalk  10 t  t t  t t  twalk  10 nRegion name t  tmin   day t  tTotal                                  score   369 19244                text    fullTitle  Percent of population aged 16 years or older whose commute to work is 10 or more minutes day by walking or biking npublisher  California Department of Public Health nIn the Bay Area between 2005 2007  70 287 out of 3 122 743 workers  or 2 25   walked for 10 or more minutes per day  This number increased to 81 737 out of 3 202 761 workers  or 2 55   in the 2008 2010 period  In Butte  1 266 out of 85 042 workers  or 1 49   walked for 10 or more minutes per day between 2005 2007  This increased to 1 678 out of 81 321 workers  or 2 06   in 2008 2010                    Recommended data  The most significant property returned by the search response is the  text  property  which contains a compilation of suggested texts with their corresponding extra search data   In other words  it s a string that includes all the automatically rewritten text chunks and the extra search data that was manually added to the original content     As already mentioned  this content that is automatically generated by GroundX s internal LLM provides you with intelligible  contextualized  and machine understandable content that you can straightforwardly pass on to your LLM to work with         Advanced data handling  Although we recommend passing the  search text  property directly to the LLM you re working with for response generation  you can also add your own logic to handle the  search results  property         Let s take a closer look       The  results  property contains a list of all the original text chunks that matched the search query  Chunks are ordered by a score based on semantic search and re ranking methodologies    Here  you can also find the suggested text we ve been mentioning  which is the more intelligible and performant version of the text chunk     Furthermore  you can find automatically generated extra search data  like summaries of the document and section the text chunks are found in     The extra search data added to your content is available as well   And to help keep source information available  the GroundX URL  document ID  and bucket ID of the content where the text chunks were extracted from are also provided        Final details  You re now ready to integrate GroundX with your LLM to generate accurate  hallucination free responses using data from your own content   

Search

  Supported Document File Types    The currently supported file types are       pdf    jpg  or jpeg     png    docx    pptx    xlsx    csv    tsv    json    txt       Additional data sources    The GroundX ingestion pipeline can also crawl and ingest the content from websites using the  api Document crawlWebsite  endpoint     Note  the crawler scrapes the page content from the source HTML and can sometimes be confused by the structure of the page 

Supported Document File Types

  Quick Start   iframe width  560  height  315  src  https   www youtube com embed wH0DR7U0l3c si yPkV9ub3OdWx0ZlG  title  YouTube video player  frameborder  0  allow  accelerometer  autoplay  clipboard write  encrypted media  gyroscope  picture in picture  web share  allowfullscreen   iframe     This Quick Start guide is intended to show you how to upload documents and search them using the GroundX APIs  The code within this QuickStart guide can be downloaded from the  code samples GitHub repository  https   github com groundxai code samples  in  Python  https   github com groundxai code samples tree master python getting started  and  TypeScript  https   github com groundxai code samples tree master typescript getting started         Step 1  Download GroundX SDK  optional     This step is optional  Do this step only if you want to use our  TypeScript  https   github com groundxai groundx sdks blob main sdks typescript README md   for javascript and NodeJS projects  or  Python  https   github com groundxai groundx sdks blob main sdks python README md  SDKs     Use the following shell commands to install        code       python  pip install groundx python sdk            typescript  npm install groundx typescript sdk   save                   Step 2  Register and Get Your API Key    Before you can use our APIs  you will need to  create an account  https   dashboard groundx ai auth register  here     Log into the  GroundX Dashboard  https   dashboard groundx ai  and navigate to API Keys       Navigate to API Keys     imgs gs1 webp  Navigate to API Keys      Copy your API Key and save it somewhere for use later in this tutorial       Copy Your API Key     imgs gs2 webp  Copy Your API Key         Step 3  Initialize Your SDK  optional     If you re using one of our SDKs  you will use your API key to initialize your client        code       python  from groundx import Groundx  ApiException    groundx   Groundx     api key groundxKey                typescript  import fs from  fs    import   Groundx   from  groundx typescript sdk      const groundx   new Groundx      apiKey  groundxKey                      Replace  groundxKey  with your GroundX API key        Step 4  Make Your First Request    All of our API requests require your API Key in the header  Here is an example of how to include it        code       bash  curl https   api groundx ai api v1 bucket         H  X API KEY   your api key              python  try     bucket response   groundx buckets list        bucketId   bucket response body  buckets   0   bucketId              typescript  const bucketResponse   await groundx buckets list       bucketId   bucketResponse data buckets 0  bucketId                 For bash requests  replace  your api key  with your actual API key     If your request is successful  you will receive a response that looks something like this        json        buckets                    bucketId    unique system generated id           fileCount    number of documents in bucket           fileSize     files size total            name     your bucket name                           This request will return a list of your content buckets where you can upload your documents  Make note of the  bucketId  for use later in this tutorial        Step 5  Upload Content    To upload document content  use the  Document Upload API   reference Documents Document ingestRemote         code       bash  curl https   api groundx ai api v1 ingest documents          X POST          H  X API KEY   your api key            d                 documents                                      bucketId   bucketId                   fileType   fileType                   sourceUrl   ingestHosted                                                       python    Upload local documents to GroundX    ingest   groundx documents ingest local     body                  blob   open ingestLocal   rb            metadata               bucketId   bucketId            fileName   fileName            fileType   fileType                                   Upload hosted documents to GroundX    ingest   groundx documents ingest remote     documents                  bucketId   bucketId          sourceUrl   ingestHosted          fileType   fileType                             typescript     Upload local documents to GroundX    let ingest   await groundx documents ingestLocal             blob  fs readFileSync ingestLocal        metadata           bucketId  bucketId         fileName  fileName         fileType  fileType                            Upload hosted documents to GroundX    let ingest   await groundx documents ingestRemote      documents                  bucketId  bucketId         fileType  fileType         sourceUrl  ingestHosted                                  Replace  ingestLocal  with the path to your local file   ingestHosted  to the URL of your hosted file   fileType  with one of the enumerated file types  e g  txt  pdf    fileName  with a name for your file  and  bucketId  with the content bucket you would like to add the document to     If your request is successful  will receive a response that looks something like this        json        ingest           processId     unique system generated id         status     enumerated status                   Make note of  processId   You will need it in the next step of this Quick Start guide        Step 6  Check the Status of Your Upload    Make the following request to query the status of your document upload as it is processed through the GroundX ingestion pipeline        code       bash  curl https   api groundx ai api v1 ingest  processId          H  X API KEY   your api key            H  Content Type  application json             python  ingest   groundx documents get processing status by id     process id processId               typescript  ingest   await groundx documents getProcessingStatusById      processId  processId                      Replace  processId  the processId from the previous step     If your request is successful  will receive a response that looks something like this        json        ingest           processId     unique system generated id          progress             complete               documents                                documentId     unique system generated id                  fileName     given file name                  fileSize     files size total                  fileType     file type                  bucketId    your bucket id                 processId     unique system generated id                  sourceUrl     document url                  status     enumerated status                                      total   1                        status     enumerated status                   The value of  status  will be one of  queued    processing    error    or  complete         Step 7  Search Your Content    Make the following request to search your ingested content        code       bash  curl https   api groundx ai api v1 search  id          X POST          H  X API KEY   your api key            H  Content Type  application json           d    query  query              python  content response   groundx search content id bucketId  query query             typescript  const searchResponse   await groundx search content      id  bucketId     query  query                      You can also use  projectId  or  groupId  in place of  bucketId  in your search query  Any of the 3 values will work for search queries  Replace  query  with the query you want to use to search your content     If your request is successful  will receive a response that looks something like this        json        search           count    int number of results         query     your query         score    float highest relevance score in results         text     combined text of search results          nextToken     token for next set of results          results                       documentId     unique system generated id              score    float relevance score of result             searchData                   document metadata                        sourceUrl     source document url              suggestedText     rewritten text for LLM completions              text      original text of result                                   We strongly recommend you use  search text  for your LLM completions  We provide  search results  in case you want to create your own context from the search results  If you choose to do this  rather than use  search text   we strongly recommend you use  search results n  suggestedText  for your context     If you need to look up a  projectId    groupId   or  bucketId   you can find them in the  GroundX Dashboard  https   dashboard groundx ai  or by querying for them using the APIs     Make the following request to query for your projects        code       bash  curl https   api groundx ai api v1 project          H  X API KEY   your api key              python  project response   groundx project list      projectId   project response body  projects   0   projectId              typescript  const projectResponse   await groundx project list       projectId   projResponse data projects 0  projectId                 If your request is successful  will receive a response that looks something like this        json        projects                    buckets                            bucketId    unique system generated id               fileCount    number of documents in bucket               fileSize     files size total                name     your bucket name                                created     project created date time            fileCount    number of documents in project           fileSize     files size total            fileCount    number of documents in project           name     your project name            projectId    unique system generated id           updated     project updated date time                             Next Steps    Now that you ve successfully used the GroundX APIs to upload and search content  you can dive into our  tutorials   docs chatgpt  and  documentation   reference      Remember  our APIs are still in closed beta  so we appreciate your patience and feedback as we continue to improve our service 

Quick Start

  Integrating with ChatGPT  Welcome to this hands on tutorial where we ll delve into Retrieval Augmented Generation  RAG  by merging the capabilities of GroundX s search results with OpenAI s ChatGPT   In this tutorial  we will use Python to show you how to seamlessly extract relevant content from GroundX and then channel it into ChatGPT for enhanced  context aware conversational experiences  This will not only broaden the knowledge base of the ChatGPT model but also leverage the precise search and retrieval capabilities of GroundX   By the end of this tutorial  you ll possess the skills to create a more informed and enriched conversational agent that can draw upon your private data  Let s get started      note You can download the code for this tutorial from the GroundX code sample repository in  TypeScript  https   github com groundxai code samples blob master typescript chatGPT completions demo js   for javascript and NodeJS projects  or  Python  https   github com groundxai code samples blob master python chatGPT completions demo py           Step 1  Set Up Your Environment  You will need a   GroundX API Key    which you can find in your  GroundX account  https   dashboard groundx ai apikey    You will also need a  projectId    groupId   or  bucketId  for a project  group  or bucket that contains ingested content  This can also be found in the  GroundX dashboard  https   dashboard groundx ai projects    For this tutorial  you will also need an   OpenAI API Key    which can be found in your  OpenAI account  https   platform openai com account api keys    Finally  you will need to install the OpenAI and GroundX SDKs  either in  TypeScript  https   github com groundxai groundx sdks blob main sdks typescript README md  or  Python  https   github com groundxai groundx sdks blob main sdks python README md   using the package manager of your choice  Here are examples using  pip  and  npm       code     python pip install groundx python sdk openai         typescript npm install groundx typescript sdk openai   save              Step 2  Initialize Your Clients  Initialize both OpenAI and GroundX clients with the appropriate API keys      code     python groundx   Groundx    api key groundxKey     openai api key   openaiKey         typescript const groundx   new Groundx     apiKey  groundxKey       const openai   new OpenAI     apiKey  openaiKey                Replace  groundxKey  with your GroundX API key and  openaiKey  with your OpenAI key      Step 3  Search Your Content on GroundX  Next  you will submit a query to the GroundX Search API   A request will look something like this      code     python content response   groundx search content id groundxId  query query   results   content response body  search           typescript const result   await groundx search content     id  groundxId    query  query               Replace  groundxId  with your  projectId    groupId   or  bucketId  and  query  with the query you want to use to search your content   Your search results should look something like this      json      search          count    int number of results        query     your query        score    float highest relevance score in results        text     combined text of search results         nextToken     token for next set of results         results                     documentId     unique system generated id             score    float relevance score of result            sourceUrl     source document url             text      text of result                               Step 4  Curate the Search Results  Once you have the results from your GroundX search  you will want to curate the relevant content to send to OpenAI for completion  You will need to be cognizant of the token limits for OpenAI models and reserve space for the completion  since the token limits apply to the combined total of tokens used for the request and completion   Here is an example of what that could look like      code     python llmText   results  text           typescript let llmText   result data search text            Use the  search text  field from the response  It contains the suggested context for LLM completion  If  search text  is too long for your model context limits  either truncate  search text  or iterate  search results  and use the  suggestedText  field of each search result   Reference the OpenAI documentation for the most current model token limits  Keep in mind that a token is roughly equivalent to 3 5 characters and that token limits apply to the combined size of the request and completion response      Step 5  Submit a Completion Request to OpenAI  Once you have relevant text for your request  you will need to combine the text with instructions and submit them to OpenAI   Here is an example of what a completion instruction could look like      txt You are a helpful virtual assistant that answers questions using the content below  Your task is to create detailed answers to the questions by combining your understanding of the world with the content provided below  Do not share links       Combine your completion instructions with your curated GroundX search results      code     python completion   openai ChatCompletion create    model openaiModel    messages                role    system          content       s      s                  instruction  llmText                role    user    content   query                  typescript const completion   await openai chat completions create     model  openaiModel    messages                 role    system          content      instruction        llmText                     role    user    content   query                      Replace  openaiModel  with your preferred model   instruction  with your completion instructions   llmText  with your curated GroundX search results  and  query  with your query   Print the results of your request and you re done      note You can download the code for this tutorial from the GroundX code sample repository in  TypeScript  https   github com groundxai code samples blob master typescript chatGPT completions demo js   for javascript and NodeJS projects  or  Python  https   github com groundxai code samples blob master python chatGPT completions demo py       

Integrating with ChatGPT

  Upload Documents from a Remote Location to GroundX  This tutorial will show you how to use GroundX s Typescript and Python SDK libraries to  upload hosted documents  https   documentation groundx ai reference Documents Document ingestRemote  to your GroundX  buckets  https   documentation groundx ai docs concepts concepts buckets    Through a simple API request you can  effortlessly upload your content to GroundX  https   documentation groundx ai docs welcome welcome effortless content upload  and  automatically pre process  https   documentation groundx ai docs welcome welcome sophisticated automated pre processing  your data to get it ready to be  searched  https   documentation groundx ai docs welcome welcome superior search capabilities  through        Prerequisites   Node js installed  for Javascript or Typescript projects    Python 3 7 or higher installed  for Python projects      info title  Code samples   You can download the code for this tutorial from the GroundX code sample repository in  Typescript  https   github com groundxai code samples tree master typescript upload remote   for Javascript and Node js projects  or  Python  https   github com groundxai code samples tree master python upload remote           Step 1  Set up your environment If you haven t already done so  follow the steps below to get your GroundX API key and install the GroundX SDK for your project   1  To get your GroundX API key  log in to your GroundX dashboard and go to the  API Keys  https   dashboard groundx ai apikey  section      note If you don t have a GroundX account  you can sign up for a free account at  https   www groundx ai  https   www groundx ai        2  Install the GroundX SDK for either  Typescript  https   github com groundxai groundx sdks blob main sdks typescript README md  or  Python  https   github com groundxai groundx sdks tree main sdks python  with the following commands      code     python pip install groundx python sdk         typescript npm install groundx typescript sdk   save              Step 2  Import required libraries In your project  import the GroundX SDK library      code     python from groundx import Groundx  ApiException         typescript import   Groundx   from  groundx typescript sdk                Step 3  Set up your API key Set up your API key by creating a new GroundX object and passing your API key as a parameter      code     python groundxKey    YOUR GROUNDX KEY          typescript const groundxKey    YOUR GROUNDX KEY                warn It is recommended that you store your API key in an environment variable and access it from there  For example  using the  dotenv  https   www npmjs com package dotenv  library in Node js or the  os  https   docs python org 3 library os html os environ  library in Python          Step 4  Set up content ingestion parameters Set up the parameters for the content ingestion request  For more information on the parameters for uploading hosted documents to GroundX  go to the  api Document ingestRemote  reference guide   1  Indicate the ID of the  bucket  https   documentation groundx ai docs concepts concepts buckets  you want to ingest the content into by setting the  bucket  parameter      code     python bucketID   0         typescript let bucketId   0               note For simplicity  in this tutorial we ll set  bucketID  to  0  to ingest the content into a default bucket  See  Step 7   step 7 get default bucket id         1  Set a variable to indicate the type of content you want to ingest  Currently  the supported file types are    txt   docx   pptx   xlsx   pdf   png   jpg  For example      code         python fileType     FILE TYPE           typescript const fileType     FILE TYPE              3  Set a variable to indicate the URL of the content you want to ingest  For example      code     python ingestHosted     URL           typescript const ingestHosted     URL              4  Optional  Include an object containing metadata for your content  For example      code     python contentMetadata          title    Sample Title        description    Sample Description        author    Sample Author        tags     Sample Tag 1    Sample Tag 2             typescript const contentMetadata         title   Sample Title       description   Sample Description       author   Sample Author       tags    Sample Tag 1    Sample Tag 2                    info title  Metadata   Although metadata is automatically extracted from the content during the ingestion process  you can also include your own metadata   When GroundX carries out search querires  it searches through not only your content but also the associated metadata  This helps provide more accurate search results and returns document chunks with the corresponding metadata so that you don t lose the context the metadata provides           Step 5  Set parameter validation Optional  Set up parameter validation to check if all the required parameters are set  For example      code     python if groundxKey            raise Exception  set your GroundX key    if ingestHosted            raise Exception  set the hosted file URL    if fileType            raise Exception  set the file type to a supported enumerated type  e g  txt  pdf            typescript if  groundxKey      YOUR GROUNDX KEY         throw Error  set your GroundX key       if  ingestHosted               throw Error  set the hosted file URL       if  fileType               throw Error  set the file type to a supported enumerated type  e g  txt  pdf                     Step 6  Initialize the GroundX client Initialize the GroundX client by creating a new GroundX object and passing your API key as a parameter  For example      code     python groundx   Groundx      api key groundxKey            typescript const groundx   new Groundx     apiKey  groundxKey                   Step 7  Get default bucket ID Before uploading the content  we ll set the default bucket ID  Since we set the bucket ID to  0  in  Step 4 1   step 4 set up content ingestion parameters   we ll now call the  api Bucket list  endpoint to check if any buckets exist and get the ID of the first bucket in the list  For example      code     python if bucketId    0        list buckets request     try          bucket response   groundx buckets list            if len bucket response body  buckets      1              print bucket response body  buckets                raise Exception  no results from buckets            bucketId   bucket response body  buckets   0   bucketId       except ApiException as e          print  Exception when calling BucketApi list   s n    e          typescript    Note  Insert this code within a function   if  bucketId     0             List buckets request       const bucketResponse   await groundx buckets list          if   bucketResponse     bucketResponse status    bucketResponse status    200               bucketResponse data     bucketResponse data buckets            console error bucketResponse           throw Error  GroundX bucket request failed                       if  bucketResponse data buckets length   1            console error  no results from buckets            console log bucketResponse data buckets           throw Error  no results from GroundX bucket query                  console log bucketResponse data         bucketId   bucketResponse data buckets 0  bucketId                     Step 8  Upload the content Upload the content by calling the  api Document ingestRemote  endpoint with the parameters you set in  Step 4   step 4 set up content ingestion parameters  as arguments  For example      code     python   Upload hosted documents to GroundX request try      ingest   groundx documents ingest remote          documents                                  bucketId   bucketId                   metadata   contentMetadata                   sourceUrl   ingestHosted                   fileType   fileType                                         typescript    Note  Insert this code within a function      Upload hosted documents to GroundX     let ingest   await groundx documents ingestRemote         documents                        bucketId  bucketId            type  fileType            metadata  contentMetadata            sourceUrl  ingestHosted                                      The  api Document ingestRemote  endpoint returns a response object indicating the status of the ingestion process    For example      json    Successful request response      ingest          processId    string      Object ID of the ingest process      status    string      queued     processing     error     complete                 Step 9  Get ingest status To check the status of the ingestion process  we ll use the request response and the  api Document getProcessingStatusById  endpoint  For example      code     python   Insert this code after the Try block in Step 8  while           ingest body  ingest    status       complete          and ingest body  ingest    status       error                 ingest   groundx documents get processing status by id              process id ingest body  ingest    processId             except ApiException as e      print  Exception when calling DocumentApi ingest remote   s n    e          typescript    Note  Insert this code within a function   if   ingest     ingest status    ingest status    200           ingest data     ingest data ingest          console error ingest         throw Error  GroundX ingest request failed                      poll ingest status     while  ingest data ingest status      complete     ingest data ingest status      error           ingest   await groundx documents getProcessingStatusById           processId  ingest data ingest processId                  if   ingest     ingest status    ingest status    200             ingest data     ingest data ingest            console error ingest           throw Error  GroundX ingest request failed                       await new Promise  resolve     setTimeout resolve  3000                       Step 10  Test your code 1  After you have adjustmented the code accordingly  run your code to upload the content to GroundX  2  Call the  api Document list  endpoint to from GroundX s interactive  API Reference guide  https   documentation groundx ai reference Documents Document list  to get a list of all the documents in your GroundX buckets  3  Check if the content you uploaded is listed in the response     Congratulations       You ve successfully ingested a hosted document to GroundX that you can now search through using GroundX s  search API  https   documentation groundx ai reference Search Search content   

Upload Documents from a Remote Location to GroundX

  Upload Local Documents to GroundX  This tutorial will show you how to use GroundX s Typescript and Python SDK libraries to  upload local documents  https   documentation groundx ai reference Documents Document ingestLocal  to your GroundX  buckets  https   documentation groundx ai docs concepts concepts buckets    Through a simple API request you can  effortlessly upload your content to GroundX  https   documentation groundx ai docs welcome welcome effortless content upload  and  automatically pre process  https   documentation groundx ai docs welcome welcome sophisticated automated pre processing  your data to get it ready to be  searched  https   documentation groundx ai docs welcome welcome superior search capabilities  through        Prerequisites   Node js installed  for Javascript or Typescript projects    Python 3 7 or higher installed  for Python projects      note You can download the code for this tutorial from the GroundX code sample repository in  Typescript  https   github com groundxai code samples blob master typescript upload local demo js   for Javascript and Node js projects  or  Python  https   github com groundxai code samples blob master python upload local demo py           Step 1  Set up your environment If you haven t already done so  follow the steps below to get your GroundX API key and install the GroundX SDK for your project   1  To get your GroundX API key  log in to your GroundX dashboard and go to the  API Keys  https   dashboard groundx ai apikey  section      note If you don t have a GroundX account  you can sign up for a free account at  https   www groundx ai  https   www groundx ai        2  Install the GroundX SDK for either  Typescript  https   github com groundxai groundx sdks blob main sdks typescript README md  or  Python  https   github com groundxai groundx sdks tree main sdks python  with the following commands      code     python pip install groundx python sdk         typescript npm install groundx typescript sdk   save              Step 2  Import required libraries In your project  import the GroundX SDK library      code     python from groundx import Groundx  ApiException         typescript import   Groundx   from  groundx typescript sdk                Step 3  Set up your API key Set up your API key by creating a new GroundX object and passing your API key as a parameter      code     python groundxKey    YOUR GROUNDX KEY          typescript const groundxKey    YOUR GROUNDX KEY                warn It is recommended that you store your API key in an environment variable and access it from there  For example  using the  dotenv  https   www npmjs com package dotenv  library in Node js or the  os  https   docs python org 3 library os html os environ  library in Python          Step 4  Set up content ingestion parameters Set up the parameters for the content ingestion request  For more information on the parameters for uploading local documents to GroundX  go to the  api Document ingestLocal  reference guide   1  Indicate the ID of the  bucket  https   documentation groundx ai docs concepts concepts buckets  you want to ingest the content into by setting the  bucket  parameter      code     python bucketID   0         typescript let bucketId   0               note For simplicity  in this tutorial we ll set  bucketID  to  0  to ingest the content into a default bucket  See  Step 7   step 7 get default bucket id         1  Set a variable to indicate the type of content you want to ingest  Currently  the supported file types are    txt   docx   pptx   xlsx   pdf   png   jpg  For example      code         python fileType     FILE TYPE           typescript const fileType     FILE TYPE              3  Set a variable to indicate the relative path of the local content you want to ingest  For example      code     python ingestLocal     RELATIVE LOCAL PATH           typescript const ingestLocal     RELATIVE LOCAL PATH              4  Optional  Include an object containing metadata for your content  For example      code     python contentMetadata          title    Sample Title        description    Sample Description        author    Sample Author        tags     Sample Tag 1    Sample Tag 2             typescript const contentMetadata         title   Sample Title       description   Sample Description       author   Sample Author       tags    Sample Tag 1    Sample Tag 2                    info title  Metadata   Although metadata is automatically extracted from the content during the ingestion process  you can also include your own metadata   When GroundX carries out search querires  it searches through not only your content but also the associated metadata  This helps provide more accurate search results and returns document chunks with the corresponding metadata so that you don t lose the context the metadata provides           Step 5  Set parameter validation Optional  Set up parameter validation to check if all the required parameters are set  For example      code     python if groundxKey     YOUR GROUNDX KEY       raise Exception  set your GroundX key    if ingestLocal            raise Exception  set the local file path    if fileType            raise Exception  set the file type to a supported enumerated type  e g  txt  pdf     if fileName            raise Exception  set a name for the file           typescript if  groundxKey      YOUR GROUNDX KEY       throw Error  set your GroundX key       if  ingestLocal             throw Error  set the local file path      if  fileType             throw Error  set the file type to a supported enumerated type  e g  txt  pdf       if  fileName             throw Error  set a name for the file                   Step 6  Initialize the GroundX client Initialize the GroundX client by creating a new GroundX object and passing your API key as a parameter  For example      code     python groundx   Groundx      api key groundxKey            typescript const groundx   new Groundx     apiKey  groundxKey                   Step 7  Get default bucket ID Before uploading the content  we ll set the default bucket ID  Since we set the bucket ID to  0  in  Step 4 1   step 4 set up content ingestion parameters   we ll now call the  api Bucket list  endpoint to check if any buckets exist and get the ID of the first bucket in the list  For example      code     python if bucketId    0        list buckets     try          bucket response   groundx buckets list            if len bucket response body  buckets      1              print bucket response body  buckets                raise Exception  no results from buckets            bucketId   bucket response body  buckets   0   bucketId       except ApiException as e          print  Exception when calling BucketApi list   s n    e          typescript    Note  Insert this code within a function   if  bucketId     0             list buckets       const bucketResponse   await groundx buckets list          if   bucketResponse     bucketResponse status    bucketResponse status    200               bucketResponse data     bucketResponse data buckets            console error bucketResponse           throw Error  GroundX bucket request failed                       if  bucketResponse data buckets length   1            console error  no results from buckets            console log bucketResponse data buckets           throw Error  no results from GroundX bucket query                       bucketId   bucketResponse data buckets 0  bucketId                     Step 8  Upload the content Upload the content by calling the  api Document ingestLocal  endpoint with the parameters you set in  Step 4   step 4 set up content ingestion parameters  as arguments  For example      code     python   upload local documents to GroundX try      ingest   groundx documents ingest local          body                                  blob   open ingestLocal   rb                     metadata                          bucketId   bucketId                       fileName   fileName                       fileType   fileType                        optional metadata field                       content is added to document chunks                       fields are search during search requests                       and returned in search results                      metadata   contentMetadata                                                            typescript    Note  Insert this code within a function      ingest local documents to GroundX     let ingest   await groundx documents ingestLocal                   blob  fs readFileSync ingestLocal           metadata              bucketId  bucketId            fileName  fileName            fileType  fileType            metadata  contentMetadata                                       The  api Document ingestLocal  endpoint returns a response object indicating the status of the ingestion process    For example      json    Successful request response      ingest          processId    string      Object ID of the ingest process      status    string      queued     processing     error     complete                 Step 9  Get ingest status To check the status of the ingestion process  we ll use the request response and the  api Document getProcessingStatusById  endpoint  For example      code     python   Insert this code after the Try block in Step 8   while           ingest body  ingest    status       complete          and ingest body  ingest    status       error                 ingest   groundx documents get processing status by id              process id ingest body  ingest    processId             except ApiException as e      print  Exception when calling DocumentApi ingest local   s n    e          typescript    Note  Insert this code within a function       if   ingest     ingest status    ingest status    200           ingest data     ingest data ingest          console error ingest         throw Error  GroundX ingest request failed                      poll ingest status     while  ingest data ingest status      complete     ingest data ingest status      error           ingest   await groundx documents getProcessingStatusById           processId  ingest data ingest processId                  if   ingest     ingest status    ingest status    200             ingest data     ingest data ingest            console error ingest           throw Error  GroundX ingest request failed                       await new Promise  resolve     setTimeout resolve  3000                       Step 10  Test your code 1  After you have adjustmented the code accordingly  run your code to upload the content to GroundX  2  Call the  api Document list  endpoint to from GroundX s interactive  API Reference guide  https   documentation groundx ai reference Documents Document list  to get a list of all the documents in your GroundX buckets  3  Check if the content you uploaded is listed in the response     And that s it       You ve successfully ingested a local document to GroundX that you can now search through using GroundX s  search API  https   documentation groundx ai reference Search Search content   

Upload Local Documents to GroundX

Ingest documents hosted on public URLs to a GroundX bucket. 

Interact with the "Request Body" below to explore the arguments of this function. Enter your GroundX API key to send a request directly from this web page. Select your language of choice to structure a code snippet based on your specified arguments.


ingest_remote

Upload documents hosted on a local file system for ingestion into a GroundX bucket.

Interact with the "Request Body" below to explore the arguments of this function. Enter your GroundX API key to send a request directly from this web page. Select your language of choice to structure a code snippet based on your specified arguments.


ingest_local

Upload the content of a publicly accessible website for ingestion into a GroundX bucket. This is done by following links within a specified URL, recursively, up to a specified depth or number of pages.

Interact with the "Request Body" below to explore the arguments of this function. Enter your GroundX API key to send a request directly from this web page. Select your language of choice to structure a code snippet based on your specified arguments.


crawl_website

Get the current status of an ingest, initiated with documents.ingest_remote, documents.ingest_local, or documents.crawl_website, by specifying the processId (the processId is included in the response of the documents.ingest functions).

Interact with the "Request Body" below to explore the arguments of this function. Enter your GroundX API key to send a request directly from this web page. Select your language of choice to structure a code snippet based on your specified arguments.


get_processing_status_by_id

lookup the document(s) associated with a processId, bucketId, groupId, or projectId.

Interact with the "Request Body" below to explore the arguments of this function. Enter your GroundX API key to send a request directly from this web page. Select your language of choice to structure a code snippet based on your specified arguments.


lookup

Delete multiple documents hosted on GroundX

Interact with the "Request Body" below to explore the arguments of this function. Enter your GroundX API key to send a request directly from this web page. Select your language of choice to structure a code snippet based on your specified arguments.


delete (multiple)

lookup all documents across all resources which are currently on GroundX

Interact with the "Request Body" below to explore the arguments of this function. Enter your GroundX API key to send a request directly from this web page. Select your language of choice to structure a code snippet based on your specified arguments.


list

Delete a single document hosted on GroundX

Interact with the "Request Body" below to explore the arguments of this function. Enter your GroundX API key to send a request directly from this web page. Select your language of choice to structure a code snippet based on your specified arguments.


delete (singular)

Look up an existing document by documentId.

Interact with the "Request Body" below to explore the arguments of this function. Enter your GroundX API key to send a request directly from this web page. Select your language of choice to structure a code snippet based on your specified arguments.


Search documents on GroundX for the most relevant information to a given query.

The result of this query is typically used in one of two ways; result['search']['text'] can be used to provide context to a language model, facilitating RAG, or result['search']['results'] can be used to observe chunks of text which are relevant to the query, facilitating citation.

Interact with the "Request Body" below to explore the arguments of this function. Enter your GroundX API key to send a request directly from this web page. Select your language of choice to structure a code snippet based on your specified arguments.


search.content

Search documents on GroundX for the most relevant information to a given query by documentId(s).

The result of this query is typically used in one of two ways; result['search']['text'] can be used to provide context to a language model, facilitating RAG, or result['search']['results'] can be used to observe chunks of text which are relevant to the query, facilitating citation.

Interact with the "Request Body" below to explore the arguments of this function. Enter your GroundX API key to send a request directly from this web page. Select your language of choice to structure a code snippet based on your specified arguments.


search.documents

List all buckets within your GroundX account

Interact with the "Request Body" below to explore the arguments of this function. Enter your GroundX API key to send a request directly from this web page. Select your language of choice to structure a code snippet based on your specified arguments.


Create a new bucket.

Interact with the "Request Body" below to explore the arguments of this function. Enter your GroundX API key to send a request directly from this web page. Select your language of choice to structure a code snippet based on your specified arguments.


create

Delete a bucket.

Interact with the "Request Body" below to explore the arguments of this function. Enter your GroundX API key to send a request directly from this web page. Select your language of choice to structure a code snippet based on your specified arguments.


delete

Look up a specific bucket by its bucketId.

Interact with the "Request Body" below to explore the arguments of this function. Enter your GroundX API key to send a request directly from this web page. Select your language of choice to structure a code snippet based on your specified arguments.


Rename a bucket.

Interact with the "Request Body" below to explore the arguments of this function. Enter your GroundX API key to send a request directly from this web page. Select your language of choice to structure a code snippet based on your specified arguments.


update

list all groups within your GroundX account.

Interact with the "Request Body" below to explore the arguments of this function. Enter your GroundX API key to send a request directly from this web page. Select your language of choice to structure a code snippet based on your specified arguments.


create a new group, a group being a collection of buckets which can be searched.

Interact with the "Request Body" below to explore the arguments of this function. Enter your GroundX API key to send a request directly from this web page. Select your language of choice to structure a code snippet based on your specified arguments.


Delete a group.

Interact with the "Request Body" below to explore the arguments of this function. Enter your GroundX API key to send a request directly from this web page. Select your language of choice to structure a code snippet based on your specified arguments.


look up a specific group by its groupId.

Interact with the "Request Body" below to explore the arguments of this function. Enter your GroundX API key to send a request directly from this web page. Select your language of choice to structure a code snippet based on your specified arguments.


Rename a group

Interact with the "Request Body" below to explore the arguments of this function. Enter your GroundX API key to send a request directly from this web page. Select your language of choice to structure a code snippet based on your specified arguments.


remove a bucket from a group. Buckets and groups can be associated many to many, this removes one bucket to group association without disturbing others.

Interact with the "Request Body" below to explore the arguments of this function. Enter your GroundX API key to send a request directly from this web page. Select your language of choice to structure a code snippet based on your specified arguments.


removeBucket

Add an existing bucket to an existing group. Buckets and groups can be associated many to many.

Interact with the "Request Body" below to explore the arguments of this function. Enter your GroundX API key to send a request directly from this web page. Select your language of choice to structure a code snippet based on your specified arguments.


addBucket

Get the account information associated with the API key.


List the current health status of all services. Statuses update every 5 minutes.


Look up the current health status of a specific service. Statuses update every 5 minutes.


GroundX

EyeLevel's GroundX APIs

Discover the supported document file types including pdf, jpg, png, docx, pptx, xlsx, csv, tsv, json, and txt. Learn how GroundX can crawl website content with the Document_crawlWebsite API endpoint.

Unauthorized to update bucket with given ID

the processId for the ingest process being checked

processId

Unauthorized to access process with given ID

a processId, bucketId, groupId, or projectId

The maximum number of returned documents. Accepts 1-100 with a default of 20.

Only documents with names that contain the filter string will be returned in the results.

filter

The document attribute that will be used to sort the results.

sort

The order in which to sort the results. A value for sort must also be set.

sortOrder

A status filter on the get documents query. If this value is set, then only documents with this status will be returned in the results.

status

A token for pagination. If the number of documents for a given query is larger than n, the response will include a "nextToken" value. That token can be included in this field to retrieve the next batch of n documents.

nextToken

Invalid process, bucket, group, or project ID

Unauthorized to access process, bucket, group, or project with given ID

A list of documentIds which correspond to documents ingested by GroundX

documentIds

Unauthorized to delete document with given ID

A documentId which correspond to a document ingested by GroundX

documentId

The documentId of the document for which GroundX information will be provided.

Unauthorized to access document with given ID

The bucketId, groupId, projectId, or documentId to be searched. The document or documents within the specified container will be compared to the query, and relevant information will be extracted.

The maximum number of returned search results. Accepts 1-100 with a default of 20.

A token for pagination. If the number of search results for a given query is larger than n, the response will include a "nextToken" value. That token can be included in this field to retrieve the next batch of n search results.

The amount of data returned with each search result. 0 == no search results, only the recommended context. 1 == search results but no searchData. 2 == search results and searchData.

verbosity

Unauthorized to access resource with given ID

The maximum number of returned buckets. Accepts 1-100 with a default of 20.

A token for pagination. If the number of buckets for a given query is larger than n, the response will include a "nextToken" value. That token can be included in this field to retrieve the next batch of n buckets.

The bucketId of the bucket being deleted.

bucketId

Unauthorized to delete bucket with given ID

Unauthorized to access bucket with given ID

The bucketId of the bucket being updated.

The maximum number of returned groups. Accepts 1-100 with a default of 20.

A token for pagination. If the number of groups for a given query is larger than n, the response will include a "nextToken" value. That token can be included in this field to retrieve the next batch of n groups.

groupId

Unauthorized to delete group with given ID

Unauthorized to access group with given ID

Unauthorized to update group with given ID

The groupId of the group which the bucket will be removed from.

The bucketId of the bucket which will be removed from the group.

The groupId of the group which the bucket will be added to.

The bucketId of the bucket being added to the group.