Skip to content

improvement: update llm utility methods#331

Open
overtonch wants to merge 5 commits intodevelopfrom
coverton/update-llm-util-docstrings
Open

improvement: update llm utility methods#331
overtonch wants to merge 5 commits intodevelopfrom
coverton/update-llm-util-docstrings

Conversation

@overtonch
Copy link
Contributor

Before this PR

The LLM utils:

  1. Did not have docs indicating that they are intended for usage with the official provider SDKs. This is important since the URLs returned by the various utils may not work with clients from other SDKs (e.g., Vercel) if their client expects the URL in a different format, even for the same model.
  2. Did not expose the gateway base URL method to users despite it being required if they need to construct custom URLs for use with other provider SDKs
  3. Did not strip an extra https:// from the beginning of the context var which is present in some environments, leading to connection failures when trying to call a URL starting with https://https://

After this PR

==COMMIT_MSG==
Updates the LLM utility methods
==COMMIT_MSG==

Possible downsides?

None

@changelog-app
Copy link

changelog-app bot commented Mar 20, 2026

Generate changelog in changelog/@unreleased

Type (Select exactly one)

  • Feature (Adding new functionality)
  • Improvement (Improving existing functionality)
  • Fix (Fixing an issue with existing functionality)
  • Break (Creating a new major version by breaking public APIs)
  • Deprecation (Removing functionality in a non-breaking way)
  • Migration (Automatically moving data/functionality to a new system)

Description

Updates the LLM utility methods

Check the box to generate changelog(s)

  • Generate changelog entry

hostname = HOSTNAME_VAR.get()
if hostname is None:
raise RuntimeError("Foundry API gateway base URL is not available in the current context.")
if hostname.startswith("https://"):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do all callers of this method handle the non-prefixed hostname as well?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the only callers today are in this file, and they already are assuming no scheme is present and prepend https://

@bryantpark04
Copy link
Contributor

language_models/utils.py was created by excavator PR #324. Are we sure this isn't one of our generated files that has to be updated upstream in python-platform-sdk?

@overtonch overtonch changed the title update logic improvement: update llm utility methods Mar 20, 2026
@overtonch
Copy link
Contributor Author

Thanks for checking me, I actually put them in the wrong place.

These are a special case where we add them to the assets directory and then have entries in config.json that copy them over into the generated package after generation is re-run

overtonch added 3 commits March 20, 2026 16:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants