Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

External URL Handling Error for Non-Image contentType in experimental_attachments #4443

Open
jongreen96 opened this issue Jan 19, 2025 · 0 comments
Labels
bug Something isn't working

Comments

@jongreen96
Copy link

Description

When providing an external URL with a non-image contentType (e.g., text/javascript) in the experimental_attachments field of a Message, the system throws an error. However, using a Data URL for the same file type does not trigger an error. The type definitions suggest that URLs for hosted files should be supported, but this behaviour is inconsistent.

Steps to Reproduce:

  1. Send a message using the following structure in the messages field:
{
  role: 'user',
  content: 'test',
  experimental_attachments: [
    {
      name: 'test.txt',
      contentType: 'text/plain',
      url: 'https://pub-*******************.r2.dev/user-attachments/gP3VdxrFT9/test.txt'
    }
  ]
}
  1. Observe the following error:
Error [AI_UnsupportedFunctionalityError]: 'File content part type text/javascript in user messages' functionality not supported.
    at Array.map (<anonymous>) {
  cause: undefined,
  functionality: 'File content part type text/javascript in user messages'
}
[TypeError: Cannot read properties of undefined (reading 'toolCalls')]
  1. Repeat the test, but replace the url with a Data URL:
{
  role: 'user',
  content: 'test',
  experimental_attachments: [
    {
      name: 'test.txt',
      contentType: 'text/plain',
      url: 'data:text/plain;base64,dGVzdA0K'
    }
  ]
}

Note that no error occurs for the Data URL.

Code example

export async function POST(req: Request) {
  const { messages } = await req.json();

  const result = await streamText({
    model: openai('gpt-4o'),
    messages: {
      role: 'user',
      content: 'test',
      experimental_attachments: [
        {
          name: 'test.txt',
          contentType: 'text/plain',
          url: 'https://pub-*******************.r2.dev/user-attachments/gP3VdxrFT9/test.txt',
        },
      ],
    },
  });

  return result.toDataStreamResponse();
}


### AI provider

"@ai-sdk/openai": "^1.0.11"

### Additional context

The docs and type definition states:
* The URL of the attachment. It can either be a URL to a hosted file or a [Data URL](https://developer.mozilla.org/en-US/docs/Web/HTTP/Basics_of_HTTP/Data_URLs).

But unless the hosted file is an image, it throws this error.
@jongreen96 jongreen96 added the bug Something isn't working label Jan 19, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant