Skip to content

Add --verbose flag for HTTP debugging and clarify Bluesky identifier format #146

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Draft
wants to merge 5 commits into
base: main
Choose a base branch
from
Draft

Add --verbose flag for HTTP debugging and clarify Bluesky identifier format #146

wants to merge 5 commits into from
+184 −7

Conversation

Copy link
Contributor

Copilot AI commented Oct 13, 2025 •
edited
Loading

Problem

Users attempting to debug network issues with various social media integrations (particularly Bluesky) were encountering opaque error messages that made troubleshooting difficult. For example:

❌ Bluesky failed.
SyntaxError: Unexpected token '<', "<!DOCTYPE "... is not valid JSON

Without visibility into the actual HTTP responses, users couldn't determine whether the issue was due to incorrect credentials, wrong identifier format, server errors, or other problems.

Additionally, the README documentation for Bluesky setup didn't clearly specify whether the identifier should include the @ symbol or use the DID format, leading to configuration confusion.

Solution

1. Added --verbose CLI Flag

Implemented comprehensive HTTP request/response logging that activates when the --verbose flag is present. The logging system:

  • Captures all HTTP requests: URL, method, headers, and request body
  • Captures all HTTP responses: Status code, headers, and response body (regardless of success or failure)
  • Handles multiple content types: JSON responses are pretty-printed, text responses are displayed as-is
  • Includes security safeguards:
    • Redacts sensitive headers (Authorization, Cookie, Set-Cookie, X-API-Key, API-Key)
    • Redacts sensitive request body fields (password, access_token, api_key)
    • Truncates large payloads to 5000 characters to prevent memory exhaustion and console overflow

Example output:

$ crosspost --verbose --bluesky "Testing"

--- HTTP Request ---
URL: https://bsky.social/xrpc/com.atproto.server.createSession
Method: POST
Headers: {
  "Content-Type": "application/json"
}
Body: {"identifier":"username.bsky.social","password":"[REDACTED]"}

--- HTTP Response ---
Status: 401 Unauthorized
Headers: {
  "content-type": "text/html"
}
Body: <!DOCTYPE html>
<html>
<head><title>401 Unauthorized</title></head>
<body><h1>401 Unauthorized</h1></body>
</html>
--- End Response ---

❌ Bluesky failed.
SyntaxError: Unexpected token '<', "<!DOCTYPE "... is not valid JSON

Now users can immediately see that the server returned an HTML error page instead of JSON, indicating an authentication problem.

2. Updated Bluesky README Documentation

Clarified the BLUESKY_IDENTIFIER format in the setup section:

  • Should be the human-readable username without the @ symbol (e.g., username.bsky.social)
  • Should not be the DID (Decentralized Identifier like did:plc:...)

Implementation Details

The verbose logging uses a global fetch wrapper approach, which is minimal and non-invasive:

  • Only wraps fetch when --verbose flag is present (zero performance impact otherwise)
  • No changes required to existing strategy implementations
  • Properly handles different header object types (Headers API vs plain objects)
  • Robust error handling for various request/response body types

Testing

  • All 299 existing tests continue to pass
  • Added new test to verify --verbose flag appears in help output
  • Linting and type checking pass
  • Full test suite (unit, build, type tests) passes

Fixes #[issue-number]

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

  • httpbin.org
    • Triggering command: node /tmp/test-verbose.js --verbose (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

Original prompt

This section details on the original issue you should resolve

<issue_title>Debug option for responses from networks</issue_title>
<issue_description>I am currently setting up various strategies, but I'm having trouble getting my Bluesky integration to work. It would be nice if crosspost --verbose or some other option would be available that might print a readable output of the response from the network:

> crosspost --bluesky "Testing"
❌ Bluesky failed.
SyntaxError: Unexpected token '<', "<!DOCTYPE "... is not valid JSON
    at JSON.parse (<anonymous>)
    at parseJSONFromBytes (node:internal/deps/undici/undici:6223:19)
    at successSteps (node:internal/deps/undici/undici:6204:27)
    at readAllBytes (node:internal/deps/undici/undici:5173:13)
    at process.processTicksAndRejections (node:internal/process/task_queues:105:5)

The above is what I get when I call. I assume an error page is returned, but I don't know its content at this point. Is there an easy way to somehow log (either into the terminal or a logfile) what the responses are?

Regarding the Bluesky issue, I assume it's the IDENTIFIER for my user. The docs are not very clear about what identifier that is (Username with or without @, DID from Bluesky). We cross that bridge when we come to it :)</issue_description>

<agent_instructions>Add a --verbose CLI option that, when present, causes the CLI to output the HTTP response from each request to the console. This should happen regardless of the HTTP status code.

We also need to update the README for the BlueSky strategy to indicate that the username must be the username without @.</agent_instructions>

Comments on the Issue (you are @copilot in this section)

@nzakas Sorry I missed this. I've been buried under notifications for a while now.

Yes, I think we can add a --verbose flag that outputs the raw response from each request. I think it should do so regardless of response success or failure to keep with the meaning of the flag even if it's most helpful for failures.

And we can update the README so that it states the Bluesky username should not contain @ and should be the human-readable username and not the DiD</comment_new>

Fixes #142

💬 Share your feedback on Copilot coding agent for the chance to win a $200 gift card! Click here to start the survey.

Copilot AI changed the title [WIP] Add debug option for network response output Add --verbose flag for HTTP debugging and clarify Bluesky identifier format Oct 13, 2025
Copilot AI requested a review from nzakas October 13, 2025 19:05
Copilot finished work on behalf of nzakas October 13, 2025 19:05
nzakas
nzakas reviewed Oct 13, 2025
View reviewed changes
README.md
**Important:** Do not use your login password with the API.
**Important:**
- Do not use your login password with the API.
- The identifier should be your human-readable username without the `@` symbol (e.g., `username.bsky.social`), not your DID (Decentralized Identifier).
Copy link
Contributor

@nzakas nzakas Oct 13, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- The identifier should be your human-readable username without the `@` symbol (e.g., `username.bsky.social`), not your DID (Decentralized Identifier).
- The identifier should be your human-readable username without the `@` symbol and without the host (e.g., `username`), not your DID (Decentralized Identifier).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@nzakas nzakas nzakas left review comments

Assignees

@nzakas nzakas

Copilot code review Copilot

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Debug option for responses from networks

2 participants

@nzakas