Currently, when a request to the inference endpoint fails within the AskAI feature, the system returns a generic error stating that the operation failed. This lack of context makes debugging and troubleshooting difficult.
I would like to propose the following enhancements:
1- Detailed Error Reporting: Instead of masking the error, the system should surface the specific error details and status codes returned by the inference endpoint.
2- Retry and Fallback Mechanism: Introduce a configurable retry and/or fallback strategy for failed inference requests to improve resilience.
3- Scope Extension: Apply this same retry/fallback mechanism to the MCP server, as it currently suffers from the same generic failure behavior.
Implementing these changes will significantly improve the observability of the system and provide a more robust experience for both the AskAI feature and the MCP server.
Currently, when a request to the inference endpoint fails within the AskAI feature, the system returns a generic error stating that the operation failed. This lack of context makes debugging and troubleshooting difficult.
I would like to propose the following enhancements:
1- Detailed Error Reporting: Instead of masking the error, the system should surface the specific error details and status codes returned by the inference endpoint.
2- Retry and Fallback Mechanism: Introduce a configurable retry and/or fallback strategy for failed inference requests to improve resilience.
3- Scope Extension: Apply this same retry/fallback mechanism to the MCP server, as it currently suffers from the same generic failure behavior.
Implementing these changes will significantly improve the observability of the system and provide a more robust experience for both the AskAI feature and the MCP server.