Skip to content

Conversation

@priyanshu-kun
Copy link
Contributor

Successfully finalized the server-side of the OpenAPI web search project. Developed an indexing endpoint, implemented indexing functionality, and incorporated efficient retrieval capabilities for search functionality.

const CC_SERVER_URL = 'https://index.commoncrawl.org/';
const RABBIT_MQ_URL = 'amqp://localhost:5672';
const ELASTIC_LOCAL_URL = 'https://localhost:9200';
const ELASTIC_HTTP_CRED = '/home/priyanshu/Downloads/ELK/primary-node/config/certs/http_ca.crt';
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@priyanshu-kun remove/edit this path to be something generic.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's pretty difficult to determined the location of http_ca.crt file because it varied system to system so used has to set that accordingly.

const ELASTIC_HTTP_CRED = '/home/priyanshu/Downloads/ELK/primary-node/config/certs/http_ca.crt';
const INDEX_NAME = 'openapi_definition';
const QUEUE_NAME = 'index-files-jobs';
const ELASTIC_USERNAME = 'elastic';
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please dont store usernames and passwords

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

they didn't make any difference because these credentials are for local elasticsearch instance and if someone somehow get them in that case they are completely usless. But for production environment I have to rewrite the elasticseach connection driver altogether according to docs.

@HimanshuS129
Copy link
Collaborator

One suggestion, swagger validator seems to be filtering alot, I think it will also filter multi file schemas.
We should also index non-validated defintions also. WDYT @vinitshahdeo ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants