Skip to content

Update the user-agent for preview purposes. #27

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
imjihun wants to merge 4 commits into
base: main
Choose a base branch
from
Open

Update the user-agent for preview purposes. #27

Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
9b5b5d9
Update the user-agent for preview purposes.
imjihun Mar 2, 2022
f68abd9
Bug: fix module html split range check text
Saehanseul Apr 5, 2022
9737ed3
Feat: title 보다 og:title 태그가 우선 반영되도록 수정
imjihun Apr 28, 2022
7d7ebef
Bug: tsc bug 수정
imjihun Jan 9, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 12 additions & 11 deletions src/utils.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -83,8 +83,9 @@ export const getPreviewData = async (text: string, requestTimeout = 5000) => {

const request = fetch(url, {
headers: {
'User-Agent':
'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/74.0.3729.169 Safari/537.36',
/* very slow */
// 'User-Agent': 'googlebot',
'User-Agent': 'facebookexternalhit/1.1',
},
signal: abortController.signal,
})
Expand Down Expand Up @@ -112,7 +113,7 @@ export const getPreviewData = async (text: string, requestTimeout = 5000) => {
// Some pages return undefined
if (!html) return previewData

const head = html.substring(0, html.indexOf('<body'))
const head = html.substring(0, html.indexOf('</head>'))

// Get page title
const title = REGEX_TITLE.exec(head)
Expand All @@ -121,7 +122,7 @@ export const getPreviewData = async (text: string, requestTimeout = 5000) => {
let matches: RegExpMatchArray | null
const meta: RegExpMatchArray[] = []
while ((matches = REGEX_META.exec(head)) !== null) {
meta.push([...matches])
meta.push([...matches] as RegExpMatchArray)
}

const metaPreviewData = meta.reduce<{
Expand Down Expand Up @@ -152,18 +153,18 @@ export const getPreviewData = async (text: string, requestTimeout = 5000) => {
title: ogTitle ? getHtmlEntitiesDecodedText(ogTitle) : acc.title,
}
},
{ title: previewData.title }
{}
)

previewData.description = metaPreviewData.description
previewData.image = await getPreviewDataImage(metaPreviewData.imageUrl)
previewData.title = metaPreviewData.title
previewData.title = metaPreviewData.title || previewData.title

if (!previewData.image) {
let imageMatches: RegExpMatchArray | null
const tags: RegExpMatchArray[] = []
while ((imageMatches = REGEX_IMAGE_TAG.exec(html)) !== null) {
tags.push([...imageMatches])
tags.push([...imageMatches] as RegExpMatchArray)
}

let images: PreviewDataImage[] = []
Expand Down Expand Up @@ -201,17 +202,17 @@ export const getPreviewDataImage = async (url?: string) => {
const image: PreviewDataImage = { height, url, width }
return image
}
} catch {}
} catch { }
}

export const oneOf =
<T extends (...args: A) => any, U, A extends any[]>(
truthy: T | undefined,
falsy: U
) =>
(...args: Parameters<T>): ReturnType<T> | U => {
return truthy ? truthy(...args) : falsy
}
(...args: Parameters<T>): ReturnType<T> | U => {
return truthy ? truthy(...args) : falsy
}

export const REGEX_EMAIL = /([a-zA-Z0-9+._-]+@[a-zA-Z0-9._-]+\.[a-zA-Z0-9_-]+)/g
export const REGEX_IMAGE_CONTENT_TYPE = /image\/*/g
Expand Down