Skip to content

Commit

Permalink
Merge pull request #1 from ursnj/expose-node-methods
Browse files Browse the repository at this point in the history
Programmatic sitemap generator
  • Loading branch information
ursnj authored Oct 18, 2024
2 parents e2e56fb + 075bad0 commit dc3bb07
Show file tree
Hide file tree
Showing 7 changed files with 225 additions and 99 deletions.
4 changes: 2 additions & 2 deletions .github/workflows/npm-publish.yml
Original file line number Diff line number Diff line change
Expand Up @@ -16,8 +16,8 @@ jobs:
- name: Set up Node.js
uses: actions/setup-node@v3
with:
node-version: '20'
cache: 'npm'
node-version: "20"
cache: "npm"

# Add npm authentication for publishing
- name: Authenticate with npm
Expand Down
58 changes: 49 additions & 9 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,37 +12,77 @@
- 📦 Customizable Output Path.
- 🎨 Flexible Change Frequency.

## 🕹 Usage
## 🕹 CLI Usage

### Simple generation usage:

```
npx sitemaper generate -w https://www.nayanui.com
```

### Advanced generation Usage:

```
npx sitemaper --website https://www.example.com --depth 10 --output ./sitemap.xml --changefreq daily
npx sitemaper generate --website https://www.nayanui.com --depth 10 --output ./sitemap.xml --changefreq daily
```

You can also use the shorter version of this command.

```
npx sitemaper -w https://www.example.com -d 10 -o ./sitemap.xml -f daily
npx sitemaper generate -w https://www.nayanui.com -d 10 -o ./sitemap.xml -f daily
```

You can also integrate Sitemaper with your localhost to generate sitemaps without any deployments.

```
npx sitemaper -w http://localhost:3000 -r https://www.nayanui.com -d 10 -o ./sitemap.xml -f daily
npx sitemaper generate -w http://localhost:3000 -r https://www.nayanui.com -d 10 -o ./sitemap.xml -f daily
```

this case it crawl your localhost URL and replace it with replacement URL.

| Parameter | Default | Usage |
|-------------------|-------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| --website / -w | https://www.example.com | Pass website base URL to start crawling. |
| --replacer / -r | '' | Pass replacement URL to replace crawled url, this will be mostly useful to crawl localhost and replace it with original URL. |
| ----------------- | ----------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| --website / -w | https://www.nayanui.com | Pass website base URL to start crawling. |
| --replacer / -r | '' | Pass replacement URL to replace crawled url, this will be mostly useful to crawl localhost and replace it with original URL. |
| --depth / -d | 10 | Pass depth to let the generator know how depth it need to crawl. |
| --output / -o | ./sitemap.xml | Pass output to let the generator know where to keep generated sitemap. |
| --changefreq / -f | daily | Pass change frequency to let the generator know how frequently your content change, possible options are ***always, hourly, daily, weekly, monthly, yearly, never***. |
| --changefreq / -f | daily | Pass change frequency to let the generator know how frequently your content change, possible options are **_always, hourly, daily, weekly, monthly, yearly, never_**. |

### Sitemap validation usage:

```
npx sitemaper validate --output ./sitemap.xml
```

You can also use the shorter version of this command.

```
npx sitemaper validate -o ./sitemap.xml
```

| Parameter | Default | Usage |
| ------------- | ------------- | ------------------------------------------------------------------------- |
| --output / -o | ./sitemap.xml | Pass output to let the generator know where to find and validate sitemap. |

## 🕹 Programatic Usage

You can also use Sitemaper programmatically with Node.js. Check out the implementation below.

```
import { generateSitemap, validateSitemap } from 'sitemaper';
generateSitemap('https://www.nayanui.com', '', 10, './sitemap.xml', 'daily');
validateSitemap('./sitemap.xml');
```

## 🖥 Future plans

Create a web application to automatically generate and submit sitemaps to search engines on a schedule.
- [x] Create Sitemaper CLI tool to generate and validate sitemaps efficiently.
- [x] Create Sitemaper programmatic API's to generate and validate sitemaps efficiently.
- [ ] Support multiple sitemaps if website is bigger than certain limit.
- [ ] Create a web application to automatically generate and submit sitemaps to search engines on a schedule.

## 🤝 Contributing

Expand Down
33 changes: 31 additions & 2 deletions package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

8 changes: 5 additions & 3 deletions package.json
Original file line number Diff line number Diff line change
@@ -1,17 +1,18 @@
{
"name": "sitemaper",
"version": "1.1.0",
"version": "1.2.0",
"private": false,
"sideEffects": false,
"description": "Simple tool for generating sitemaps for your website.",
"description": "Sitemaper is a powerful sitemap generator designed to simplify the process of creating accurate and efficient sitemaps for websites. It crawls through your site, maps its structure, and generates an optimized sitemap, helping improve SEO and site visibility.",
"type": "module",
"main": "./src/index.ts",
"homepage": "https://www.nayanui.com/devtools/sitemaper",
"bin": {
"sitemaper": "./dist/index.js"
},
"scripts": {
"build": "tsc"
"build": "tsc",
"format": "npx prettier --write --print-width 150 ."
},
"author": "Niranjan Devasani",
"license": "ISC",
Expand All @@ -23,6 +24,7 @@
"ora": "^8.1.0",
"ts-node": "^10.9.2",
"typescript": "^5.6.3",
"xml2js": "^0.6.2",
"xmlbuilder": "^15.1.1"
},
"files": [
Expand Down
84 changes: 26 additions & 58 deletions src/index.ts
Original file line number Diff line number Diff line change
@@ -1,71 +1,39 @@
#!/usr/bin/env node

import { Command } from 'commander';
import { crawlWebsite } from "./sitemaper.js";
import { existsSync } from 'fs';
import { URL } from 'url';
import { readFileSync } from "fs";
import { Command } from "commander";
import { generateSitemap, validateSitemap } from "./sitemaper.js";
import { validateChangefreq, validateDepth, validateOutput, validateWebsite } from "./utils.js";
const { name, version, description } = JSON.parse(readFileSync("./package.json", "utf8"));

const program = new Command();

// Helper function to validate depth is a positive integer
const validateDepth = (depth: string) => {
const parsedDepth = parseInt(depth, 10);
if (isNaN(parsedDepth) || parsedDepth < 1) {
throw new Error('Depth must be a positive integer greater than 0.');
}
return parsedDepth;
};

// Helper function to validate changefreq value
const validateChangefreq = (changefreq: string) => {
const validOptions = ['always', 'hourly', 'daily', 'weekly', 'monthly', 'yearly', 'never'];
if (!validOptions.includes(changefreq)) {
throw new Error(`Invalid changefreq value. Accepted values are: ${validOptions.join(', ')}`);
}
return changefreq;
};

// Helper function to validate website URL
const validateWebsite = (website: string) => {
try {
new URL(website);
return website;
} catch (error) {
throw new Error('Invalid website URL.');
}
};

// Helper function to validate output path (for example, it checks if the directory exists)
const validateOutput = (output: string) => {
const pathParts = output.split('/');
const dirPath = pathParts.slice(0, -1).join('/');
if (dirPath && !existsSync(dirPath)) {
throw new Error(`The directory ${dirPath} does not exist.`);
}
return output;
};
program.name(name).description(description).version(version);

program
.name('sitemaper')
.description('Simple tool for generating sitemaps for your website.')
.version('1.0.0');
.command("generate")
.option("-w, --website <url>", "The URL of the website to crawl", validateWebsite)
.option("-r, --replacer <url>", "The URL of the website to be replaced", validateWebsite)
.option("-d, --depth <number>", "Depth of the website to crawl", validateDepth)
.option("-o, --output <path>", "Output path for the sitemap.xml", validateOutput)
.option("-f, --changefreq <value>", "Change frequency for the sitemap (always, hourly, daily, weekly, monthly, yearly, never)", validateChangefreq)
.action((options) => {
const website = options.website || "https://www.nayanui.com";
const replacer = options.replacer || "";
const depth = options.depth || 10;
const output = options.output || "./sitemap.xml";
const changefreq = options.changefreq || "daily";
generateSitemap(website, replacer, depth, output, changefreq);
});

program
.option('-w, --website <url>', 'The URL of the website to crawl', validateWebsite)
.option('-r, --replacer <url>', 'The URL of the website to be replaced', validateWebsite)
.option('-d, --depth <number>', 'Depth of the website to crawl', validateDepth)
.option('-o, --output <path>', 'Output path for the sitemap.xml', validateOutput)
.option('-f, --changefreq <value>', 'Change frequency for the sitemap (always, hourly, daily, weekly, monthly, yearly, never)', validateChangefreq)
.command("validate")
.option("-o, --output <path>", "Output path for the sitemap.xml", validateOutput)
.action((options) => {
const website = options.website || 'https://www.example.com';
const replacer = options.replacer || '';
const depth = options.depth || 10;
const output = options.output || './sitemap.xml';
const changefreq = options.changefreq || 'daily';

// console.log({ website, replacer, depth, output, changefreq });

crawlWebsite(website, replacer, depth, output, changefreq);
const output = options.output || "./sitemap.xml";
validateSitemap(output);
});

export { generateSitemap, validateSitemap };

program.parse(process.argv);
Loading

0 comments on commit dc3bb07

Please sign in to comment.