pax_global_header00006660000000000000000000000064130037014250014505gustar00rootroot0000000000000052 comment=7ceea730d54726affeaca62d6e362db0b6881f93 node-klaw-1.3.1/000077500000000000000000000000001300370142500133705ustar00rootroot00000000000000node-klaw-1.3.1/.gitignore000066400000000000000000000000161300370142500153550ustar00rootroot00000000000000node_modules/ node-klaw-1.3.1/.npmignore000066400000000000000000000000401300370142500153610ustar00rootroot00000000000000tests/ appveyor.yml .travis.yml node-klaw-1.3.1/.travis.yml000066400000000000000000000003261300370142500155020ustar00rootroot00000000000000sudo: false language: node_js node_js: - "0.10" - "0.12" - "io.js" - "4" - "5" matrix: include: - node_js: "4" env: TEST_SUITE=lint env: - TEST_SUITE=unit script: npm run-script $TEST_SUITE node-klaw-1.3.1/CHANGELOG.md000066400000000000000000000021101300370142500151730ustar00rootroot000000000000001.3.1 / 2016-10-25 ------------------ ### Added - `graceful-fs` added as an `optionalDependencies`. Thanks [ryanzim]! 1.3.0 / 2016-06-09 ------------------ ### Added - `filter` option to pre-filter and not walk directories. 1.2.0 / 2016-04-16 ------------------ - added support for custom `fs` implementation. Useful for https://github.com/tschaub/mock-fs 1.1.3 / 2015-12-23 ------------------ - bugfix: if `readdir` error, got hung up. See: https://github.com/jprichardson/node-klaw/issues/1 1.1.2 / 2015-11-12 ------------------ - assert that param `dir` is a `string` 1.1.1 / 2015-10-25 ------------------ - bug fix, options not being passed 1.1.0 / 2015-10-25 ------------------ - added `queueMethod` and `pathSorter` to `options` to affect searching strategy. 1.0.0 / 2015-10-25 ------------------ - removed unused `filter` param - bugfix: always set `streamOptions` to `objectMode` - simplified, converted from push mode (streams 1) to proper pull mode (streams 3) 0.1.0 / 2015-10-25 ------------------ - initial release [ryanzim]: https://github.com/ryanzim node-klaw-1.3.1/LICENSE000066400000000000000000000020741300370142500144000ustar00rootroot00000000000000(The MIT License) Copyright (c) 2015-2016 JP Richardson Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the 'Software'), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED 'AS IS', WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. node-klaw-1.3.1/README.md000066400000000000000000000161511300370142500146530ustar00rootroot00000000000000Node.js - klaw ============== A Node.js file system walker extracted from [fs-extra](https://github.com/jprichardson/node-fs-extra). [![npm Package](https://img.shields.io/npm/v/klaw.svg?style=flat-square)](https://www.npmjs.org/package/klaw) [![build status](https://api.travis-ci.org/jprichardson/node-klaw.svg)](http://travis-ci.org/jprichardson/node-klaw) [![windows build status](https://ci.appveyor.com/api/projects/status/github/jprichardson/node-klaw?branch=master&svg=true)](https://ci.appveyor.com/project/jprichardson/node-klaw/branch/master) Standard Install ------- npm i --save klaw Name ---- `klaw` is `walk` backwards :p Usage ----- ### klaw(directory, [options]) Returns a [Readable stream](https://nodejs.org/api/stream.html#stream_class_stream_readable) that iterates through every file and directory starting with `dir` as the root. Every `read()` or `data` event returns an object with two properties: `path` and `stats`. `path` is the full path of the file and `stats` is an instance of [fs.Stats](https://nodejs.org/api/fs.html#fs_class_fs_stats). - `directory`: The directory to recursively walk. Type `string`. - `options`: [Readable stream options](https://nodejs.org/api/stream.html#stream_new_stream_readable_options) and the following: - `queueMethod` (`string`, default: `'shift'`): Either `'shift'` or `'pop'`. On `readdir()` array, call either `shift()` or `pop()`. - `pathSorter` (`function`, default: `undefined`): Sorting [function for Arrays](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array/sort). - `fs` (`object`, default: `require('fs')`): Use this to hook into the `fs` methods or to use [`mock-fs`](https://github.com/tschaub/mock-fs) - `filter` (`function`, default: `undefined`): Filtering [function for Arrays](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array/filter) **Streams 1 (push) example:** ```js var klaw = require('klaw') var items = [] // files, directories, symlinks, etc klaw('/some/dir') .on('data', function (item) { items.push(item.path) }) .on('end', function () { console.dir(items) // => [ ... array of files] }) ``` **Streams 2 & 3 (pull) example:** ```js var klaw = require('klaw') var items = [] // files, directories, symlinks, etc klaw('/some/dir') .on('readable', function () { var item while ((item = this.read())) { items.push(item.path) } }) .on('end', function () { console.dir(items) // => [ ... array of files] }) ``` If you're not sure of the differences on Node.js streams 1, 2, 3 then I'd recommend this resource as a good starting point: https://strongloop.com/strongblog/whats-new-io-js-beta-streams3/. ### Error Handling Listen for the `error` event. Example: ```js var klaw = require('klaw') klaw('/some/dir') .on('readable', function () { var item while ((item = this.read())) { // do something with the file } }) .on('error', function (err, item) { console.log(err.message) console.log(item.path) // the file the error occurred on }) .on('end', function () { console.dir(items) // => [ ... array of files] }) ``` ### Aggregation / Filtering / Executing Actions (Through Streams) On many occasions you may want to filter files based upon size, extension, etc. Or you may want to aggregate stats on certain file types. Or maybe you want to perform an action on certain file types. You should use the module [`through2`](https://www.npmjs.com/package/through2) to easily accomplish this. Install `through2`: npm i --save through2 **Example (skipping directories):** ```js var klaw = require('klaw') var through2 = require('through2') var excludeDirFilter = through2.obj(function (item, enc, next) { if (!item.stats.isDirectory()) this.push(item) next() }) var items = [] // files, directories, symlinks, etc klaw('/some/dir') .pipe(excludeDirFilter) .on('data', function (item) { items.push(item.path) }) .on('end', function () { console.dir(items) // => [ ... array of files without directories] }) ``` **Example (ignore hidden directories):** ```js var klaw = require('klaw') var path = require('path') var filterFunc = function(item){ var basename = path.basename(item) return basename === '.' || basename[0] !== '.' } klaw('/some/dir', { filter : filterFunc }) .on('data', function(item){ // only items of none hidden folders will reach here }) ``` **Example (totaling size of PNG files):** ```js var klaw = require('klaw') var path = require('path') var through2 = require('through2') var totalPngsInBytes = 0 var aggregatePngSize = through2.obj(function (item, enc, next) { if (path.extname(item.path) === '.png') { totalPngsInBytes += item.stats.size } this.push(item) next() }) klaw('/some/dir') .pipe(aggregatePngSize) .on('data', function (item) { items.push(item.path) }) .on('end', function () { console.dir(totalPngsInBytes) // => total of all pngs (bytes) }) ``` **Example (deleting all .tmp files):** ```js var fs = require('fs') var klaw = require('klaw') var through2 = require('through2') var deleteAction = through2.obj(function (item, enc, next) { this.push(item) if (path.extname(item.path) === '.tmp') { item.deleted = true fs.unklink(item.path, next) } else { item.deleted = false next() } }) var deletedFiles = [] klaw('/some/dir') .pipe(deleteAction) .on('data', function (item) { if (!item.deleted) return deletedFiles.push(item.path) }) .on('end', function () { console.dir(deletedFiles) // => all deleted files }) ``` You can even chain a bunch of these filters and aggregators together. By using multiple pipes. **Example (using multiple filters / aggregators):** ```js klaw('/some/dir') .pipe(filterCertainFiles) .pipe(deleteSomeOtherFiles) .on('end', function () { console.log('all done!') }) ``` **Example passing (piping) through errors:** Node.js does not `pipe()` errors. This means that the error on one stream, like `klaw` will not pipe through to the next. If you want to do this, do the following: ```js var klaw = require('klaw') var through2 = require('through2') var excludeDirFilter = through2.obj(function (item, enc, next) { if (!item.stats.isDirectory()) this.push(item) next() }) var items = [] // files, directories, symlinks, etc klaw('/some/dir') .on('error', function (err) { excludeDirFilter.emit('error', err) }) // forward the error on .pipe(excludeDirFilter) .on('data', function (item) { items.push(item.path) }) .on('end', function () { console.dir(items) // => [ ... array of files without directories] }) ``` ### Searching Strategy Pass in options for `queueMethod` and `pathSorter` to affect how the file system is recursively iterated. See the code for more details, it's less than 50 lines :) License ------- MIT Copyright (c) 2015 [JP Richardson](https://github.com/jprichardson) node-klaw-1.3.1/appveyor.yml000066400000000000000000000011331300370142500157560ustar00rootroot00000000000000# Test against this version of Node.js environment: matrix: # node.js - nodejs_version: "0.10" - nodejs_version: "0.12" - nodejs_version: "4" - nodejs_version: "5" # Install scripts. (runs after repo cloning) install: # Get the latest stable version of Node.js or io.js - ps: Install-Product node $env:nodejs_version # install modules - npm config set loglevel warn - npm install --silent # Post-install test scripts. test_script: # Output useful info for debugging. - node --version - npm --version # run tests - npm test # Don't actually build. build: off node-klaw-1.3.1/package.json000066400000000000000000000016171300370142500156630ustar00rootroot00000000000000{ "name": "klaw", "version": "1.3.1", "description": "File system walker with Readable stream interface.", "main": "./src/index.js", "scripts": { "lint": "standard", "test": "npm run lint && npm run unit", "unit": "tape tests/**/*.js | tap-spec" }, "repository": { "type": "git", "url": "git+https://github.com/jprichardson/node-klaw.git" }, "keywords": [ "walk", "walker", "fs", "fs-extra", "readable", "streams" ], "author": "JP Richardson", "license": "MIT", "bugs": { "url": "https://github.com/jprichardson/node-klaw/issues" }, "homepage": "https://github.com/jprichardson/node-klaw#readme", "devDependencies": { "mkdirp": "^0.5.1", "mock-fs": "^3.8.0", "rimraf": "^2.4.3", "standard": "^8.4.0", "tap-spec": "^4.1.1", "tape": "^4.2.2" }, "optionalDependencies": { "graceful-fs": "^4.1.9" } } node-klaw-1.3.1/src/000077500000000000000000000000001300370142500141575ustar00rootroot00000000000000node-klaw-1.3.1/src/assign.js000066400000000000000000000007451300370142500160070ustar00rootroot00000000000000// simple mutable assign (extracted from fs-extra) // I really like object-assign package, but I wanted a lean package with zero deps function _assign () { var args = [].slice.call(arguments).filter(function (i) { return i }) var dest = args.shift() args.forEach(function (src) { Object.keys(src).forEach(function (key) { dest[key] = src[key] }) }) return dest } // thank you baby Jesus for Node v4 and Object.assign module.exports = Object.assign || _assign node-klaw-1.3.1/src/index.js000066400000000000000000000033071300370142500156270ustar00rootroot00000000000000var assert = require('assert') var fs try { fs = require('graceful-fs') } catch (e) { fs = require('fs') } var path = require('path') var Readable = require('stream').Readable var util = require('util') var assign = require('./assign') function Walker (dir, options) { assert.strictEqual(typeof dir, 'string', '`dir` parameter should be of type string. Got type: ' + typeof dir) var defaultStreamOptions = { objectMode: true } var defaultOpts = { queueMethod: 'shift', pathSorter: undefined, filter: undefined } options = assign(defaultOpts, options, defaultStreamOptions) Readable.call(this, options) this.root = path.resolve(dir) this.paths = [this.root] this.options = options this.fs = options.fs || fs // mock-fs } util.inherits(Walker, Readable) Walker.prototype._read = function () { if (this.paths.length === 0) return this.push(null) var self = this var pathItem = this.paths[this.options.queueMethod]() self.fs.lstat(pathItem, function (err, stats) { var item = { path: pathItem, stats: stats } if (err) return self.emit('error', err, item) if (!stats.isDirectory()) return self.push(item) self.fs.readdir(pathItem, function (err, pathItems) { if (err) { self.push(item) return self.emit('error', err, item) } pathItems = pathItems.map(function (part) { return path.join(pathItem, part) }) if (self.options.filter) pathItems = pathItems.filter(self.options.filter) if (self.options.pathSorter) pathItems.sort(self.options.pathSorter) pathItems.forEach(function (pi) { self.paths.push(pi) }) self.push(item) }) }) } function walk (root, options) { return new Walker(root, options) } module.exports = walk node-klaw-1.3.1/tests/000077500000000000000000000000001300370142500145325ustar00rootroot00000000000000node-klaw-1.3.1/tests/_test.js000066400000000000000000000013341300370142500162070ustar00rootroot00000000000000var mkdirp = require('mkdirp') var os = require('os') var path = require('path') var rimraf = require('rimraf') var tape = require('tape') // for all practical purposes, this is a beforeEach and afterEach function test (desc, testFn) { tape(desc, function (t) { var testDir = path.join(os.tmpdir(), 'klaw-tests') rimraf(testDir, function (err) { if (err) return t.end(err) mkdirp(testDir, function (err) { if (err) return t.end(err) var oldEnd = t.end t.end = function () { rimraf(testDir, function (err) { err ? oldEnd.apply(t, [err]) : oldEnd.apply(t, arguments) }) } testFn(t, testDir) }) }) }) } module.exports = test node-klaw-1.3.1/tests/fixtures.json000066400000000000000000000001201300370142500172670ustar00rootroot00000000000000[ "a/b/c/d.txt", "a/e.jpg", "h/i/j/k.txt", "h/i/l.txt", "h/i/m.jpg" ] node-klaw-1.3.1/tests/fixtures_path-sorter.json000066400000000000000000000000301300370142500216170ustar00rootroot00000000000000[ "a", "b", "c" ] node-klaw-1.3.1/tests/walk.test.js000066400000000000000000000033021300370142500170020ustar00rootroot00000000000000var fs = require('fs') var mkdirp = require('mkdirp') var path = require('path') var test = require('./_test') var klaw = require('../') var fixtures = require('./fixtures') test('should work w/ streams 1', function (t, testDir) { fixtures.forEach(function (f) { f = path.join(testDir, f) var dir = path.dirname(f) mkdirp.sync(dir) fs.writeFileSync(f, path.basename(f, path.extname(f))) }) var items = [] klaw(testDir) .on('data', function (item) { items.push(item.path) }) .on('error', t.end) .on('end', function () { items.sort() var expected = ['a', 'a/b', 'a/b/c', 'a/b/c/d.txt', 'a/e.jpg', 'h', 'h/i', 'h/i/j', 'h/i/j/k.txt', 'h/i/l.txt', 'h/i/m.jpg'] expected = expected.map(function (item) { return path.join(path.join(testDir, item)) }) expected.unshift(testDir) t.same(items, expected) t.end() }) }) test('should work w/ streams 2/3', function (t, testDir) { fixtures.forEach(function (f) { f = path.join(testDir, f) var dir = path.dirname(f) mkdirp.sync(dir) fs.writeFileSync(f, path.basename(f, path.extname(f))) }) var items = [] klaw(testDir) .on('readable', function () { var item while ((item = this.read())) { items.push(item.path) } }) .on('error', t.end) .on('end', function () { items.sort() var expected = ['a', 'a/b', 'a/b/c', 'a/b/c/d.txt', 'a/e.jpg', 'h', 'h/i', 'h/i/j', 'h/i/j/k.txt', 'h/i/l.txt', 'h/i/m.jpg'] expected = expected.map(function (item) { return path.join(path.join(testDir, item)) }) expected.unshift(testDir) t.same(items, expected) t.end() }) }) node-klaw-1.3.1/tests/walk_filter.test.js000066400000000000000000000017471300370142500203620ustar00rootroot00000000000000var fs = require('fs') var mkdirp = require('mkdirp') var path = require('path') var test = require('./_test') var klaw = require('../') var fixtures = require('./fixtures') test('should not fire event on filtered items', function (t, testDir) { fixtures.forEach(function (f) { f = path.join(testDir, f) var dir = path.dirname(f) mkdirp.sync(dir) fs.writeFileSync(f, path.basename(f, path.extname(f))) }) var items = [] var filter = function (filepath) { return path.basename(filepath) !== 'a' } klaw(testDir, {filter: filter}) .on('data', function (item) { if (fs.lstatSync(item.path).isFile()) items.push(item.path) }) .on('error', t.end) .on('end', function () { var expected = ['c', 'b', 'a'] expected = expected.map(function (item) { return path.join(testDir, item) }) expected.unshift(testDir) t.ok(items.length < fixtures.length, 'we should see less items due to filter') t.end() }) }) node-klaw-1.3.1/tests/walk_mockfs.test.js000066400000000000000000000021501300370142500203440ustar00rootroot00000000000000var mkdirp = require('mkdirp') var mockfs = require('mock-fs') var os = require('os') var path = require('path') var test = require('tape') var klaw = require('../') var fixtures = require('./fixtures') var fs = mockfs.fs() test('walk directory on mockfs', function (t) { var testDir = path.join(os.tmpdir(), '/does/not/matter/in/mockfs') fixtures.forEach(function (f) { f = path.join(testDir, f) var dir = path.dirname(f) mkdirp.sync(dir, { fs: fs }) fs.writeFileSync(f, path.basename(f, path.extname(f))) }) t.plan(1) var items = [] klaw(testDir, { fs: fs }) .on('readable', function () { var item while ((item = this.read())) { items.push(item.path) } }) .on('error', t.end) .on('end', function () { items.sort() var expected = ['a', 'a/b', 'a/b/c', 'a/b/c/d.txt', 'a/e.jpg', 'h', 'h/i', 'h/i/j', 'h/i/j/k.txt', 'h/i/l.txt', 'h/i/m.jpg'] expected = expected.map(function (item) { return path.join(path.join(testDir, item)) }) expected.unshift(testDir) t.same(items, expected) t.end() }) }) node-klaw-1.3.1/tests/walk_path-sorter.test.js000066400000000000000000000030701300370142500213340ustar00rootroot00000000000000var fs = require('fs') var mkdirp = require('mkdirp') var path = require('path') var test = require('./_test') var klaw = require('../') var fixtures = require('./fixtures_path-sorter') test('should sort in reverse order [z -> a]', function (t, testDir) { fixtures.forEach(function (f) { f = path.join(testDir, f) var dir = path.dirname(f) mkdirp.sync(dir) fs.writeFileSync(f, path.basename(f, path.extname(f))) }) var items = [] var pathSorter = function (a, b) { return b > a } klaw(testDir, { pathSorter: pathSorter }) .on('data', function (item) { items.push(item.path) }) .on('error', t.end) .on('end', function () { var expected = ['c', 'b', 'a'] expected = expected.map(function (item) { return path.join(testDir, item) }) expected.unshift(testDir) t.same(items, expected) t.end() }) }) test('should sort in order [a -> z]', function (t, testDir) { fixtures.forEach(function (f) { f = path.join(testDir, f) var dir = path.dirname(f) mkdirp.sync(dir) fs.writeFileSync(f, path.basename(f, path.extname(f))) }) var items = [] var pathSorter = function (a, b) { return a > b } klaw(testDir, { pathSorter: pathSorter }) .on('data', function (item) { items.push(item.path) }) .on('error', t.end) .on('end', function () { var expected = ['a', 'b', 'c'] expected = expected.map(function (item) { return path.join(testDir, item) }) expected.unshift(testDir) t.same(items, expected) t.end() }) }) node-klaw-1.3.1/tests/walk_read-dir-error.test.js000066400000000000000000000013511300370142500217020ustar00rootroot00000000000000var fs = require('fs') var mkdirp = require('mkdirp') var path = require('path') var test = require('./_test') var klaw = require('../') test('walk directory, if error on readdir, at least end', function (t, testDir) { // simulate directory issue var unreadableDir = path.join(testDir, 'unreadable-dir') mkdirp.sync(unreadableDir) fs.chmodSync(unreadableDir, '0222') // not able to simulate on windows if (process.platform === 'win32') return t.end() t.plan(2) var items = [] klaw(testDir) .on('data', function (item) { items.push(item.path) }) .on('error', function (err) { t.true(err, 'caught error') }) .on('end', function () { t.true(true, 'be sure we end') t.end() }) })