Merging batched data in JS — reduce accumulator vs concat

Background: scraping school records via JSONP in the browser console. The lesson is not the site — it is that reduce is not always the right merge tool when your batches are not plain arrays.

After reading David Walsh on combining arrays and Chinese translations, I tried:

1
2
3
4
5
// g, b are arrays
g = b.reduce(function (coll, item) {
coll.push(item);
return coll;
}, g);

With larger batches this became very slow. The payload looked like:

Object { 1=[8], 2=[8], 3=[8], ... }

That is a map-like object, not a dense array. reduce walks array indices; pushing into a growing accumulator on huge lists is costly compared with a single concat.

Why concat worked — from MDN Array.prototype.concat:

  • Builds a new array from the receiver plus arguments.
  • For objects/arrays, stores references (shallow copy).
  • For primitives, copies values.

Know your shape: array batches → concat / spread; object maps → Object.values then merge, or push into a real array first.


2D de-duplication sketch (PHP):

1
2
3
4
5
6
7
8
9
10
11
function array_unique_deep($array2D) {
foreach ($array2D as $v) {
$v = join(',', $v);
$temp[] = $v;
}
$temp = array_unique($temp);
foreach ($temp as $k => $v) {
$temp[$k] = explode(',', $v);
}
return $temp;
}

Read N CSV lines (PHP):

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
function csv_get_lines($csvfile, $lines, $offset = 0) {
if (!$fp = fopen($csvfile, 'r')) {
return false;
}
$i = $j = 0;
while (false !== ($line = fgets($fp))) {
if ($i++ < $offset) {
continue;
}
break;
}
$data = array();
while (($j++ < $lines) && !feof($fp)) {
$_tmp = fgetcsv($fp);
if (!empty($_tmp[0])) {
$data[] = $_tmp;
}
}
fclose($fp);
return $data;
}
Questions? Leave a comment below or email: newbvirgil@gmail.com
Share   |  Updated: