R/Rcpp: How can I convert 2-byte Hex values to integers efficiently?

Question

I want to convert a raw type vector that contains 2-byte hex numbers (little-endian) into a vector of integers in R (e.g. ff ff -> 0xffff = 65535). One way to do this is to extract even and odd elements from a raw vector, and paste into characters, and then convert into integers as below:

> a <- c(as.raw(255), as.raw(254), as.raw(253), as.raw(252))
> a
[1] ff fe fd fc
> even_elem <- a[seq(2,length(a),2)]
> odd_elem <- a[seq(1,length(a),2)]
> as.integer(paste0("0x", even_elem, odd_elem))
[1] 65279 64765
> c(0xfeff, 0xfcfd)
[1] 65279 64765

The problem is that I want to do this for a vector with >10^8 elements. If I do this with the approach above, it takes minutes. I wanted something more efficient. I thought I could try to use Rcpp to speed this up, so I wrote a piece of cpp code (I'm new to Rcpp/c++),

#include <Rcpp.h>
using namespace Rcpp;
// [[Rcpp::export]]
IntegerVector raw2intC(CharacterVector vec){
  int n = vec.size();
  int m;
  Rcpp::IntegerVector x(n/2);
  for (int i = 0; i < n/2; i++) {
    std::string h1 = Rcpp::as<std::string>(vec[i*2]);
    std::string h2 = Rcpp::as<std::string>(vec[i*2 + 1]);
    h2 += h1;
    std::stringstream ss;
    ss << std::hex << h2;
    ss >> m;
    x[i] = m;
  }
return(x);
}

and an R script.

raw2intR <- function(obj){
  val <- raw2intC(obj)
  val
}

This Rcpp code worked, and the result of microbenchmark looked encouraging.

> microbenchmark(raw2intR(a), as.integer(paste0("0x", even_elem, odd_elem)))
Unit: microseconds
expr    min      lq     mean  median      uq     max
raw2intR(a)  4.953  5.9130  7.68194  7.4800  8.4585  42.658
as.integer(...) 36.297 40.4275 44.06539 42.8565 44.9420 147.110
> identical(raw2intR(a), as.integer(paste0("0x", even_elem, odd_elem)))
[1] TRUE

However, when tested with a larger vector, there was not much difference in execution time between the R and Rcpp solutions. In fact, the R solution was slightly faster.

> b <- raw(1000000)
> even_elem <- b[seq(2,length(a),2)]
> odd_elem <- b[seq(1,length(a),2)]
> microbenchmark(raw2intR(b), as.integer(paste0("0x", even_elem, odd_elem)), times=10)
Unit: milliseconds
expr      min       lq     mean   median       uq
raw2intR(b) 309.4139 309.7920 316.6345 313.6219 321.5353
as.integer(...) 274.3523 279.6978 287.5415 288.1744 291.1616
> identical(raw2intR(b), as.integer(paste0("0x", even_elem, odd_elem)))
[1] TRUE

How can this task be sped up? I'm hoping to achieve 10x improvement.

Thanks for your advice.

MrFlick MrFlick · Accepted Answer · 2016-09-06T14:53:00

Rather than building strings to convert back to a number, you can just tell R to interpret those raw values as integers directly with readBin. For example

a <- as.raw(c(255, 254, 253, 252))
readBin(a, "integer", n=length(a)/2, size=2, signed=FALSE)
# [1] 65279 64765

R/Rcpp: How can I convert 2-byte Hex values to integers efficiently?

1 Answers