libtorrent - storage_interface readv explanation

Question

I have implemented a custom storage interface in libtorrent as described in the help section here.

The storage_interface is working fine, although I can't figure out why readv is only called randomly while downloading a torrent. From my view the overriden virtual function readv should get called each time I call handle->read_piece in piece_finished_alert. It should read the piece for read_piece_alert?

The buffer is provided in read_piece_alert without getting notified in readv.

So the question is why it is called only randomly and why it's not called on a read_piece() call? Is my storage_interface maybe wrong?

The code looks like this:

struct temp_storage : storage_interface
{

        virtual int readv(file::iovec_t const* bufs, int num_bufs
            , int piece, int offset, int flags, storage_error& ec)
        {
            // Only called on random pieces while downloading a larger torrent 
            std::map<int, std::vector<char> >::const_iterator i = m_file_data.find(piece);
                         if (i == m_file_data.end()) return 0;
                         int available = i->second.size() - offset;
                         if (available <= 0) return 0;
                         if (available > num_bufs) available = num_bufs;
                         memcpy(&bufs, &i->second[offset], available); 
                         return available;
        }
        virtual int writev(file::iovec_t const* bufs, int num_bufs
            , int piece, int offset, int flags, storage_error& ec)
        {
            std::vector<char>& data = m_file_data[piece];
                         if (data.size() < offset + num_bufs) data.resize(offset + num_bufs);
                         std::memcpy(&data[offset], bufs, num_bufs);
                         return num_bufs;
        }
        virtual bool has_any_file(storage_error& ec) { return false; }
        virtual ... 
        virtual ...
}

Intialized with

storage_interface* temp_storage_constructor(storage_params const& params)
{
    printf("NEW INTERFACE\n");
    return new temp_storage(*params.files);
}
p.storage = &temp_storage_constructor;

The function below sets up alerts and invokes read_piece on each completed piece.

  while(true) {

      std::vector<alert*> alerts;
      s.pop_alerts(&alerts);

          for (alert* i : alerts)
          {
              switch (i->type()) {

                      case read_piece_alert::alert_type:
                      {
                              read_piece_alert* p = (read_piece_alert*)i;
                              if (p->ec) {
                                      // read_piece failed
                                      break;
                              }
                         // piece buffer, size is provided without readv
                         // notification after invoking read_piece in piece_finished_alert
                         break;
                      }
                      case piece_finished_alert::alert_type: {
                           piece_finished_alert* p = (piece_finished_alert*)i;               

                       p->handle.read_piece(p->piece_index);
                    // Once the piece is finished, we read it to obtain the buffer in read_piece_alert.  
                       break;
                  }
                  default:
                          break;
                  }
              }
              Sleep(100);
          }

do you have the disk cache enabled? If so, the data you return from the first call to readv() is probably cached and reused the next time someone wants it — Arvid
By the way: I plan to run multiple sessions, each session in a different thread. Should I take care of something or turn off any features when doing that? (i. e. ban own IPs, so they don't connect to each other) — user3606329

user3606329 user3606329 · Accepted Answer · 2017-08-21T03:40:45

I will answer my own question. As Arvid said in the comments: readv was not invoked because of caching. Setting settings_pack::use_read_cache to false will invoke readv always.

libtorrent - storage_interface readv explanation

1 Answers