How to properly convert char* into std::string? (issues while using expat / std::string(char*))

Question

Problem Description

I'm using Expat with a custom C++ wrapper, which I already tested on other projects. I'm running into problems, because the original data (c_str) is not converted to a std::string in the right way. This concers me, because I did not change the source of the wrapper.

It seems like the string gets null-terminated chars after this conversion:

onCharacterData( std::string( pszData, nLength ) ) // --> std::string( char* pszData)

How can I fix this?

Own expat wrapper

// Wrapper defines the class Expat and implements for example:
void XMLCALL Expat::CharacterDataHandler( void *pUserData, const XML_Char *pszData,
                                          int nLength )
{
  Expat* pThis = static_cast<Expat*>( pUserData );

  // XML_Char is char, therefore this call contains i.e.: std::string("hello", 5) 
  pThis->onCharacterData( std::string( pszData, nLength ) );
}

Custom parser

// Parser is defined as: class Parser : Expat
void Parser::onCharacterData(const std::string& data )
{
  // data is no longer char*, but a std::string.
  // It seems to contain \0 after each character which is wrong!

  // [...]
}

Character data within the expat wrapper (char*)

Character data within the parser (std::string)

Answer 1

Your pszData appears to be in some implementation-specific Unicode-derived format, where each "character" takes up two chars.

This means the source data is broken; it should have been a wchar_t buffer, perhaps.

Answer 2

It looks like the expat is using wide chars and/or UTF-16. Try using std::wstring on a way back.

EDIT I found in docs that it is using wchar_t if XML_UNICODE or XML_UNICODE_WCHAR_T macro are defined.

Answer 3

As others have pointed out it appears pszData is a multibyte character string. You should try using std::basic_string<XML_Char> in place of std::string or std::wstring. Use a typedef if that seems too verbose.

Of course, if XML_Char is neither a char nor a wchar_t you might have to provide a template specialization for std::char_traits

EDIT:
Some googling revealed that XML_Char is UTF-8; the library can be made to use UTF-16 if you define XML_UNICODE or XML_UNICODE_WCHAR_T.