Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.c++ > #10057 > unrolled thread

Multiple files in one

Back to article view | Back to comp.lang.c++

Contents

  Multiple files in one fryme <oleg.fry@gmail.com> - 2011-09-12 11:44 -0700
    Re: Multiple files in one Alexander Bartolich <alexander.bartolich@gmx.at> - 2011-09-12 17:31 -0700
    Re: Multiple files in one Paavo Helde <myfirstname@osa.pri.ee> - 2011-09-12 17:32 -0700
    Re: Multiple files in one red floyd <no.spam.here@its.invalid> - 2011-09-12 17:32 -0700
    Re: Multiple files in one Daniel Krügler <daniel.kruegler@googlemail.com> - 2011-09-12 17:32 -0700
    Re: Multiple files in one Jorgen Grahn <grahn+nntp@snipabacken.se> - 2011-09-12 17:34 -0700
    Re: Multiple files in one Ulrich Eckhardt <ulrich.eckhardt@dominolaser.com> - 2011-09-13 10:51 -0700
    Re: Multiple files in one Daniel James <daniel@me.invalid> - 2011-09-13 10:53 -0700

#10057 — Multiple files in one

I'm trying to create my own file format. I want to store image file
and some text description in that file.
File format will be something like that:
image_file_size
image_data
desctiption_file_size
description_data

But without '\n' symbols.
For that purpose i'm using std::ios::binary. Here is some code,
describes that process (it is sketch, not last variant):
Writing my file.

long long image_length, desctiption_length;

std::fstream m_out(output_file_path, std::ios::out |
std::ios::binary);
std::ifstream input_image(m_image_file_path.toUtf8().data());

input_image.seekg(0, std::ios::end);
image_length = input_image.tellg();
input_image.seekg(0, std::ios::beg);

// writing image length to output file
m_out.write( (const char *)&image_length, sizeof(long long) );

char *buffer = new char[image_length];
input.read(buffer, image_length);

// writing image to file
m_out.write(buffer, image_length);

// writing description file the same way
// ...

Reading my file.

std::fstream m_in(m_file_path.toUtf8().data(), std::ios::in );

long long xml_length, image_length;

m_in.seekg(0, std::ios::beg);
m_in.read((char *)&image_length, sizeof(long long));
m_in.seekg(sizeof(long long));

char *buffer = new char[image_length];
m_in.read(buffer, image_length );

std::fstream fs("E:\\Temp\\out.jpg");
fs.write(buffer, image_length);

And now image (E:\Temp\out.jpg) is broken. I'm watching it in hex
editor and there are some extra bits.

Can somebody help me and tell what i'm doing wrong?


-- 
      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

[toc] | [next] | [standalone]

#10061

fryme wrote:
> [...]
> For that purpose i'm using std::ios::binary. Here is some code,
> describes that process (it is sketch, not last variant):

You expect us to find bugs in a "sketch"?

> [...]
> std::fstream m_out(output_file_path, std::ios::out |
> std::ios::binary);
> std::ifstream input_image(m_image_file_path.toUtf8().data());
                                              ^^^^^^^^^^^^^^^
All right, you are using Qt.

> input_image.seekg(0, std::ios::end);
> image_length = input_image.tellg();
> input_image.seekg(0, std::ios::beg);
>
> // writing image length to output file
> m_out.write( (const char *)&image_length, sizeof(long long) );
>
> char *buffer = new char[image_length];
> input.read(buffer, image_length);
  ^^^^^
I guess you mean "input_image", not "input".

> [...]
> std::fstream m_in(m_file_path.toUtf8().data(), std::ios::in );

There is no "std::ios::binary" here. Is this just a detail missing
from the "sketch"?

> m_in.seekg(0, std::ios::beg);
> m_in.read((char *)&image_length, sizeof(long long));
> m_in.seekg(sizeof(long long));

As far as I know, the second argument of "seekg" has no default value.

> [...]
> std::fstream fs("E:\\Temp\\out.jpg");

Again no "std::ios::binary" here.

Please, create working example code.

-- 
host -t mx moderators.isc.org


      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

[toc] | [prev] | [next] | [standalone]

#10062

fryme <oleg.fry@gmail.com> wrote in news:a51089c8-6be0-4fa7-9237-
39f686aa99a8@z18g2000yqb.googlegroups.com:

> I'm trying to create my own file format. I want to store image file
> and some text description in that file.

Strongly discouraged as this file can be read only by your own software,
not a good idea. What's wrong with writing two files, or writing the
description in the EXIF section of jpeg or whatever?

> File format will be something like that:
> image_file_size
> image_data
> desctiption_file_size
> description_data
> 
> But without '\n' symbols.
> For that purpose i'm using std::ios::binary. Here is some code,
> describes that process (it is sketch, not last variant):
> Writing my file.
> 
> long long image_length, desctiption_length;
> 
> std::fstream m_out(output_file_path, std::ios::out |
> std::ios::binary);
> std::ifstream input_image(m_image_file_path.toUtf8().data());

binary flag missing

> 
> input_image.seekg(0, std::ios::end);
> image_length = input_image.tellg();
> input_image.seekg(0, std::ios::beg);
> 
> // writing image length to output file
> m_out.write( (const char *)&image_length, sizeof(long long) );

sizeof(long long) is platform-dependent, you should be using something
more fixed like std::uint64_t, for example. Even then the endianness
would be platform-dependent.

> 
> char *buffer = new char[image_length];
> input.read(buffer, image_length);
> 
> // writing image to file
> m_out.write(buffer, image_length);
> 
> // writing description file the same way
> // ...
> 
> Reading my file.
> 
> std::fstream m_in(m_file_path.toUtf8().data(), std::ios::in );

binary flag missing

> 
> long long xml_length, image_length;
> 
> m_in.seekg(0, std::ios::beg);
> m_in.read((char *)&image_length, sizeof(long long));
> m_in.seekg(sizeof(long long));
> 
> char *buffer = new char[image_length];
> m_in.read(buffer, image_length );
> 
> std::fstream fs("E:\\Temp\\out.jpg");

binary flag missing

> fs.write(buffer, image_length);
> 
> And now image (E:\Temp\out.jpg) is broken. I'm watching it in hex
> editor and there are some extra bits.
> 
> Can somebody help me and tell what i'm doing wrong?

Using C++ streams does not help you here in any way if your file is
binary. Also, AFAIK the stream read()/write() can change the data even if
the stream is in the binary mode, the correct way would be to use the
stream buffer read/write functionality IIRC. I would personally not
bother with this and used fopen() and friends instead (and not forgetting
binary flags!).

hth
Paavo


-- 
      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

[toc] | [prev] | [next] | [standalone]

#10063

On 9/12/2011 11:44 AM, fryme wrote:
> [redacted]
> long long image_length, desctiption_length;
>
> std::fstream m_out(output_file_path, std::ios::out |
> std::ios::binary);
> std::ifstream input_image(m_image_file_path.toUtf8().data());
>
> [redacted]
> // ...
>
> Reading my file.
>
> std::fstream m_in(m_file_path.toUtf8().data(), std::ios::in );
std::fstream m_in(m_file_path.toUtf8().data(), std::ios::in |
                                               std::ios::binary);


-- 
      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

[toc] | [prev] | [next] | [standalone]

#10064

Am 12.09.2011 20:44, schrieb fryme:
> I'm trying to create my own file format. I want to store image file
> and some text description in that file.

[..]

> For that purpose i'm using std::ios::binary. Here is some code,
> describes that process (it is sketch, not last variant):
> Writing my file.
>
> long long image_length, desctiption_length;
>
> std::fstream m_out(output_file_path, std::ios::out |
> std::ios::binary);
> std::ifstream input_image(m_image_file_path.toUtf8().data());

[..]

> Reading my file.
>
> std::fstream m_in(m_file_path.toUtf8().data(), std::ios::in );

It seems to me that both input_image and m_in need to be opened as
binary streams as well.

HTH & Greetings from Bremen,

Daniel Krügler


-- 
      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

[toc] | [prev] | [next] | [standalone]

#10065

["Followup-To:" header set to comp.lang.c++.]
On Mon, 2011-09-12, fryme wrote:
> I'm trying to create my own file format. I want to store image file
> and some text description in that file.

Probably unwise, but that's not relevant to the question.

> File format will be something like that:
> image_file_size
> image_data
> desctiption_file_size
> description_data
>
> But without '\n' symbols.
> For that purpose i'm using std::ios::binary. Here is some code,
> describes that process (it is sketch, not last variant):
> Writing my file.
>
> long long image_length, desctiption_length;
>
> std::fstream m_out(output_file_path, std::ios::out |
> std::ios::binary);
> std::ifstream input_image(m_image_file_path.toUtf8().data());
>
> input_image.seekg(0, std::ios::end);
> image_length = input_image.tellg();
> input_image.seekg(0, std::ios::beg);
>
> // writing image length to output file
> m_out.write( (const char *)&image_length, sizeof(long long) );

Ugh! Please don't -- this introduces endianness bugs in your files.
Instead specify in your file format definition exactly *what* the
"image_file_size" field means and how its formatted, and the code will
follow.

For example:

  image_file_size: 6 octets, MSB first, containing the length (in
  octets) of the immediately following image file data. May not be
  zero.

I didn't read the rest. Sorry.

/Jorgen

-- 
  // Jorgen Grahn <grahn@  Oo  o.   .     .
\X/     snipabacken.se>   O  o   .


      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

[toc] | [prev] | [next] | [standalone]

#10080

fryme wrote:
> I'm trying to create my own file format. I want to store image file
> and some text description in that file.

Take a look at the approach that many recent programs take, i.e. using a
simple archive of files. See for example
http://tanguy.ortolo.eu/blog/article24/repack-zip-container, where the
approach is described a bit. Note that this allows use of standard tools for
manipulation.


> i'm using std::ios::binary.

This flag does probably not do what you expect it to do. In order to output
preformatted bytes or read bytes in order to parse them yourself, it is
vital though.


> std::fstream m_out(output_file_path, std::ios::out |
> std::ios::binary);

Use ofstream instead, then you don't have to specify the "ios::out" and you
get some checking that you don't accidentally do input from the compiler.


> char *buffer = new char[image_length];

If you find yourself using the vector new, you are probably doing something
very advanced or a beginner's mistake. In this case, I suspect the latter,
and the remedy is to use a std::vector. Further looking at your code, you
continue the mistake and fail to free the allocated data.


> std::fstream m_in(m_file_path.toUtf8().data(), std::ios::in );
> 
> long long xml_length, image_length;
> 
> m_in.seekg(0, std::ios::beg);

Useless, you are at the beginning of the file.

> m_in.read((char *)&image_length, sizeof(long long));

Two things:
1. Use "sizeof image_length" to make clear that it is that variable's size
that you want to read. Of course, this is non-portable, as others already
pointed out.
2. Use of C-style casts is dangerous, as they suppress any warning. In this
case though, there is no real alternative except two static casts, because
C++ iostreams are for writing texts and not for writing preformatted data.
They can be used for that, but typically that results in awkward code like
the above.

> m_in.seekg(sizeof(long long));

Also redundant, since reading advances the read position automatically.


Good luck!

Uli

-- 
Domino Laser GmbH
Geschäftsführer: Thorsten Föcking, Amtsgericht Hamburg HR B62 932


      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

[toc] | [prev] | [next] | [standalone]

#10081

In article
<a51089c8-6be0-4fa7-9237-39f686aa99a8@z18g2000yqb.googlegroups.com>,
Fryme wrote:
> I'm trying to create my own file format.

Others have already questioned the wisdom of such an approach.

Rather than inventing a whole new file format that is peculiar to your
application, why not use a standard container format (such as the
zipfile format) and write your image file and a descriptive text file
to that?

Not only will the resultant file be in a recognizably standard format,
but the process of reading and writing the file can be simplified,
relative to your home-grown approach, by the use of one of many freely
available libraries.

-- 
Regards,
  Daniel.
[Lest there be any confusion I am NOT the Boost maintainer of the same
name]


      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.c++

csiph-web