Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.c > #111740 > unrolled thread

How to determine if file is binary or text?

Back to article view | Back to comp.lang.c

Contents

  How to determine if file is binary or text? DFS <nospam@dfs.com> - 2017-06-07 14:40 -0400
    Re: How to determine if file is binary or text? GOTHIER Nathan <nathan.gothier@gmail.com> - 2017-06-07 20:46 +0200
    Re: How to determine if file is binary or text? bartc <bc@freeuk.com> - 2017-06-07 19:58 +0100
    Re: How to determine if file is binary or text? scott@slp53.sl.home (Scott Lurndal) - 2017-06-07 19:05 +0000
    Re: How to determine if file is binary or text? supercat@casperkitty.com - 2017-06-07 12:10 -0700
    Re: How to determine if file is binary or text? "James R. Kuyper" <jameskuyper@verizon.net> - 2017-06-07 16:25 -0400
    Re: How to determine if file is binary or text? Jorgen Grahn <grahn+nntp@snipabacken.se> - 2017-06-07 21:44 +0000
      Re: How to determine if file is binary or text? DFS <nospam@dfs.com> - 2017-06-07 18:14 -0400
        Re: How to determine if file is binary or text? GOTHIER Nathan <nathan.gothier@gmail.com> - 2017-06-08 00:23 +0200
          Re: How to determine if file is binary or text? Robert Wessel <robertwessel2@yahoo.com> - 2017-06-08 00:11 -0500
            Re: How to determine if file is binary or text? GOTHIER Nathan <nathan.gothier@gmail.com> - 2017-06-08 15:14 +0200
              Re: How to determine if file is binary or text? Robert Wessel <robertwessel2@yahoo.com> - 2017-06-08 11:06 -0500
            Re: How to determine if file is binary or text? raltbos@xs4all.nl (Richard Bos) - 2017-06-17 18:28 +0000
        Re: How to determine if file is binary or text? Ben Bacarisse <ben.usenet@bsb.me.uk> - 2017-06-07 23:30 +0100
          Re: How to determine if file is binary or text? DFS <nospam@dfs.com> - 2017-06-07 20:26 -0400
            Re: How to determine if file is binary or text? bartc <bc@freeuk.com> - 2017-06-08 01:30 +0100
              Re: How to determine if file is binary or text? DFS <nospam@dfs.com> - 2017-06-07 23:50 -0400
                Re: How to determine if file is binary or text? mark.bluemel@gmail.com - 2017-06-08 02:13 -0700
                  Re: How to determine if file is binary or text? mark.bluemel@gmail.com - 2017-06-08 04:05 -0700
                  Re: How to determine if file is binary or text? DFS <nospam@dfs.com> - 2017-06-08 09:21 -0400
                    Re: How to determine if file is binary or text? Ike Naar <ike@iceland.freeshell.org> - 2017-06-08 14:19 +0000
                      Re: How to determine if file is binary or text? DFS <nospam@dfs.com> - 2017-06-08 12:07 -0400
                        Re: How to determine if file is binary or text? scott@slp53.sl.home (Scott Lurndal) - 2017-06-08 16:14 +0000
                          Re: How to determine if file is binary or text? DFS <nospam@dfs.com> - 2017-06-08 12:37 -0400
                            Re: How to determine if file is binary or text? scott@slp53.sl.home (Scott Lurndal) - 2017-06-08 16:54 +0000
                              Re: How to determine if file is binary or text? DFS <nospam@dfs.com> - 2017-06-08 13:14 -0400
                              An old version of what??? (Was: How to determine if file is binary or text?) gazelle@shell.xmission.com (Kenny McCormack) - 2017-06-09 17:13 +0000
                Re: How to determine if file is binary or text? Ben Bacarisse <ben.usenet@bsb.me.uk> - 2017-06-08 11:15 +0100
            Re: How to determine if file is binary or text? scott@slp53.sl.home (Scott Lurndal) - 2017-06-08 12:49 +0000
            Re: How to determine if file is binary or text? Jorgen Grahn <grahn+nntp@snipabacken.se> - 2017-06-08 14:01 +0000
            Re: How to determine if file is binary or text? Michael Forsythe <forsythe@example.com> - 2017-06-08 15:33 +0000
              Re: How to determine if file is binary or text? DFS <nospam@dfs.com> - 2017-06-08 18:43 -0400
                Re: How to determine if file is binary or text? jak <please@nospam.tnx> - 2017-06-09 08:46 +0200
        Re: How to determine if file is binary or text? "James R. Kuyper" <jameskuyper@verizon.net> - 2017-06-07 19:23 -0400
        Re: How to determine if file is binary or text? Barry Schwarz <schwarzb@dqel.com> - 2017-06-07 16:50 -0700
        Re: How to determine if file is binary or text? "Pascal J. Bourguignon" <pjb@informatimago.com> - 2017-06-08 15:51 +0200
    Re: How to determine if file is binary or text? Joe Pfeiffer <pfeiffer@cs.nmsu.edu> - 2017-06-07 17:17 -0600
      Re: How to determine if file is binary or text? Keith Thompson <kst-u@mib.org> - 2017-06-07 19:21 -0700
        Re: How to determine if file is binary or text? Joe Pfeiffer <pfeiffer@cs.nmsu.edu> - 2017-06-07 21:37 -0600
    Re: How to determine if file is binary or text? Noob <root@127.0.0.1> - 2017-06-08 08:40 +0200
    Re: How to determine if file is binary or text? gordonb.t9rom@burditt.org (Gordon Burditt) - 2017-06-08 12:03 -0500

Page 2 of 3 — ← Prev page 1 [2] 3  Next page →

#111816

On 2017-06-08, DFS <nospam@dfs.com> wrote:
> int is_textfile(char *fname)
> {
> 	//run the GNUWin32 file command
> 	char psBuffer[128];
> 	FILE *pPipe;
>
> 	char cmd[] = {0};

The array 'cmd' is only 1 byte long.

> 	sprintf(cmd,"file %s",fname);

Yet you try to store (strlen("file ") + strlen(fname) + 1) bytes into it.
That won't fit.

> 		
> 	if((pPipe = _popen(cmd,"rt")) == NULL) {exit(1);}
> 	while(fgets(psBuffer, 128, pPipe))
> 	{
> 	   //how to check for 'ASCII'?
> 	   printf(psBuffer);    //this prints the stream
> 	}  		
> 	if (feof(pPipe)) {_pclose(pPipe);}
> 	return 0;
> }

[toc] | [prev] | [next] | [standalone]

#111829

On 6/8/2017 10:19 AM, Ike Naar wrote:
> On 2017-06-08, DFS <nospam@dfs.com> wrote:
>> int is_textfile(char *fname)
>> {
>> 	//run the GNUWin32 file command
>> 	char psBuffer[128];
>> 	FILE *pPipe;
>>
>> 	char cmd[] = {0};
> 
> The array 'cmd' is only 1 byte long.
> 
>> 	sprintf(cmd,"file %s",fname);
> 
> Yet you try to store (strlen("file ") + strlen(fname) + 1) bytes into it.
> That won't fit.


It works fine.

printf("%s\n",cmd) prints 'file filename', and _popen executes it no 
problem.

[toc] | [prev] | [next] | [standalone]

#111830

DFS <nospam@dfs.com> writes:
>On 6/8/2017 10:19 AM, Ike Naar wrote:
>> On 2017-06-08, DFS <nospam@dfs.com> wrote:
>>> int is_textfile(char *fname)
>>> {
>>> 	//run the GNUWin32 file command
>>> 	char psBuffer[128];
>>> 	FILE *pPipe;
>>>
>>> 	char cmd[] = {0};
>> 
>> The array 'cmd' is only 1 byte long.
>> 
>>> 	sprintf(cmd,"file %s",fname);
>> 
>> Yet you try to store (strlen("file ") + strlen(fname) + 1) bytes into it.
>> That won't fit.
>
>
>It works fine.

No, it doesn't.  It corrupts the stack.

>
>printf("%s\n",cmd) prints 'file filename', and _popen executes it no 
>problem.

You just don't see the corruption _in this particular case_.  If the
stack grew in the other direction (which it does on some processors),
or if you had declared anything after cmd, you would likely have wiped
out that data or the return address on the stack.

You should _NEVER_ use sprintf.  Instead, use snprintf - e.g.

#define MAXIMUM_COMMAND_LENGTH 512

  char cmd[MAXIMUM_COMMAND_LENGTH];

  snprintf(cmd, sizeof(cmd), "file %s", file);

  And check the return code from snprintf to make sure you
didn't get truncated output because sizeof(cmd) is less than
the space required for the given format specifier plus
argument(s).

[toc] | [prev] | [next] | [standalone]

#111835

On 6/8/2017 12:14 PM, Scott Lurndal wrote:
> DFS <nospam@dfs.com> writes:
>> On 6/8/2017 10:19 AM, Ike Naar wrote:
>>> On 2017-06-08, DFS <nospam@dfs.com> wrote:
>>>> int is_textfile(char *fname)
>>>> {
>>>> 	//run the GNUWin32 file command
>>>> 	char psBuffer[128];
>>>> 	FILE *pPipe;
>>>>
>>>> 	char cmd[] = {0};
>>>
>>> The array 'cmd' is only 1 byte long.
>>>
>>>> 	sprintf(cmd,"file %s",fname);
>>>
>>> Yet you try to store (strlen("file ") + strlen(fname) + 1) bytes into it.
>>> That won't fit.
>>
>>
>> It works fine.
> 
> No, it doesn't.  It corrupts the stack.
> 
>>
>> printf("%s\n",cmd) prints 'file filename', and _popen executes it no
>> problem.
> 
> You just don't see the corruption _in this particular case_.  If the
> stack grew in the other direction (which it does on some processors),
> or if you had declared anything after cmd, you would likely have wiped
> out that data or the return address on the stack.
> 
> You should _NEVER_ use sprintf.  Instead, use snprintf - e.g.
> 
> #define MAXIMUM_COMMAND_LENGTH 512
> 
>    char cmd[MAXIMUM_COMMAND_LENGTH];
> 
>    snprintf(cmd, sizeof(cmd), "file %s", file);
> 
>    And check the return code from snprintf to make sure you
> didn't get truncated output because sizeof(cmd) is less than
> the space required for the given format specifier plus
> argument(s).


Thanks.  I just approximated your example and it works fine.


Test:

char cmd[5];
if ((snprintf(cmd,sizeof(cmd),"file %s",fname)) == -1)
{printf("max cmd length insufficient\n");}

[toc] | [prev] | [next] | [standalone]

#111838

DFS <nospam@dfs.com> writes:
>On 6/8/2017 12:14 PM, Scott Lurndal wrote:
>> DFS <nospam@dfs.com> writes:

>> 
>> You should _NEVER_ use sprintf.  Instead, use snprintf - e.g.
>> 
>> #define MAXIMUM_COMMAND_LENGTH 512
>> 
>>    char cmd[MAXIMUM_COMMAND_LENGTH];
>> 
>>    snprintf(cmd, sizeof(cmd), "file %s", file);
>> 
>>    And check the return code from snprintf to make sure you
>> didn't get truncated output because sizeof(cmd) is less than
>> the space required for the given format specifier plus
>> argument(s).
>
>
>Thanks.  I just approximated your example and it works fine.
>
>
>Test:
>
>char cmd[5];
>if ((snprintf(cmd,sizeof(cmd),"file %s",fname)) == -1)
>{printf("max cmd length insufficient\n");}
>

$ man 3 snprintf
  ...
   Return value
       Upon successful return, these functions return the number of
       characters printed (excluding the null byte used to end output to strings).

       The functions snprintf() and vsnprintf() do not write more than
       size bytes (including the terminating null byte ('\0')).  If the output
       was truncated due to this limit  then  the
       return  value  is  the  number  of characters (excluding the
       terminating null byte) which would have been written to the final
       string if enough space had been available.  Thus, a
       return value of size or more means that the output was truncated.

       If an output error is encountered, a negative value is returned.

NOTES:
       The glibc implementation of the functions snprintf() and vsnprintf()
       conforms to the C99 standard, that is, behaves as described above,
       since  glibc  version  2.1.   Until  glibc
       2.0.6 they would return -1 when the output was truncated.


Looks like you have an old version of glibc.

[toc] | [prev] | [next] | [standalone]

#111841

On 6/8/2017 12:54 PM, Scott Lurndal wrote:

> NOTES:
>         The glibc implementation of the functions snprintf() and vsnprintf()
>         conforms to the C99 standard, that is, behaves as described above,
>         since  glibc  version  2.1.   Until  glibc
>         2.0.6 they would return -1 when the output was truncated.
> 
> 
> Looks like you have an old version of glibc.

Windows 8.1
tcc 0.9.26 (Feb 2013)

[toc] | [prev] | [next] | [standalone]

#111895 — An old version of what??? (Was: How to determine if file is binary or text?)

In article <wjf_A.28954$dS7.27122@fx43.iad>,
Scott Lurndal <slp53@pacbell.net> wrote:
...
>Looks like you have an old version of glibc.

Given that he is working on Windows, he probably doesn't have *any* version
of glibc...

-- 
The motto of the GOP "base": You can't *be* a billionaire, but at least you
can vote like one.

[toc] | [prev] | [next] | [standalone]

#111803

DFS <nospam@dfs.com> writes:

> On 6/7/2017 8:30 PM, bartc wrote:
>> On 08/06/2017 01:26, DFS wrote:
>>> On 6/7/2017 6:30 PM, Ben Bacarisse wrote:
>>>> DFS <nospam@dfs.com> writes:
>>>>
>>>>> On 6/7/2017 5:44 PM, Jorgen Grahn wrote:
>>>>>> On Wed, 2017-06-07, DFS wrote:
>>>>>>> binary, such as .xls or .exe or Linux ELF executable
>>>>>>> text, such as an ascii .txt file
>>>>>>
>>>>>> Why do you want to find out?  Few programs have a need for that.
>>>>>
>>>>> validate file inputs to my program.
>>>>>
>>>>> If someone tries to pass a binary file I want to sniff it out.
>>>>
>>>> Now that's a very helpful insight.  
>
> Sarcasm?

Not at all.  You want to stop some kinds of file being passed to your
program, but that will be for a specific reason.  Whatever it is that
makes these files difficult for you program is exactly what you should
be testing for.

<snip>
>> To capture actual text output (that would be printed in the screen), try:
>>
>>   system("command params >tempfile");      // stdout output
>>   system("command params 2>tempfile");     // stderr output
>>
>> But then you have to open "tempfile" and print it out. Maybe another command:
>>
>>   system("type tempfile");
>>
>> if on Windows.
>
> Very kludgey.

Yes, very.  And you will still have to decide which of the thousands of
file types reported by 'file' you will or will not accept which will
involve parsing the output from 'file' and will tie your program to the
specific version of file (and its 'magic' files) that you are using.

To advance this thread, say what your program does with the input file
and why some kinds of file are not acceptable.  I'm sure you will then
get better advice on how to spot those files.

<snip>
-- 
Ben.

[toc] | [prev] | [next] | [standalone]

#111809

DFS <nospam@dfs.com> writes:
>On 6/7/2017 6:30 PM, Ben Bacarisse wrote:

>I'm on Windows, but I installed the GNUWin32 port which includes 
>'file.exe' (the Linux 'file' command).
>
>
>int is_textfile(char *fname)
>{
>	char cmd[] = {0};
>	sprintf(cmd,"file %s",fname);
>	char *ftype = {system(cmd)};
>	printf("filetype = %s\n",ftype);
>}
>
>
>output is
>
>filename.txt; ASCII English text, with very long lines, with CRLF line 
>terminators
>filetype = (null)
>
>
>So it runs the 'file' command and prints it, but it isn't getting 
>captured in the ftype variable.
>
>I tried _popen(cmd,"r") but the same thing happens: the value from the 
>file command isn't captured.

After you popen the command, you need to read the file descriptor
returned by the popen command to get the output from the file command.

[toc] | [prev] | [next] | [standalone]

#111815

On Thu, 2017-06-08, DFS wrote:
...
> I'm on Windows, but I installed the GNUWin32 port which includes 
> 'file.exe' (the Linux 'file' command).
>
> int is_textfile(char *fname)
> {
> 	char cmd[] = {0};
> 	sprintf(cmd,"file %s",fname);
> 	char *ftype = {system(cmd)};
> 	printf("filetype = %s\n",ftype);
> }
>
> output is
>
> filename.txt; ASCII English text, with very long lines, with CRLF line 
> terminators
> filetype = (null)

You may want to run file with the option to output a MIME type
instead, e.g. "text/plain" or "binary/whatever". It's easier to
parse.  Or you could use libmagic directly I suppose.

But file's best guess about a file may or may not be enough for you.
You might even have different definitions of "binary".

/Jorgen

-- 
  // Jorgen Grahn <grahn@  Oo  o.   .     .
\X/     snipabacken.se>   O  o   .

[toc] | [prev] | [next] | [standalone]

#111823

DFS <nospam@dfs.com> wrote:

[snip]

> I'm on Windows, but I installed the GNUWin32 port which includes 
> 'file.exe' (the Linux 'file' command).
> 
> 
> int is_textfile(char *fname)
> {
>        char cmd[] = {0};
>        sprintf(cmd,"file %s",fname);
>        char *ftype = {system(cmd)};
>        printf("filetype = %s\n",ftype);
> }
> 
> 
> output is
> 
> filename.txt; ASCII English text, with very long lines, with CRLF line 
> terminators
> filetype = (null)
> 
> 
> So it runs the 'file' command and prints it, but it isn't getting 
> captured in the ftype variable.

Others have mentioned popen, and you've already discovered the pain
that comes with that. At least on my systems, file comes with the
helpful libmagic library. If you can link against that:

        #include <stdio.h>
        #include <string.h>
        
        #include <magic.h>
        
        static magic_t mime_cookie;
        
        int is_textfile(const char *fname)
        {
                return !strcmp(magic_file(mime_cookie, fname),
                               "text/plain");
        }
        
        int main(int argc, char **argv)
        {
                if (!(mime_cookie = magic_open(MAGIC_MIME_TYPE))) {
                        perror("magic_open");
                        return -1;
                }
        
                if (magic_load(mime_cookie, 0) < 0) {
                        perror("magic_load");
                        return -1;
                }
        
                if (argc != 2) {
                        printf("Usage: %s /path/to/file\n", argv[0]);
                        return -1;
                }
        
                if (is_textfile(argv[1])) {
                        printf("Yup\n");
                } else {
                        printf("Nope\n");
                }
        
                return 0;
        }

        $ cc -std=c99 -o a a.c -lmagic
        $ ./a /dev/kmem
        Nope
        $ ./a c89-draft.txt
        Yup
        $ ./a n1256.pdf
        Nope
        $ ./a UTF-8-demo.txt
        Yup

The library also has magic_buffer in case you've already loaded the
file into memory.

Of course, as others have already said, you probably want to do
some processing with the file after you've loaded it (treat it as
a configuration file, extract all email addresses, treat it as a
sequence of SAN chess moves, whatever). If so, you're probably
better off treating the file as a bunch of bytes and attempting to
specifically validate for whatever you're going to use the file
for.

[toc] | [prev] | [next] | [standalone]

#111868

On 6/8/2017 11:33 AM, Michael Forsythe wrote:


> Others have mentioned popen, and you've already discovered the pain
> that comes with that. At least on my systems, file comes with the
> helpful libmagic library. If you can link against that:
> 
<snip code>


> The library also has magic_buffer in case you've already loaded the
> file into memory.
> 
> Of course, as others have already said, you probably want to do
> some processing with the file after you've loaded it (treat it as
> a configuration file, extract all email addresses, treat it as a
> sequence of SAN chess moves, whatever). If so, you're probably
> better off treating the file as a bunch of bytes and attempting to
> specifically validate for whatever you're going to use the file
> for.

Thanks for the suggestions.  I got the _popen/pipes working on Windows, 
but I'm making my code cross-platform (Windows and Linux), and the above 
looks like more fun on Linux.

[toc] | [prev] | [next] | [standalone]

#111874

Il 09/06/2017 00:43, DFS ha scritto:
> On 6/8/2017 11:33 AM, Michael Forsythe wrote:
>
>
>> Others have mentioned popen, and you've already discovered the pain
>> that comes with that. At least on my systems, file comes with the
>> helpful libmagic library. If you can link against that:
>>
> <snip code>
>
>
>> The library also has magic_buffer in case you've already loaded the
>> file into memory.
>>
>> Of course, as others have already said, you probably want to do
>> some processing with the file after you've loaded it (treat it as
>> a configuration file, extract all email addresses, treat it as a
>> sequence of SAN chess moves, whatever). If so, you're probably
>> better off treating the file as a bunch of bytes and attempting to
>> specifically validate for whatever you're going to use the file
>> for.
>
> Thanks for the suggestions.  I got the _popen/pipes working on Windows,
> but I'm making my code cross-platform (Windows and Linux), and the above
> looks like more fun on Linux.
>
>
Your domain is not sufficiently clear: you want to check if a file 
contains binary data or text data. My question is: text or ascii text?
If your answer is ascii text, the only way is to open the file in binary 
mode and test each byte with the macro / function isascii (); this with 
respect to portability. But if you want to check if the file contains 
just some generic text, then you can't because the file might contain 
text in any format other than the ascii, such as utf8, utf16, utf32 and 
so on. Unfortunately, these last encodings use the value of a byte in full.

[toc] | [prev] | [next] | [standalone]

#111769

On 06/07/2017 06:14 PM, DFS wrote:
> On 6/7/2017 5:44 PM, Jorgen Grahn wrote:
>> On Wed, 2017-06-07, DFS wrote:
>>> binary, such as .xls or .exe or Linux ELF executable
>>> text, such as an ascii .txt file
>>
>> Why do you want to find out?  Few programs have a need for that.
>
> validate file inputs to my program.
>
> If someone tries to pass a binary file I want to sniff it out.

In general, I've found that it's best to only validate whether or not an 
input has some feature that will make my own code malfunction. For 
instance, if it contains a count that could cause an array subscript to 
exceed to bounds of that array. Testing for features that don't matter 
to my code has a couple of problems:

If there's two different pieces of code interacting, each of which will 
fail for different reasons if they receive unexpected input, each piece 
of code should test only it's own failure condition; it should rely on 
the other piece of code to test the other failure condition. The 
combined program should fail if either part indicates a problem. If each 
part simply tests everything that could be tested, whether or not it's 
relevant to that part of the code, you end up wasting time repeating the 
same tests over and over again.

Another reason is code reusability. If you routinely test things that 
your code doesn't need to test in order to work properly, it can make it 
hard to re-use that code to work in a different context where those 
tests are violated.

Real life example: most of my programs produce output in HDF format, 
which is a self-documenting format. As a result, you can write code that 
extracts the Latitude information by name from an HDF file, without 
having any idea what any of the other contents of the file are. You 
don't even need to know the data type, the array rank, the array 
dimensions, the valid range of the data or the fill values: that 
information is all stored in the file as metadata that can be queried.

Well, a program that reads one of my files started failing with an error 
message. The person currently responsible for the code didn't even 
understand what the message meant. The person previously responsible for 
the code knew what it meant, but had no idea why the error condition was 
being tested - there'd been some loss of institutional memory. I was 
left to figure that out myself.

It turns out that their code was checking whether the file had the 
expected number of dimension maps. The only thing you need to understand 
about dimension maps, in order to understand this story, is that each 
dimension map is associated with particular components of the file, and 
you only ever need to know about about a given dimension map if you're 
reading one of that associated components. HDF deliberately insulates 
you from having to know anything about the parts of the file you don't 
use, but this code deliberately broke that.
It turns out that I'd added some new components to the file, components 
that the other program had no need to read, and that those components 
required a new dimension map. I'd assumed that anybody who needed to use 
the new components would also write their code to deal with the 
dimension maps correctly, and that, given the nature of HDF, no one 
else's code would be affected. I hadn't counted on code that "validated" 
irrelevant input file features.
Their code got fixed, not mine - we had very good reasons for adding 
those dimension maps; they had no reason whatsoever for validating the 
number of dimension maps.

Moral: if there's some feature of a binary file that would cause 
problems for your code, test that feature when it becomes relevant. 
Don't just randomly go testing the entire file for whether it's binary.

>> (One I can think of is Gnu diff, which says "binary files differ"
>> instead of trying to show the textual difference between two
>> executables.)
>
> Is that all it says?

Yes. What's happened is that the parts that differ contain characters 
that are not supposed to occur in text files, so it doesn't bother 
printing them, for fear that the output might be hard to interpret. If 
there's only a few such characters in the file, that's often not a 
problem, so they also provide the -a option, which treats the input 
files as if they were text files, even if they aren't. I've often needed 
to use that option.

[toc] | [prev] | [next] | [standalone]

#111771

On Wed, 7 Jun 2017 18:14:32 -0400, DFS <nospam@dfs.com> wrote:

>On 6/7/2017 5:44 PM, Jorgen Grahn wrote:
>> On Wed, 2017-06-07, DFS wrote:
>>> binary, such as .xls or .exe or Linux ELF executable
>>> text, such as an ascii .txt file
>> 
>> Why do you want to find out?  Few programs have a need for that.
>
>validate file inputs to my program.
>
>If someone tries to pass a binary file I want to sniff it out.

So the real question is what constitutes valid input.  Presumably you
are willing to accept all printable characters.  What character set
are you using?  What about the escape sequences?

If you build a string of acceptable characters, strspn() can tell you
if there are any other characters in the data you read.

On the other hand, if you assume an invalid file will always contain a
'\0', then you could use fread and memchr to search for it.

-- 
Remove del for email

[toc] | [prev] | [next] | [standalone]

#111814

DFS <nospam@dfs.com> writes:

> On 6/7/2017 5:44 PM, Jorgen Grahn wrote:
>> On Wed, 2017-06-07, DFS wrote:
>>> binary, such as .xls or .exe or Linux ELF executable
>>> text, such as an ascii .txt file
>>
>> Why do you want to find out?  Few programs have a need for that.
>
> validate file inputs to my program.
>
> If someone tries to pass a binary file I want to sniff it out.

The algorithms that validate files (or data formats in general) are
called *parsers*.  A parser specifically validates a given format, and
returns true or false, whether the file is in the parsed "language".

So the question is whether you can define this "binary" language?  Do
you have a grammar for it?


I'll bet that you have a grammar for your actual input files!

This is what you should parse, and reject if it doesn't match the input
language!!!


-- 
__Pascal J. Bourguignon
http://www.informatimago.com

[toc] | [prev] | [next] | [standalone]

#111768

DFS <nospam@dfs.com> writes:

> binary, such as .xls or .exe or Linux ELF executable
> text, such as an ascii .txt file
>
> I've tried:
>
> _Bool is_binary(const void *data, size_t len)
> {
> 	return memchr(data, '\0', len) != NULL;
> }
>
> fgets(line,sizeof line, file);
> is_binary(line,strlen(line));
>
> but is_binary() always returns 0 no matter the data passed to it.

There's no real way to do it; since the bytes in an ascii file are a
subset of the bytes in any other file type.  The closest you can really
come is to look at all the bytes and look for bytes that aren't ascii
characters.

You might look at isascii() and related functions.  There's every chance
passing every byte through isascii() will work well enough.

[toc] | [prev] | [next] | [standalone]

#111780

Joe Pfeiffer <pfeiffer@cs.nmsu.edu> writes:
> DFS <nospam@dfs.com> writes:
>> binary, such as .xls or .exe or Linux ELF executable
>> text, such as an ascii .txt file
>>
>> I've tried:
>>
>> _Bool is_binary(const void *data, size_t len)
>> {
>> 	return memchr(data, '\0', len) != NULL;
>> }
>>
>> fgets(line,sizeof line, file);
>> is_binary(line,strlen(line));
>>
>> but is_binary() always returns 0 no matter the data passed to it.
>
> There's no real way to do it; since the bytes in an ascii file are a
> subset of the bytes in any other file type.  The closest you can really
> come is to look at all the bytes and look for bytes that aren't ascii
> characters.
>
> You might look at isascii() and related functions.  There's every chance
> passing every byte through isascii() will work well enough.

isascii() is not part of ISO C (because C doesn't require ASCII), though
it's likely to be available as an extension (POSIX defines it, but warns
that it may be removed).  It tests whether its argument is in the range
0..127, which you can check manually if you want to be portable.

isascii() will return true for ASCII control characters, including the
null character, and false for, for example, accented letters.  And like
the other is*() functions, it has undefined behavior if its argument is
negative and not equal to EOF.

-- 
Keith Thompson (The_Other_Keith) kst-u@mib.org  <http://www.ghoti.net/~kst>
Working, but not speaking, for JetHead Development, Inc.
"We must do something.  This is something.  Therefore, we must do this."
    -- Antony Jay and Jonathan Lynn, "Yes Minister"

[toc] | [prev] | [next] | [standalone]

#111786

Keith Thompson <kst-u@mib.org> writes:

> Joe Pfeiffer <pfeiffer@cs.nmsu.edu> writes:
>> DFS <nospam@dfs.com> writes:
>>> binary, such as .xls or .exe or Linux ELF executable
>>> text, such as an ascii .txt file
>>>
>>> I've tried:
>>>
>>> _Bool is_binary(const void *data, size_t len)
>>> {
>>> 	return memchr(data, '\0', len) != NULL;
>>> }
>>>
>>> fgets(line,sizeof line, file);
>>> is_binary(line,strlen(line));
>>>
>>> but is_binary() always returns 0 no matter the data passed to it.
>>
>> There's no real way to do it; since the bytes in an ascii file are a
>> subset of the bytes in any other file type.  The closest you can really
>> come is to look at all the bytes and look for bytes that aren't ascii
>> characters.
>>
>> You might look at isascii() and related functions.  There's every chance
>> passing every byte through isascii() will work well enough.
>
> isascii() is not part of ISO C (because C doesn't require ASCII), though
> it's likely to be available as an extension (POSIX defines it, but warns
> that it may be removed).  It tests whether its argument is in the range
> 0..127, which you can check manually if you want to be portable.
>
> isascii() will return true for ASCII control characters, including the
> null character, and false for, for example, accented letters.  And like
> the other is*() functions, it has undefined behavior if its argument is
> negative and not equal to EOF.

Ah, my reading was that it would throw those out.  Thanks for the
correction.

Really, the other responses (variations on just parse the input, and
throw an error when you hit a character you aren't expecting) are better
than mine.

[toc] | [prev] | [next] | [standalone]

#111796

On 07/06/2017 20:40, DFS wrote:
> How to determine if file is binary or text?> binary, such as .xls or .exe or Linux ELF executable> text, such as an ascii .txt file

It involves a whole lotta guessin'

http://man7.org/linux/man-pages/man3/libmagic.3.html
http://man7.org/linux/man-pages/man1/file.1.html
https://en.wikipedia.org/wiki/File_%28command%29#Specification

And now for a trick question:

English Shellcode
http://web.cs.jhu.edu/~sam/ccs243-mason.pdf
https://en.wikipedia.org/wiki/Shellcode

This is shellcode (i.e. executable code) using only ASCII
characters. Is it "binary" or "text" ?

Regards.

[toc] | [prev] | [next] | [standalone]

Page 2 of 3 — ← Prev page 1 [2] 3  Next page →

Back to top | Article view | comp.lang.c

csiph-web