Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.os.msdos.programmer > #1364 > unrolled thread

detecting emulation or console windows

Back to article view | Back to comp.os.msdos.programmer

Contents

  detecting emulation or console windows "Rod Pemberton" <dont_use_email@xnothavet.cqm> - 2014-05-31 18:31 -0400
    Re: detecting emulation or console windows Ross Ridge <rridge@csclub.uwaterloo.ca> - 2014-05-31 19:59 -0400
      Re: detecting emulation or console windows "Rod Pemberton" <dont_use_email@xnothavet.cqm> - 2014-05-31 20:54 -0400
        Re: detecting emulation or console windows "Rod Pemberton" <dont_use_email@xnothavet.cqm> - 2014-05-31 21:43 -0400
          Re: detecting emulation or console windows "James Harris" <james.harris.1@gmail.com> - 2014-06-01 22:48 +0100
            Re: detecting emulation or console windows "Rod Pemberton" <dont_use_email@xnothavet.cqm> - 2014-06-02 00:46 -0400
              Re: detecting emulation or console windows "James Harris" <james.harris.1@gmail.com> - 2014-06-02 06:15 +0100
                Re: detecting emulation or console windows "Rod Pemberton" <dont_use_email@xnothavet.cqm> - 2014-06-02 03:21 -0400
                  Re: detecting emulation or console windows "Rod Pemberton" <dont_use_email@xnothavet.cqm> - 2014-06-02 03:32 -0400
                  Re: detecting emulation or console windows "James Harris" <james.harris.1@gmail.com> - 2014-06-02 09:51 +0100
                    Re: detecting emulation or console windows "Rod Pemberton" <dont_use_email@xnothavet.cqm> - 2014-06-02 09:27 -0400
                      Re: detecting emulation or console windows "James Harris" <james.harris.1@gmail.com> - 2014-06-02 17:05 +0100
                        Re: detecting emulation or console windows "Rod Pemberton" <dont_use_email@xnothavet.cqm> - 2014-06-02 19:25 -0400
                          Re: detecting emulation or console windows "James Harris" <james.harris.1@gmail.com> - 2014-06-04 07:55 +0100
                            Re: detecting emulation or console windows "Rod Pemberton" <dont_use_email@xnothavet.cqm> - 2014-06-04 03:20 -0400
        Re: detecting emulation or console windows Ross Ridge <rridge@csclub.uwaterloo.ca> - 2014-06-01 14:03 -0400
          Re: detecting emulation or console windows "Rod Pemberton" <dont_use_email@xnothavet.cqm> - 2014-06-02 00:53 -0400
            Re: detecting emulation or console windows Ross Ridge <rridge@csclub.uwaterloo.ca> - 2014-06-02 15:55 -0400
              Re: detecting emulation or console windows "Rod Pemberton" <dont_use_email@xnothavet.cqm> - 2014-06-02 19:27 -0400
                Re: detecting emulation or console windows Ross Ridge <rridge@csclub.uwaterloo.ca> - 2014-06-03 17:00 -0400
    Re: detecting emulation or console windows Sjouke Burry <burrynulnulfour@ppllaanneett.nnll> - 2014-06-01 06:25 +0200
      Re: detecting emulation or console windows "Rod Pemberton" <dont_use_email@xnothavet.cqm> - 2014-06-01 02:50 -0400
    Re: detecting emulation or console windows "James Harris" <james.harris.1@gmail.com> - 2014-06-01 07:53 +0100
      Re: detecting emulation or console windows "Rod Pemberton" <dont_use_email@xnothavet.cqm> - 2014-06-01 04:53 -0400
        Re: detecting emulation or console windows "James Harris" <james.harris.1@gmail.com> - 2014-06-01 20:57 +0100
          Re: detecting emulation or console windows "Rod Pemberton" <dont_use_email@xnothavet.cqm> - 2014-06-02 00:48 -0400
        Re: detecting emulation or console windows "Rod Pemberton" <dont_use_email@xnothavet.cqm> - 2014-06-02 03:20 -0400
    Re: detecting emulation or console windows JJ <duh@nah.meh> - 2014-06-02 03:43 +0700
      Re: detecting emulation or console windows "Rod Pemberton" <dont_use_email@xnothavet.cqm> - 2014-06-02 00:34 -0400
      Re: detecting emulation or console windows "Rod Pemberton" <dont_use_email@xnothavet.cqm> - 2014-06-03 07:06 -0400
        Re: detecting emulation or console windows JJ <duh@nah.meh> - 2014-06-03 20:25 +0700
    Re: detecting emulation or console windows "wolfgang kern" <nowhere@never.at> - 2014-06-03 20:52 +0200
    Re: detecting emulation or console windows "James Harris" <james.harris.1@gmail.com> - 2014-06-03 22:59 +0100
      Re: detecting emulation or console windows CN <qmbmnp3799@pacbell.net> - 2014-06-03 18:39 -0700
        Re: detecting emulation or console windows "James Harris" <james.harris.1@gmail.com> - 2014-06-04 07:06 +0100
          Re: detecting emulation or console windows "Rod Pemberton" <dont_use_email@xnothavet.cqm> - 2014-06-04 03:07 -0400
            Re: detecting emulation or console windows "James Harris" <james.harris.1@gmail.com> - 2014-06-04 09:16 +0100
    Re: detecting emulation or console windows "James Harris" <james.harris.1@gmail.com> - 2014-06-03 23:00 +0100
    Re: detecting emulation or console windows "Rod Pemberton" <dont_use_email@xnothavet.cqm> - 2014-06-10 17:37 -0400
      Re: detecting emulation or console windows Wildman <best_lay@yahoo.com> - 2014-06-11 00:37 +0000
        Re: detecting emulation or console windows "Rod Pemberton" <dont_use_email@xnothavet.cqm> - 2014-07-27 12:11 -0400

Page 2 of 3 — ← Prev page 1 [2] 3  Next page →

#1368

On 01.06.14 0:31, Rod Pemberton wrote:
>
> Do you guys have any pointers on detecting whether or not code
> is executing in an emulator or in console window?
>

Do a system call like:   system("ver > settings.txt")
then read the settings.txt file, which should tell you
whether you are in DOS or not.

[toc] | [prev] | [next] | [standalone]

#1369

On Sun, 01 Jun 2014 00:25:00 -0400, Sjouke Burry  
<burrynulnulfour@ppllaanneett.nnll> wrote:
> On 01.06.14 0:31, Rod Pemberton wrote:

>> Do you guys have any pointers on detecting whether or not code
>> is executing in an emulator or in console window?
>>
>
> Do a system call like:   system("ver > settings.txt")
> then read the settings.txt file, which should tell you
> whether you are in DOS or not.
>

I didn't do a system call.  I doubt that the system call makes
any difference.  I.e., it just respawns a new instance of
command.com etc.  So, I just entered 'ver' at the command line.

DOSBox ver reports:
   "DOSBox version 0.74. Reported DOS version 5.00."

dosemu ver reports the MS-DOS version installed on the
partition that was mounted by dosemu:
   "Windows 98 [Version 4.10.2222]"

Obviously, that has nothing to do with Linux dosemu, which
just provides the console with, not the DOS which is executed.
That is MS-DOS v7.10 which is part of Windows 98/SE.  I.e.,
if I had FreeDOS on a partition and booted it via dosemu,
'ver' should return whatever FreeDOS reports.

So, some emulation will return different results, but other
emulation just returns the results for the installed version
of DOS.


Rod Pemberton

[toc] | [prev] | [next] | [standalone]

#1370

"Rod Pemberton" <dont_use_email@xnothavet.cqm> wrote in message 
news:op.xgqwuvss6zenlw@localhost...
>
> Do you guys have any pointers on detecting whether or not code
> is executing in an emulator or in console window?
>
> Specifically, I have DJGPP (GCC) DOS code which is intended
> to execute under a DPMI host starting from RM (real-mode)
> MS-DOS.  The problem is the code is incompatible with emulated
> environments and causes a crash, i.e., programs hardware.

...

> ...  The goals are: simple,
> reliable, i.e., works everywhere, all machines, all processors,
> all types of emulation.

Just to check on what the requirement is, if there was an emulator under 
which your code would work successfully would you still want your code to 
exit or would you want it to go ahead and execute?

> So, if software emulation is good, it's hard to detect.

Yes. In principle, the better the emulation the harder it will be to detect. 
As you acknowledge, detecting whether the program is running under an 
emulator is like chasing the wind. Each time a better version of an emulator 
comes it it may become harder to detect. Hence my query about the 
requirement.

James

[toc] | [prev] | [next] | [standalone]

#1371

On Sun, 01 Jun 2014 02:53:39 -0400, James Harris  
<james.harris.1@gmail.com> wrote:

> Just to check on what the requirement is, if there was an emulator under
> which your code would work successfully would you still want your code to
> exit [...] ?

Yes, I would prefer that it exit cleanly under all emulation.  I may simply
be hoping for too much ...

If it runs correctly under emulation, it's not a detriment.  Currently, it
exits with a crash under tested emulation ...  I think it conflicts, or is
likely to conflict, with a host, any host.  It's a new project, so it's not
complete, yet.  Yup, yet another project...  Think of it as an OS, but not
a pure OS, perhaps pseudo-OS that runs on top of DOS.  I understand that it
may not be possible in all cases to exit cleanly, but the majority would be
nice.

I've found a few more ways to detect DOSBox and some more for dosemu,
e.g., wrong values for required BIOS vectors, BIOS reboot code not  
correctly
implemented, etc, but nothing consistent between the two.  So, nothing yet
to lead me to believe that something would work for QEMU, VMWare, Bochs,  
etc.
I should probably just go ahead and install QEMU and Bochs, perhaps MESS  
too.
That would likely be the majority of emulators one could expect, I'd  
hope.  Yes?
I could probably install FreeDOS on USB as a cross-check to make sure  
whatever
I find is not MS-DOS specific.

Is VMWare available for Linux?

What other "major" emulators are out there?
(Ignore different versions of Windows.)


Rod Pemberton

[toc] | [prev] | [next] | [standalone]

#1373

"Rod Pemberton" <dont_use_email@xnothavet.cqm> wrote in message 
news:op.xgrpmsas6zenlw@localhost...
> On Sun, 01 Jun 2014 02:53:39 -0400, James Harris 
> <james.harris.1@gmail.com> wrote:
>
>> Just to check on what the requirement is, if there was an emulator under
>> which your code would work successfully would you still want your code to
>> exit [...] ?
>
> Yes, I would prefer that it exit cleanly under all emulation.  I may 
> simply
> be hoping for too much ...

Logically, since the emulator provides everything that your program is able 
to see about the world, including the timing, a perfect emulator may be 
impossible to detect. Detecting *imperfect* emulations would be possible and 
none of the emulators would be perfect but the tests may differ because the 
imperfections differ.

I suppose where you are now is looking for a small set of imperfections that 
would detect all current emulators or, alternatively, a way to detect real 
hardware. Either is definitely challenging because even hardware differs.

> If it runs correctly under emulation, it's not a detriment.

You may need to accept the compromise of running under emulation. How about 
thinking of the following categories of target for your code?

1. Real hardware. One spec but many variations to adapt to. Even some real 
machines don't match the spec.

2. Full emulators. These could be seen as just more variations to adapt to. 
Things like VirtualBox, VMWare, QEMU and others.

3. Incomplete emulations, ones where a change to some hardware may cause a 
crash. These would be things like a Windows command prompt where changes to 
hardware would be ignored or would break Windows.

Essentially, categories 1 and 2 would be ones you could work with if you 
wanted to. You may need to do a lot of debugging to allow your code to work 
with the differences but it should be feasible and the differences would 
make your code more robust. I think Ross is making a similar point.

That would leave you only needing to detect category 3.

> Currently, it
> exits with a crash under tested emulation ...  I think it conflicts, or is
> likely to conflict, with a host, any host.  It's a new project, so it's 
> not
> complete, yet.  Yup, yet another project...  Think of it as an OS, but not
> a pure OS, perhaps pseudo-OS that runs on top of DOS.  I understand that 
> it
> may not be possible in all cases to exit cleanly, but the majority would 
> be
> nice.

Do you know which actions cause it to crash?

Here's a thought. After altering each hardware setting is there some way you 
could check that the setting has taken effect? If it hasn't then you know 
not to continue and you are probably running in an imperfect emulation.

> I've found a few more ways to detect DOSBox and some more for dosemu,
> e.g., wrong values for required BIOS vectors, BIOS reboot code not 
> correctly
> implemented, etc, but nothing consistent between the two.  So, nothing yet
> to lead me to believe that something would work for QEMU, VMWare, Bochs, 
> etc.
> I should probably just go ahead and install QEMU and Bochs, perhaps MESS 
> too.
> That would likely be the majority of emulators one could expect, I'd 
> hope.  Yes?
> I could probably install FreeDOS on USB as a cross-check to make sure 
> whatever
> I find is not MS-DOS specific.

Something up with your line wrapping?

> Is VMWare available for Linux?
>
> What other "major" emulators are out there?
> (Ignore different versions of Windows.)

I don't know. I always use Oracle VirtualBox. Some are listed here:

  http://wiki.osdev.org/Emulators

James

[toc] | [prev] | [next] | [standalone]

#1378

On Sun, 01 Jun 2014 15:57:04 -0400, James Harris  
<james.harris.1@gmail.com> wrote:

> Here's a thought. After altering each hardware setting is there some way
> you could check that the setting has taken effect? If it hasn't then you
> know not to continue and you are probably running in an imperfect  
> emulation.

Yes, something like that might need to be done, if I stick with the
reject all emulation idea.  I also need to reject DPMI hosts that don't
have certain functionality.  Earlier today, I considered creating a test
for DPMI client.  If it fails, then exit.  Currently, I've tested a bunch
of DPMI hosts and I'm doing a general check which works for most of them.
An explicit run-time test would allow for hosts I simply can't test.

> Something up with your line wrapping?
>

IE6 could be set to auto-matically wrap at a specific length for Usenet
posts.  This browser (Opera) doesn't auto-terminate posts at a specific
length, AFAICT.  Maybe, the option is buried somewhere.  So, currently,
I have to enter a return explicitly.  I try to keep it to 72 to 74, but
those posts were a bit long.  Usenet RFC 1855 recommends a CR at 65,
but 72 is common for Usenet.  With variable width fonts, I'm constantly
guessing where 72 is ...  Typing .123456789 repeatedly becomes tiresome.

In this case, Google Groups seems to have added their own wrapping
to two posts of mine, around 71 to 77.  Or, perhaps it was the Usenet
provider they receive their posts from.  That's not something which
should be done.  Usenet servers where I read did *not* wrap the posts,
even though they were long.

Usually, it's those who use GG whose posts don't wrap correctly,
i.e., continue forever.


Rod Pemberton

[toc] | [prev] | [next] | [standalone]

#1381

On Sun, 01 Jun 2014 04:53:06 -0400, Rod Pemberton  
<dont_use_email@xnothavet.cqm> wrote:

> I've found a few more ways to detect DOSBox and some more for dosemu,
> e.g., wrong values for required BIOS vectors, BIOS reboot code not
> correctly implemented, etc, but nothing consistent between the two.
> So, nothing yet to lead me to believe that something would work for
> QEMU, VMWare, Bochs, etc.  I should probably just go ahead and install
> QEMU and Bochs, perhaps MESS too. That would likely be the majority
> of emulators one could expect, I'd hope.  Yes? I could probably
> install FreeDOS on USB as a cross-check to make sure whatever
> I find is not MS-DOS specific.
>

So far, I've concluded that both DOSBox and dosemu are *wierd*.

DOSBox's emulation, as we know, is just for games, i.e., it's
not great if you're looking for correct DOS emulation.

DOSBox won't let you set certain RM IVT vectors: 10h, 21h, ...

dosemu doesn't emulate 'lidt' in RM, i.e., you can't move
the IVT in RM in dosemu, as you can in RM on any processor
that supports PM.  However, DOSBox surprisingly *does* emulate
the 'lidt' instruction allowing you to move the RM IVT around.
Go figure ...

That's in addition to the other stuff I've found and
mentioned elsewhere in this thread.

I'm strongly considering implementing a way to reject DOSBox.
I'm not so decided on dosemu ...


Rod Pemberton

[toc] | [prev] | [next] | [standalone]

#1374

On Sat, 31 May 2014 18:31:33 -0400, Rod Pemberton wrote:
> Do you guys have any pointers on detecting whether or not code
> is executing in an emulator or in console window?
> 
> Specifically, I have DJGPP (GCC) DOS code which is intended
> to execute under a DPMI host starting from RM (real-mode)
> MS-DOS.  The problem is the code is incompatible with emulated
> environments and causes a crash, i.e., programs hardware.
> 
> Console windows, like Windows 98/SE's DOS console or Linux's
> dosemu environment, are emulating well enough that I can't
> currently detect them.  But, I'd like the code to exit cleanly,
> if possible, instead of crashing.
> 
> I'm aware that certain environments, e.g., DOSBOX, have
> a specific interrupt routine that can be called to detect.
> I use a bunch of these such calls, but they're insufficient.
> The fact such calls are available is good, but I'd prefer
> something more generic so that I'm coding up dozens upon
> dozens of tests, including tests for environments I can't test.
> 
> I've looked at CR0.PE, but dosemu emulates MOV CRO and SMSW
> accurately enough that those instructions are *not* useful,
> i.e., they indicate CR0.PE is clear even when under emulation.
> A20 has a similar problem.  Under emulation, it initially indicates
> it's not enabled.  So, now, I'm thinking about keyboard, NMI, RTC,
> or other things, like processor flags, interrupts, memory locations,
> instruction bugs, self-modifying code, etc where emulation might
> not be perfect or is at least detectable.  The goals are: simple,
> reliable, i.e., works everywhere, all machines, all processors,
> all types of emulation.
> 
> So, if software emulation is good, it's hard to detect.  I've
> been searching Google, Yahoo, and Usenet archives for some hints,
> but I'm not finding much.  It seems Shawn Hargreaves of Allegro
> ran into much the same issue years ago, but I'm not sure if he
> found a solution.
> 
> Rod Pemberton
> PS.  This is cross-posted to a few groups.  I read all, but it's
> easier for me, if you leave all groups on.

Most emulators ad virtualizators have backdoor for communicating with the
host system. For examples, Parallels and VMWare uses I/O ports, VirtualPC
uses invalid opcodes, and VirtualBox uses SYSENTER.

There's also generic detection methods that check the presence of real CPU
limitations & bugs that don't exist in the emulated/virtualized system.

And lastly, a less reliable method that search for a specific text in the
BIOS ROM can also be used.

You might want to check these: (warning, long URL)

<http://web.archive.org/web/20130316040537/http://my.opera.com/jaelanicu/blog/just-another-vm-detection-was-vm-detection-combo>

[toc] | [prev] | [next] | [standalone]

#1376

On Sun, 01 Jun 2014 16:43:02 -0400, JJ <duh@nah.meh> wrote:

> Most emulators ad virtualizators have backdoor for communicating with the
> host system. For examples, Parallels and VMWare uses I/O ports, VirtualPC
> uses invalid opcodes, and VirtualBox uses SYSENTER.

Yes, but that presents me with even more tests ...

Virtual PC opcode series:
0x0f 0x3f xx xx
0x0f 0xc7 0xc8 yy yy

DOSBox callback
0xfe 0x38

NTVDM opcode series:
0xc4 0xc4 xx xx

I'm not sure how to use those, even though I have the series.
And, I can't test NTVDM or Virtual PC, currently.

> And lastly, a less reliable method that search for a specific
> text in the BIOS ROM can also be used.

Yes, I suspect the reboot emulation errors I found in DOSBox and
dosemu won't work with QEMU because QEMU correctly implements the
major BIOS entry points.


Rod Pemberton

[toc] | [prev] | [next] | [standalone]

#1393

On Sun, 01 Jun 2014 16:43:02 -0400, JJ <duh@nah.meh> wrote:

> You might want to check these: (warning, long URL)
>
> <http://web.archive.org/web/20130316040537/http://my.opera.com/jaelanicu/blog/just-another-vm-detection-was-vm-detection-combo>

It took a few days for me to get to that link.

It was interesting that he managed to find a set
of three tests which managed to detect all but one
of the emulators he tested.

It helped me find another link which does something
similar to what Steve suggested with GDTR, but using
IDTR to detect execution under a VM.

http://web.archive.org/web/20080705214817/http://invisiblethings.org/papers/redpill.html


Rod Pemberton

[toc] | [prev] | [next] | [standalone]

#1394

On Tue, 03 Jun 2014 07:06:17 -0400, Rod Pemberton wrote:
> It helped me find another link which does something
> similar to what Steve suggested with GDTR, but using
> IDTR to detect execution under a VM.
> 
> http://web.archive.org/web/20080705214817/http://invisiblethings.org/papers/redpill.html

The detection using IDTR & LDTR was actually covered in that original
article "VM Detection Combo". It's also based on the Red Pill method.

<http://web.archive.org/web/20091109172524/http://my.opera.com/jaelanicu/blog/show.dml/4257341>

Unfortunately, the Wayback Machine doesn't have his VMDetectionCombo program
archived, but I think the article provides enough information to make that
detection from scratch.

[toc] | [prev] | [next] | [standalone]

#1395

Rod Pemberton asked:

> Do you guys have any pointers on detecting whether or not code
> is executing in an emulator or in console window?

[read all the thread rntil end so far...]

Of course there is a limit for all emulation activity. Some may
even fully fool you with a total BIOS- and even with a complete
BIOS-behaviour. But this can be detected easy enough.

Not sure that all these emulators around would really emulate
exceptions like I/O-break-points because they would normally
be used to fake any virtual environment.
Even Debug-registers could be emulated, a read of last used
linear address from FPU-debug may show you the difference.
But I'd also check if this linear pointer is whitin your
expected code range or outside.

And there are a lots of such opportunities to check if your RM code
run in a faked virtual or on actual existing hardware.

But as James already said:
A perfect emulation would show perfect result w/o telling.
But by any luck we never will see any totally perfect emulation,
so your quest, even a bit paranoid for my taste :) stands correct.
__
wolfgang

[toc] | [prev] | [next] | [standalone]

#1397

Rod doesn't seem to have responded to this but the info below looks 
promising. Copying to the other two groups.

"CN" <qmbmnp3799@pacbell.net> wrote in message 
news:lmg9hf$qmu$1@dont-email.me...
> On 5/31/2014 7:03 PM, Rod Pemberton wrote:
>> On Sat, 31 May 2014 21:31:25 -0400, Alexei A. Frounze
>> <alexfrunews@gmail.com> wrote:
>>> On Saturday, May 31, 2014 3:31:33 PM UTC-7, Rod Pemberton wrote:
>>
>>>> Do you guys have any pointers on detecting whether or not code
>>>> is executing in an emulator or in console window?
>>>
>>> I'd try looking at timing. Disable interrupts and see how many cycles
>>> (TSC) and seconds (RTC) certain instructions take when repeated many
>>> times. See the distribution. If it's got multiple distinct peaks or
>>> some other oddity, your code is likely running in a virtual
>>> environment. Power modes/frequency changes in the CPU may complicate
>>> things, though.
>
> As a person working for a company that makes virtualization software, I 
> can say that this is the most sure-fire way of checking for a modern 
> emulator that uses a hardware virtualization technology like VT-x or AMD 
> Pacifica. I don't think any of them bother to adjust guest TSC to hide 
> themselves. Simply use TSC to benchmark an instruction that is always 
> trapped by modern emulators, like CPUID.
>
> Now if you wanted to check for military-grade malicious hypervisors that 
> people say come planted into the BIOS of Chinese-made motherboards with 
> the purpose of spying on us, that's a different story. You'd want to use 
> an external timer rather than TSC, because the TSC is likely compromised.
>
> The same test probably won't work for older stuff like DOS emulators and 
> DPMI. They probably don't need, don't bother or don't have the capability 
> to trap CPUID. They must trap something else, though. Not being very 
> familiar with it, I'm sure there's something that will work well just for 
> that specific subclass, maybe like SGDT. I think you should split the task 
> of detecting old emulators from the task of detecting modern 
> virtualization software, due to the difference in what is emulated.
>

So comparing two time sources: TSC and a real-world timer like the PC's PIT 
ought to give good, i.e. fairly reliable, results. The question is what 
operation or operations to time.

James

[toc] | [prev] | [next] | [standalone]

#1400

On 6/3/2014 2:59 PM, James Harris wrote:
> Rod doesn't seem to have responded to this but the info below looks
> promising. Copying to the other two groups.
>
> "CN" <qmbmnp3799@pacbell.net> wrote in message
> news:lmg9hf$qmu$1@dont-email.me...
>> On 5/31/2014 7:03 PM, Rod Pemberton wrote:
>>> On Sat, 31 May 2014 21:31:25 -0400, Alexei A. Frounze
>>> <alexfrunews@gmail.com> wrote:
>>>> On Saturday, May 31, 2014 3:31:33 PM UTC-7, Rod Pemberton wrote:
>>>
>>>>> Do you guys have any pointers on detecting whether or not code
>>>>> is executing in an emulator or in console window?
>>>>
>>>> I'd try looking at timing. Disable interrupts and see how many cycles
>>>> (TSC) and seconds (RTC) certain instructions take when repeated many
>>>> times. See the distribution. If it's got multiple distinct peaks or
>>>> some other oddity, your code is likely running in a virtual
>>>> environment. Power modes/frequency changes in the CPU may complicate
>>>> things, though.
>>
>> As a person working for a company that makes virtualization software, I
>> can say that this is the most sure-fire way of checking for a modern
>> emulator that uses a hardware virtualization technology like VT-x or AMD
>> Pacifica. I don't think any of them bother to adjust guest TSC to hide
>> themselves. Simply use TSC to benchmark an instruction that is always
>> trapped by modern emulators, like CPUID.
>>
>> Now if you wanted to check for military-grade malicious hypervisors that
>> people say come planted into the BIOS of Chinese-made motherboards with
>> the purpose of spying on us, that's a different story. You'd want to use
>> an external timer rather than TSC, because the TSC is likely compromised.
>>
>> The same test probably won't work for older stuff like DOS emulators and
>> DPMI. They probably don't need, don't bother or don't have the capability
>> to trap CPUID. They must trap something else, though. Not being very
>> familiar with it, I'm sure there's something that will work well just for
>> that specific subclass, maybe like SGDT. I think you should split the task
>> of detecting old emulators from the task of detecting modern
>> virtualization software, due to the difference in what is emulated.
>>
>
> So comparing two time sources: TSC and a real-world timer like the PC's PIT
> ought to give good, i.e. fairly reliable, results. The question is what
> operation or operations to time.

Inside a virtual machine, the PIT is almost surely virtualized as well. 
It may or may not work the way you want.

You can just use common sense. Normally, CPUID or any other common 
instruction that doesn't talk to external devices takes a few hundred 
TSC cycles max. If it's trapped, it'll go into thousands. Just the 
VM-exit/VM-entry sequence takes more than (or at least close to) a 
thousand cycles on any modern Intel CPU.

[toc] | [prev] | [next] | [standalone]

#1401

"CN" <qmbmnp3799@pacbell.net> wrote in message 
news:lmltd0$cb9$1@dont-email.me...
> On 6/3/2014 2:59 PM, James Harris wrote:
>> Rod doesn't seem to have responded to this but the info below looks
>> promising. Copying to the other two groups.
>>
>> "CN" <qmbmnp3799@pacbell.net> wrote in message
>> news:lmg9hf$qmu$1@dont-email.me...
>>> On 5/31/2014 7:03 PM, Rod Pemberton wrote:
>>>> On Sat, 31 May 2014 21:31:25 -0400, Alexei A. Frounze
>>>> <alexfrunews@gmail.com> wrote:
>>>>> On Saturday, May 31, 2014 3:31:33 PM UTC-7, Rod Pemberton wrote:
>>>>
>>>>>> Do you guys have any pointers on detecting whether or not code
>>>>>> is executing in an emulator or in console window?
>>>>>
>>>>> I'd try looking at timing. Disable interrupts and see how many cycles
>>>>> (TSC) and seconds (RTC) certain instructions take when repeated many
>>>>> times. See the distribution. If it's got multiple distinct peaks or
>>>>> some other oddity, your code is likely running in a virtual
>>>>> environment. Power modes/frequency changes in the CPU may complicate
>>>>> things, though.
>>>
>>> As a person working for a company that makes virtualization software, I
>>> can say that this is the most sure-fire way of checking for a modern
>>> emulator that uses a hardware virtualization technology like VT-x or AMD
>>> Pacifica. I don't think any of them bother to adjust guest TSC to hide
>>> themselves. Simply use TSC to benchmark an instruction that is always
>>> trapped by modern emulators, like CPUID.
>>>
>>> Now if you wanted to check for military-grade malicious hypervisors that
>>> people say come planted into the BIOS of Chinese-made motherboards with
>>> the purpose of spying on us, that's a different story. You'd want to use
>>> an external timer rather than TSC, because the TSC is likely 
>>> compromised.

...

>> So comparing two time sources: TSC and a real-world timer like the PC's 
>> PIT
>> ought to give good, i.e. fairly reliable, results. The question is what
>> operation or operations to time.

That paragraph wasn't very clear. I meant that one could pick an operation 
to test and compare its expected times against at least two time sources. 
The TSC should help with situations such as the one you mention but if an 
emulator reasonably accurately accounts for TSC clock ticks then the TSC 
test would not detect a difference. However, subsequently comparing TSC 
against some real-world time reference or comparing multiples of the tested 
operation against a real-world time reference ought to show up differences.

I picked the PIT because it is expected to tick at a fixed frequency - 
something like 1.19 MHz. Because some operating systems use it to keep the 
time of the day it would be difficult for an emulator to hide it without 
potentially affecting the emulated code's timekeeping.

Other real-world time sources available to any PC are the RTC functions of 
the CMOS chip and, if there is access to the internet, NTP time. Apart from 
very early PCs there will also be a number of other timers available. An 
emulator could not normally adjust all of them because some are used to 
maintain time or to control the frequencies of sound etc - things that would 
be noticed by a person using the software.

> Inside a virtual machine, the PIT is almost surely virtualized as well. It 
> may or may not work the way you want.
>
> You can just use common sense. Normally, CPUID or any other common 
> instruction that doesn't talk to external devices takes a few hundred TSC 
> cycles max. If it's trapped, it'll go into thousands. Just the 
> VM-exit/VM-entry sequence takes more than (or at least close to) a 
> thousand cycles on any modern Intel CPU.

Isn't CPUID unprivileged? And also SGDT that you mentioned? If so they could 
be executed by a normal user-level program and fail to trap under certain 
kinds of emulation.

There may be emulations which trap CPUID - using CPU virualisation support, 
possibly, I don't know - but there may be other emulations which do not trap 
CPUID because they execute guest code as unprivileged and trap only on 
pivileged operations and then mimic their operation.

So are there other things that could be tested? Allowing for normal 
variations of a machine including timeslices, perhaps a failure of any one 
of the tests would indicate that the code was being emulated...?

James

[toc] | [prev] | [next] | [standalone]

#1403

On Wed, 04 Jun 2014 02:06:16 -0400, James Harris  
<james.harris.1@gmail.com> wrote:
> "CN" <qmbmnp3799@pacbell.net> wrote in message
> news:lmltd0$cb9$1@dont-email.me...
>> On 6/3/2014 2:59 PM, James Harris wrote:

>>> Rod doesn't seem to have responded to this but the info below
>>> looks promising. Copying to the other two groups.

Sorry, I saw it, I just didn't have anything to add or ask CN.

(FYI, this thread is getting some posts on a.o.d. and a.l.a. where
the other groups were dropped.)

I also considered that CN's suggestion might be more work than I was
interested in to program the timers and time instructions, etc.  I.e.,
it's an option, but it's not a top choice, at least at this time.

> Other real-world time sources available to any PC are the RTC functions
> of the CMOS chip and, if there is access to the internet, NTP time. Apart
> from very early PCs there will also be a number of other timers  
> available.
> An emulator could not normally adjust all of them because some are used
> to maintain time or to control the frequencies of sound etc - things that
> would be noticed by a person using the software.
>

My initial thought in regards to timers is about access and availability:

Which of the seven or so timers in the PC does the code have access to
under emulation?

I.e., we could probably assume the PIT, RTC, TSC are available.
Can we assume the modern HPET, LAPIC, ACPI PMT are?  What about
the obsolete DRAM timer?


Rod Pemberton

[toc] | [prev] | [next] | [standalone]

#1406

"Rod Pemberton" <dont_use_email@xnothavet.cqm> wrote in message 
news:op.xgw4q5t06zenlw@localhost...
> On Wed, 04 Jun 2014 02:06:16 -0400, James Harris 
> <james.harris.1@gmail.com> wrote:
>> "CN" <qmbmnp3799@pacbell.net> wrote in message
>> news:lmltd0$cb9$1@dont-email.me...

...

> I also considered that CN's suggestion might be more work than I was
> interested in to program the timers and time instructions, etc.  I.e.,
> it's an option, but it's not a top choice, at least at this time.

Yes, the TSC may be easy but working with the PIT is not trivial. Offset 
against that is the possibility of being able to test for real hardware 
rather than testing for emulators. If something consistently takes longer 
than it could on real hardware or the TSC varies from the PIT by too much 
then you know you are running under emulation of some sort. Then you would 
have no need to make separate tests for different emulators.

>> Other real-world time sources available to any PC are the RTC functions
>> of the CMOS chip and, if there is access to the internet, NTP time. Apart
>> from very early PCs there will also be a number of other timers 
>> available.
>> An emulator could not normally adjust all of them because some are used
>> to maintain time or to control the frequencies of sound etc - things that
>> would be noticed by a person using the software.
>>
>
> My initial thought in regards to timers is about access and availability:
>
> Which of the seven or so timers in the PC does the code have access to
> under emulation?

I expect it depends on the emulator.

> I.e., we could probably assume the PIT, RTC, TSC are available.
> Can we assume the modern HPET, LAPIC, ACPI PMT are?  What about
> the obsolete DRAM timer?

The HPET might be a handy test if available but couldn't you do all you need 
with the old standard ones?

James

[toc] | [prev] | [next] | [standalone]

#1398

Copying this post to the other two groups.

"CN" <qmbmnp3799@pacbell.net> wrote in message 
news:lmgdai$j4m$1@dont-email.me...
> On 5/31/2014 6:31 PM, Alexei A. Frounze wrote:
>
>> Certain instructions (e.g. IN/OUT) always cause a context switch to
>> the hypervisor (AMD Pacifica and Intel VT) and thus take lots of
>> cycles. Btw, there's a bit in CPUID that may tell you that there's a
>> hypervisor running/available.
>
> A funny thing, that bit. Invented by Microsoft and allegedly required for 
> some certifications from MS. They've pretty much stepped on Intel's 
> territory by creating it, and you won't find it documented anywhere, 
> except maybe some obscure help pages at microsoft.com. Intel has it 
> "reserved" in their docs. Some hypervisors do implement it, some don't. I 
> wouldn't rely on it.
>
>> Another thing to try is to exploit unspecified behavior of
>> instructions, e.g. shift and rotate instructions, e.g. when the shift
>> count is large. Several years ago I found that there are typically 3
>> implementations: AMD, Intel non-hyperthreaded CPUs, Intel
>> hyperthreaded CPUs. Things may have changed since (we've got newer
>> CPUs, we've got Intel Atom, etc) and I don't know where VIA and other
>> compatible x86 CPUs stand. Nonetheless, if shifts and rotates are
>> done in software, they are likely to differ in behavior from those
>> executed natively.
>
> Who emulates shift and rotate instructions nowadays? The only things that 
> come to mind are complete emulators that implement every instruction in 
> software, like libx86 that's used in the X server to run video card option 
> ROMs. Pretty much everybody else will directly execute them.
>
>> Yet another thing to look at is executing the same instruction using
>> different memory locations, regular RAM vs memory-mapped device (e.g.
>> video buffer). See, if there's emulation/virtualization that tries to
>> execute code natively whenever possible, but otherwise does it in
>> software, you may be able to spot inconsistency, just like with those
>> shift/rotate instructions.
>
> This does bring about an interesting point that one can't possibly emulate 
> all x86 instructions that access memory, because their name is legion. 
> It's not PowerPC where you could count all of them using fingers on your 
> two hands. And you do have to emulate them when you emulate MMIO. So, find 
> an area of device memory that the emulator must emulate, such as the Local 
> APIC page or the VGA buffer (assuming VGA is emulated). The emulator most 
> likely emulates all MOVs, TESTs, ORs, ANDs, bit ops, etc. But what happens 
> if you do an SSE store to that memory? If you SGDT to that memory? If you 
> SMSW to that memory? Compare that to what happens on bare hardware. 

[toc] | [prev] | [next] | [standalone]

#1407

On Tue, 03 Jun 2014 13:04:56 -0400, Wildman <best_lay@yahoo.com> wrote:

> I'm not sure how far you want to take this but something else
> occurred to me that might be of interest.  That is checking
> for a shelled process, i.e., command.com running within
> command.com.  This code will check for that.  All numbers
> are hex.
>
> Start:
>          push  ds           ; Save data pointer
>          xor   ax,ax        ; Get segment pointer to COMMAND.COM from
>          mov   ds,ax        ;   interrupt vector table and save it on
>          mov   ax,[0ba]     ;   the stack
>          push  ax
>          mov   ah,51        ; Get pointer to PSP of this process
>          int   21
>          mov   ds,bx        ; Get pointer to parent
>          mov   ax,[16]
>          pop   bx           ; Get COMMAND.COM's segment pointer
>          cmp   ax,bx        ; Should be the same as the parent
>          je    NotShelled   ; Jump if a shelled process
> Shelled:
>          pop   ds           ; Restore data pointer
>           .
>           .
>           .
> NotShelled:
>          pop   ds           ; Restore data pointer
>           .

Hey, are you still around?

First, I'm trying to make sense of the "0ba".  If "0ba" is a specific
value, what is it, or what was it supposed to be? ...

My personal notes indicate that MS-DOS v7.10 doesn't hook vectors 0bh
or 0ah.  They also indicate that 0bah is in a reserved region of the
IVT.  Also, I don't see any others which are hooked by DOS that have
an 'a' or 'b' in the interrupt vector.  I.e., any of the reasonable,
potential vectors I can think up, that would be indicated by "0ba",
seem to be unlikely to contain a segment for COMMAND.COM ...

 From what I can find, DOS supposedly returns the segment of
COMMAND.COM for interrupt vectors when returned by the Int 21h,
AH=35h function call.  This is apparently an undocumented feature.
I've not found anything indicating a direct read of a segment from
the IVT returns the segment of COMMAND.COM.

Which version and make of DOS?


 From RBIL Table 01378:

...
PSP+16h "segment of parent PSP"
...
PSP+2Ch "DOS 2+ segment of environment for process"
...

Is Int 21h, AH=51h, return value BX "segment of the PSP for current
process" any different from the value at PSP+2Ch?  Supposedly, the
segment of the current process' PSP is also passed in DS when the
DOS program starts.

Am I getting different segments mixed up here?

Do you need to compare the segment for COMMAND.COM with the segment
for the parent?  If the process is only a parent, won't the segment
of the parent be identical to the segment for the process?


Rod Pemberton

[toc] | [prev] | [next] | [standalone]

#1408

On Tue, 10 Jun 2014 17:37:10 -0400, Rod Pemberton wrote:

> On Tue, 03 Jun 2014 13:04:56 -0400, Wildman <best_lay@yahoo.com> wrote:
> 
>> I'm not sure how far you want to take this but something else
>> occurred to me that might be of interest.  That is checking
>> for a shelled process, i.e., command.com running within
>> command.com.  This code will check for that.  All numbers
>> are hex.
>>
>> Start:
>>          push  ds           ; Save data pointer
>>          xor   ax,ax        ; Get segment pointer to COMMAND.COM from
>>          mov   ds,ax        ;   interrupt vector table and save it on
>>          mov   ax,[0ba]     ;   the stack
>>          push  ax
>>          mov   ah,51        ; Get pointer to PSP of this process
>>          int   21
>>          mov   ds,bx        ; Get pointer to parent
>>          mov   ax,[16]
>>          pop   bx           ; Get COMMAND.COM's segment pointer
>>          cmp   ax,bx        ; Should be the same as the parent
>>          je    NotShelled   ; Jump if a shelled process
>> Shelled:
>>          pop   ds           ; Restore data pointer
>>           .
>>           .
>>           .
>> NotShelled:
>>          pop   ds           ; Restore data pointer
>>           .
> 
> Hey, are you still around?

Yep.

> First, I'm trying to make sense of the "0ba".  If "0ba" is a specific
> value, what is it, or what was it supposed to be? ...
> 
> My personal notes indicate that MS-DOS v7.10 doesn't hook vectors 0bh
> or 0ah.  They also indicate that 0bah is in a reserved region of the
> IVT.  Also, I don't see any others which are hooked by DOS that have
> an 'a' or 'b' in the interrupt vector.  I.e., any of the reasonable,
> potential vectors I can think up, that would be indicated by "0ba",
> seem to be unlikely to contain a segment for COMMAND.COM ...
> 
>  From what I can find, DOS supposedly returns the segment of
> COMMAND.COM for interrupt vectors when returned by the Int 21h,
> AH=35h function call.  This is apparently an undocumented feature.
> I've not found anything indicating a direct read of a segment from
> the IVT returns the segment of COMMAND.COM.

The value 0ba is written in a confusing way because the
syntax of the Magic Assembler.  It expects number to be
hex but the numbers must start with a numerical digit.
If I used just ba, the compiler would see it as a label,
not a number.

The value is an offset into the IVT.  Since ds is 0, the
address would be 0000:00BA.  This address contains the
segment of the entry point for the /first/ instance of
command.com.  It is not a vector address.  I found this
in a code snippet several years ago on a web site called
Programmers Heaven.  I'm sorry but I don't remember any
more about it.  But I do know from testing that it is
correct.

> Which version and make of DOS?

AFAIK the version does not matter.

>  From RBIL Table 01378:
> 
> ...
> PSP+16h "segment of parent PSP"
> ...
> PSP+2Ch "DOS 2+ segment of environment for process"
> ...
> 
> Is Int 21h, AH=51h, return value BX "segment of the PSP for current
> process" any different from the value at PSP+2Ch?  Supposedly, the
> segment of the current process' PSP is also passed in DS when the
> DOS program starts.
> 
> Am I getting different segments mixed up here?

When a program starts, whether it is run within the first
instance of command.com or it is shelled, it is given a
/copy/ of the environment, not the original so no, they
would not be the same.

> Do you need to compare the segment for COMMAND.COM with the segment
> for the parent?  If the process is only a parent, won't the segment
> of the parent be identical to the segment for the process?

Yes, that is exactly what you need to do, assuming the goal is
to determine if the process is running in the first instance
of command.com.  In that case, both the segment of command.com
and the segment of the parent will be the same because they are
the same.  If the segment addresses differ then it is shelled.

I hope this clears things up a little.

-- 
<Wildman> GNU/Linux user #557453
May the Source be with you.

[toc] | [prev] | [next] | [standalone]

Page 2 of 3 — ← Prev page 1 [2] 3  Next page →

Back to top | Article view | comp.os.msdos.programmer

csiph-web