Commit graph

79 commits

Author SHA1 Message Date
rsc
c8919e6537 kernel SMP interruptibility fixes.
Last year, right before I sent xv6 to the printer, I changed the
SETGATE calls so that interrupts would be disabled on entry to
interrupt handlers, and I added the nlock++ / nlock-- in trap()
so that interrupts would stay disabled while the hw handlers
(but not the syscall handler) did their work.  I did this because
the kernel was otherwise causing Bochs to triple-fault in SMP
mode, and time was short.

Robert observed yesterday that something was keeping the SMP
preemption user test from working.  It turned out that when I
simplified the lapic code I swapped the order of two register
writes that I didn't realize were order dependent.  I fixed that
and then since I had everything paged in kept going and tried
to figure out why you can't leave interrupts on during interrupt
handlers.  There are a few issues.

First, there must be some way to keep interrupts from "stacking
up" and overflowing the stack.  Keeping interrupts off the whole
time solves this problem -- even if the clock tick handler runs
long enough that the next clock tick is waiting when it finishes,
keeping interrupts off means that the handler runs all the way
through the "iret" before the next handler begins.  This is not
really a problem unless you are putting too many prints in trap
-- if the OS is doing its job right, the handlers should run
quickly and not stack up.

Second, if xv6 had page faults, then it would be important to
keep interrupts disabled between the start of the interrupt and
the time that cr2 was read, to avoid a scenario like:

   p1 page faults [cr2 set to faulting address]
   p1 starts executing trapasm.S
   clock interrupt, p1 preempted, p2 starts executing
   p2 page faults [cr2 set to another faulting address]
   p2 starts, finishes fault handler
   p1 rescheduled, reads cr2, sees wrong fault address

Alternately p1 could be rescheduled on the other cpu, in which
case it would still see the wrong cr2.  That said, I think cr2
is the only interrupt state that isn't pushed onto the interrupt
stack atomically at fault time, and xv6 doesn't care.  (This isn't
entirely hypothetical -- I debugged this problem on Plan 9.)

Third, and this is the big one, it is not safe to call cpu()
unless interrupts are disabled.  If interrupts are enabled then
there is no guarantee that, between the time cpu() looks up the
cpu id and the time that it the result gets used, the process
has not been rescheduled to the other cpu.  For example, the
very commonly-used expression curproc[cpu()] (aka the macro cp)
can end up referring to the wrong proc: the code stores the
result of cpu() in %eax, gets rescheduled to the other cpu at
just the wrong instant, and then reads curproc[%eax].

We use curproc[cpu()] to get the current process a LOT.  In that
particular case, if we arranged for the current curproc entry
to be addressed by %fs:0 and just use a different %fs on each
CPU, then we could safely get at curproc even with interrupts
disabled, since the read of %fs would be atomic with the read
of %fs:0.  Alternately, we could have a curproc() function that
disables interrupts while computing curproc[cpu()].  I've done
that last one.

Even in the current kernel, with interrupts off on entry to trap,
interrupts are enabled inside release if there are no locks held.
Also, the scheduler's idle loop must be interruptible at times
so that the clock and disk interrupts (which might make processes
runnable) can be handled.

In addition to the rampant use of curproc[cpu()], this little
snippet from acquire is wrong on smp:

  if(cpus[cpu()].nlock == 0)
    cli();
  cpus[cpu()].nlock++;

because if interrupts are off then we might call cpu(), get
rescheduled to a different cpu, look at cpus[oldcpu].nlock, and
wrongly decide not to disable interrupts on the new cpu.  The
fix is to always call cli().  But this is wrong too:

  if(holding(lock))
    panic("acquire");
  cli();
  cpus[cpu()].nlock++;

because holding looks at cpu().  The fix is:

  cli();
  if(holding(lock))
    panic("acquire");
  cpus[cpu()].nlock++;

I've done that, and I changed cpu() to complain the first time
it gets called with interrupts disabled.  (It gets called too
much to complain every time.)

I added new functions splhi and spllo that are like acquire and
release but without the locking:

  void
  splhi(void)
  {
    cli();
    cpus[cpu()].nsplhi++;
  }

  void
  spllo(void)
  {
    if(--cpus[cpu()].nsplhi == 0)
      sti();
  }

and I've used those to protect other sections of code that refer
to cpu() when interrupts would otherwise be disabled (basically
just curproc and setupsegs).  I also use them in acquire/release
and got rid of nlock.

I'm not thrilled with the names, but I think the concept -- a
counted cli/sti -- is sound.  Having them also replaces the
nlock++/nlock-- in trap.c and main.c, which is nice.


Final note: it's still not safe to enable interrupts in
the middle of trap() between lapic_eoi and returning
to user space.  I don't understand why, but we get a
fault on pop %es because 0x10 is a bad segment
descriptor (!) and then the fault faults trying to go into
a new interrupt because 0x8 is a bad segment descriptor too!
Triple fault.  I haven't debugged this yet.
2007-09-27 12:58:42 +00:00
rsc
c1b100e930 nits 2007-08-28 18:23:48 +00:00
rsc
9e82bfb04c rename 8253pit.c to timer.c 2007-08-28 04:40:58 +00:00
rsc
43baa1f224 nit 2007-08-28 04:14:32 +00:00
rsc
3341e30f6e nit 2007-08-28 04:13:24 +00:00
rsc
19b42cc078 Rename main0 to main. 2007-08-27 23:32:16 +00:00
rsc
558ab49f13 delete unnecessary #include lines 2007-08-27 23:26:33 +00:00
rsc
99b11b6c64 Simplify MP hardware code.
Mainly delete unused constants and code.

Move mp_startthem to main.c as bootothers.
2007-08-27 22:53:31 +00:00
rsc
b63bb0fd00 Clean up lapic code.
One initialization function now, not three.
Use #defines instead of enums (consistent with other code, but sigh).

Still boots in Bochs in SMP mode.
2007-08-27 16:57:13 +00:00
rsc
124f32ae38 tweak 2007-08-24 19:36:52 +00:00
rsc
eaea18cb9c PDF at http://am.lcs.mit.edu/~rsc/xv6.pdf
Various changes made while offline.

 + bwrite sector argument is redundant; use b->sector.
 + reformatting of files for nicer PDF page breaks
 + distinguish between locked, unlocked inodes in type signatures
 + change FD_FILE to FD_INODE
 + move userinit (nee proc0init) to proc.c
 + move ROOTDEV to param.h
 + always parenthesize sizeof argument
2007-08-22 06:01:32 +00:00
rsc
0073beee52 remove dead code 2007-08-21 19:22:27 +00:00
rsc
f32f3638f4 Various cleanup:
- Got rid of dummy proc[0].  Now proc[0] is init.
 - Added initcode.S to exec /init, so that /init is
   just a regular binary.
 - Moved exec out of sysfile to exec.c
 - Moved code dealing with fs guts (like struct inode)
   from sysfile.c to fs.c.  Code dealing with system call
   arguments stays in sysfile.c
 - Refactored directory routines in fs.c; should be simpler.
 - Changed iget to return *unlocked* inode structure.
   This solves the lookup-then-use race in namei
   without introducing deadlocks.
   It also enabled getting rid of the dummy proc[0].
2007-08-21 19:22:08 +00:00
rsc
bcca6c6bde shuffle fs.c in bottom-up order 2007-08-20 18:23:52 +00:00
rsc
8c4b5fc5b3 Gcc expects to be able to pick up the return
address off the stack, so put one there for it.
(Bug was hidden by bad segment limits.)
2007-08-14 04:56:30 +00:00
rsc
dca5b5ca2e avoid assignments in declarations 2007-08-10 17:17:42 +00:00
rsc
c664dd5d23 missing void 2007-08-08 09:32:39 +00:00
rsc
5d0fe3445b more bugs 2007-08-08 09:10:16 +00:00
rsc
b6dc6187f7 add DPL_USER constant 2007-08-08 09:02:42 +00:00
rsc
f83f7ce2f6 set init name 2007-08-08 08:57:55 +00:00
rsc
4fb684548a formatting nits 2006-09-08 15:14:43 +00:00
rsc
efb01c1dc0 only need a page 2006-09-08 15:09:48 +00:00
kaashoek
5cb7877e0f use bootstrap processor as specified by MP table. typically 0, but not
guaranteed.
2006-09-08 14:48:07 +00:00
kaashoek
8e1d1ec934 some comment changes 2006-09-08 14:36:44 +00:00
rsc
7e019461c8 fix build 2006-09-07 14:10:52 +00:00
kaashoek
e00baa9f5d get precedence of <, >, and | right
simplify
2006-09-07 02:15:28 +00:00
kaashoek
f70172129c run without lapic and ioapic, if they are not present
if no lapic available, use 8253pit for clock
now xv6 runs both on qemu (uniprocessor) and bochs (uniprocessor and MP)
2006-09-07 01:37:58 +00:00
rsc
50e514be98 fd_* => file_* 2006-09-06 18:43:45 +00:00
rsc
9e9bcaf143 standardize various * conventions 2006-09-06 17:27:19 +00:00
rsc
a650c606fe spacing fixes: no tabs, 2-space indents (for rtm) 2006-09-06 17:04:06 +00:00
rtm
dfcc5b997c prune unneeded panics and debug output 2006-08-29 19:06:37 +00:00
rtm
2b19190c13 clean up stale error checks and panics
delete unused functions
a few comments
2006-08-29 14:45:45 +00:00
rtm
ceb0e42796 proc[0] can sleep(), at least after it gets to main00()
proc[0] calls iget(rootdev, 1) before forking init
2006-08-16 01:56:00 +00:00
rtm
350e63f7a9 no more proc[] entry per cpu for idle loop
each cpu[] has its own gdt and tss
no per-proc gdt or tss, re-write cpu's in scheduler (you win, cliff)
main0() switches to cpu[0].mpstack
2006-08-15 22:18:20 +00:00
kaashoek
69332d1918 oops 2006-08-15 15:54:53 +00:00
kaashoek
e958c538fa commented out code for cwd 2006-08-15 15:53:46 +00:00
rtm
9e5970d596 link() 2006-08-13 02:12:44 +00:00
rtm
17a856577f init creates console, opens 0/1/2, runs sh
sh accepts 0-argument commands (like userfs)
reads from console
2006-08-11 13:55:18 +00:00
rtm
5be0039ce9 interrupts could be recursive since lapic_eoi() called before rti
so fast interrupts overflow the kernel stack
fix: cli() before lapic_eoi()
2006-08-10 22:08:14 +00:00
rtm
8a8be1b8c3 low-level keyboard input (not hooked up to /dev yet)
fix acquire() to cli() *before* incrementing nlock
make T_SYSCALL a trap gate, not an interrupt gate
sadly, various crashes if you hold down a keyboard key...
2006-08-10 02:07:10 +00:00
kaashoek
6fa5ffb56f devsw
checkpoint: write(fd,"hello\n",6) where fd is a console dev almost works
2006-08-09 16:04:04 +00:00
rtm
0e84a0ec6e fix race in holding() check in acquire()
give cpu1 a TSS and gdt for when it enters scheduler()
and a pseudo proc[] entry for each cpu
cpu0 waits for each other cpu to start up
read() for files
2006-08-08 19:58:06 +00:00
kaashoek
c8b29f6d03 better interrupt plan---this one appears to work
ioapic
2006-08-04 18:12:31 +00:00
rtm
32630628a9 open() 2006-07-29 09:35:02 +00:00
rtm
c59361f143 primitive exec 2006-07-27 21:10:00 +00:00
rtm
2927081628 uint32_t -> uint &c 2006-07-20 09:07:53 +00:00
rsc
b5f17007f4 standarize on unix-like lowercase struct names 2006-07-17 01:58:13 +00:00
rsc
b5ee516575 add uint and standardize on typedefs instead of unsigned 2006-07-17 01:52:13 +00:00
rsc
6e6a1dd7d7 various little fixes that should have been in earlier checkins 2006-07-16 16:06:03 +00:00
rsc
8a7eb80e47 fix main return type 2006-07-16 16:03:51 +00:00