Bug 243533 - vt_fb.c can overwrite frame buffer bounds if stride length is not a multiple of bytes-per-pixel
Summary: vt_fb.c can overwrite frame buffer bounds if stride length is not a multiple ...
Status: New
Alias: None
Product: Base System
Classification: Unclassified
Component: kern (show other bugs)
Version: CURRENT
Hardware: Any Any
: --- Affects Only Me
Assignee: freebsd-bugs mailing list
URL:
Keywords: patch, vt
Depends on:
Blocks:
 
Reported: 2020-01-23 00:44 UTC by Thomas Skibo
Modified: 2020-01-26 02:10 UTC (History)
0 users

See Also:


Attachments
fix vt_fb_blank(). (1.14 KB, text/plain)
2020-01-23 00:44 UTC, Thomas Skibo
no flags Details
fix vt_fb_blank(). (955 bytes, text/plain)
2020-01-23 16:57 UTC, Thomas Skibo
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Thomas Skibo 2020-01-23 00:44:15 UTC
Created attachment 210977 [details]
fix vt_fb_blank().

I'm developing a frame buffer driver for hardware using 3 bytes per pixel but the hardware requires the stride to be a multiple of 256 bytes.  Because the stride is not a multiple of 3 bytes, the way vt_fb_blank() is coded, it writes past the end of each stride and, on the last line, writes past the end of the frame buffer.  This is caught by a KASSERT in vt_fb_mem_wr1().

I think the loops in vt_fb_blank() could just stop at the end of the line (fb_width) instead of clearing memory all the way to the end of a stride.  The other way would be to limit the loops with fb_stride - 1, fb_stride - 2, fb_stride - 3 for the cases of 2,3,4 bytes per pixel.
Comment 1 Thomas Skibo 2020-01-23 16:57:47 UTC
Created attachment 210991 [details]
fix vt_fb_blank().
Comment 2 Thomas Skibo 2020-01-23 16:59:08 UTC
Comment on attachment 210991 [details]
fix vt_fb_blank().

My previous patch was wrong.  fb_width is the width in pixels, not bytes.  This was my other suggested fix.