Expert Tips for SCCM Log Analysis

1wearandtear

1. Locate cmtrace.exe (or another suitable “active” log viewer)
2. Open cmtrace.exe and click “yes” to register it as default log viewer
3. Consume precisely 5 quarts of a strong, caffeinated liquid substance
4. Browse to location (folder) with log files and double-click desired log file
5. Rub eyelids approximately 12 times, make sure to yawn fully and loud
6. Stare at log details and look for any lines colored in red.
7. Ignore red lines which do not actually display an error, but are instead mentioning that they’re looking for an error
8. Ignore yellow lines which do not actually display a warning, but instead show mention of looking for warnings
9. Rub eyelids 12 more times.
10. Announce to whomever interrupts that you’re busy reviewing log files (the louder the better)
11. Open another log file (selected at random)
12. Stare intently at one line, without scrolling
13. Rub chin, squint, and nod slightly. You may also say “hmmmm”
14. Scroll and repeat step 12
15. Repeat steps 10 through 13, approximately 5 more times.
16. Open browser and begin searching for fragments of error messages along with “sccm error log…”
17. Inhale deeply, exhale loudly.
18. Consume more caffeinated liquids
19. Rub eyes some more
20. Lunch break.

(Seriously) 5 Most Common SCCM Issues

Joking aside (for a few minutes anyway)…

teamamerica3

The five (5) most common root causes for SCCM site issues that I’ve seen over the past year, working as a consultant.

  • Site scale:  (smallest) 500, (largest) 180,000
  • Site types: CAS (5%), Primary alone (85%), Primary with Secondaries (5%), None (5%) aka “new install”
  • Avg staffing: (IT dept) 12-24 (SCCM admin) 1
  • Avg coffee consumption: 1 cup per 30 minutes
  • Avg sleep: 5.2 hours

1 – Lack of planning before installing the environment

In the past year alone, I’ve run across almost a dozen sites which had a CAS and didn’t need one, or Secondary sites, and didn’t need them, and so on.  Some didn’t have a FSP and could’ve used one.  Some weren’t using the appropriate credentials for client installations, network access and so on.  And lately, many seem to have pinned their plans on outdated platforms, such as Windows Server 2008 R2 or SQL Server 2012.  At least keep them patched (e.g. SQL 2012 SP3 CU9)

2 – Lack of monitoring and following-up on warnings/errors

Of the last 24 customer engagements I’ve been involved with, roughly 60% do not keep a daily watch over site issues (sites, components, clients, content distribution, deployments, etc.).  Of those that do monitor, about half ignore lingering warnings which impact site performance.

3 – Lack of cohesive management

This varies by scale/size of the organization (at least in my world).   Often it’s a matter of job roles and organizational divisions.  For example, DBA’s controlling the SQL Server environment without allowing SCCM admins any direct access (very bad).  Or AD admins who drag their feet (or push back) on requests for schema extensions, keeping AD accounts “clean” and so on.  Or Network Admins who fight back against using PXE, no matter what the rationale.  In many cases, it rolls up to team managers who don’t work well together, so resolving conflicts and barriers is difficult, especially when the CTO or CIO prefer to avoid dealing with it.  My advise: deal with it!  The good of the company outweighs your stupid personal disagreements.

4 – Lack of keeping up on updates

Whether it’s the Windows Server, SQL Server, ADK, MDT or Configuration Manager itself, all of these require persistent support and oversight. Keep them patched.  But more importantly, READ THE PATCH details first.  Understand what’s being “fixed” or “modified” (or deprecated) as well as “known issues”.  You can save yourself a shit-ton (that’s a scientific measurement, by the way) of headaches and support costs by not blindly installing without understanding.  However, do not avoid patching simply because of fear and doubt.  You work in IT, which means “change” is inevitable and continuous.  It’s why the “soft” in “software” exists (trust me, Babbage wasn’t kidding around).

5 – Inefficient use of features

This one alone could be broken out into sub-categories actually, and now that I mentioned it, I will…

a – Ignoring features which are not fully understood (not doing research)

b – Continuing to use outdated methods (disk imaging, for one, like Acronis or Ghost)

c – Ignoring other System Center capabilities (SCOM, Orchestrator, etc.)

d – Not following “best practices” (excessive permissions on common accounts, incorrect client installation settings

e – Paying for 3rd-party products which SCCM (or other System Center) capabilities could provide (depends upon the individual requirements of course)

f – Ignoring 3rd-party products out of fear of the unknown (FUD)

g – Ignoring new features added with each build (current branch), such as Azure, OMS, UA, and mobile device features

h – [my peeve] Inefficient mapping of tools to processes.  Such as ignoring Group Policy in favor of doing everything in SCCM or via scripts. Continuing to use familiar solutions even when newer and better (cheaper, faster, more efficient, more reliable) solutions are available.

i – Insufficient use of Internet search tools (Google, Bing, etc.)

Did I miss anything?

5 Tips for Fixing Broken SCCM DMZ services

080530-N-7981E-259

The following five (5) tips should help even the most seasoned SCCM expert determine the root cause for problematic DMZ environments.

Reasons you’re having trouble with your SCCM DMZ

1 – You don’t actually have a DMZ

2 – The DMZ doesn’t contain a SCCM site system, nor an AD Forest trust, nor any network connections back into the internal network.  You might also not have any SCCM clients that operate in the DMZ.

3 – You have no idea what “SCCM” or “DMZ” are.  And you don’t really care.

4 – You work in the Finance department.

5 – Why are you reading this?

Sorry – I needed a break from mind-numbing emails and phone calls today.

SCCM and Chocolatey

browsers

Trying to leverage goodness from various mixtures of Chocolatey with SCCM is definitely not new. Others have been playing around with it for quite some time. However, I wanted to pause from a month of mind-numbing work-related things to jot down some thoughts, realizations, pontifications, gyrations and abbreviations on this.

Much of this idiotic rambling that ensues hereinafter is based on the free version of Chocolatey.  There is also a “Business” version that offers many automation niceties which you might prefer.  There’s a lot more to this Chocolatey thing than I can possibly blabber out in one blog post (even for yappy little old me), such as the Agent Service features, packaging, and so more.  Visit http://chocolatey.org for more.

1 – Is it “Better”?

No.  It’s just different.  But, regardless of whether if “fits” a particular need or environment, it’s often nice to know there’s another option available “just in case”.

2 – Who might this be of use to?

I can’t list every possible scenario, but I would say that if the potential benefits are lined up it kind of points to remote users without the use of a public-facing (or VPN-exposed) distribution point resource.  It also somewhat negates the need for any distribution resource, even cloud based (Azure, AWS), since there’s no need for staging content unless you want to do so.

3 – How does SCCM fit?

At this point (build 1703) it’s best suited for use as a Package object, since there’s no real need for a detection method, or making install/uninstall deployment types.  A Program for installation, and another for uninstallation, are pretty much all that’s needed.

4 – How does an Install or Uninstall work via SCCM?

As an example, to install Git, you would make a Package, with no source content, and then create one Program as (for example only) “Install Git” using command “choco install git -y”, and another as “Uninstall Git” using “choco uninstall git -y”.  (Caveat: some packages incur dependencies, which may throw a prompt during an uninstall.  For those you can add -x before the -y, but refer to the Chocolately documentation for more details)

5 – How do you push updates to Chocolatey apps via SCCM?

You can use the above construct with a third Program named “Update Git” (for example) with command “choco upgrade git -y”.  Another option (and my preference) is to deploy a scheduled task that runs as the local System account, to run “choco upgrade all -y” at a preferred time or event (startup, login, etc.).  And, as you might have guessed by now (if you haven’t fallen asleep and face-planted into your cold pizza), someone has done this for you.

6 – Can you “bundle” apps with Chocolatey with or without SCCM?

Absolutely.  There’s a bazillion examples on the Internet, but here’s one I cobbled together for a quick lab demo a while back.  This one feeds a list of package names from a text file. You can also hard-code the list, or pull it from anywhere that PowerShell can reach it (and not just PowerShell, but any script that you can run on the intended Windows device).

7 – What about MDT?

Here’s a twist, you can deploy Chocolatey packages using MDT, or deploy MDT using Chocolatey.  How freaking cool is that?  If you sniff enough glue, you might even construct a Rube Goldberg system that deploys itself and opens a wormhole to another dimension.  By the time you find your way back, America will be a subsidiary of McDonald’s and we have real hoverboards.

8 – What about applying this to Windows Server builds?

You can.  I’d also recommend taking a look at BoxStarter, and Terraform.  I built a few BoxStarter scripts using Github Gists for demos a while back.  Here’s one example for building and SCCM primary site server, but it’s in need of dusting off and a tune up.  You can chop this up and do things all kinds of different (and probably better) ways than this.

The list of automation tools for building and configuring Windows computers is growing by the day.  By the time you read this sentence, there’s probably a few more.  Hold on, there’s another one.

PS – If you get really, really, reeeeeeally bored, and need something to either laugh at, ridicule or mock, you can poke around the rest of my Github mess.  I don’t care as long as you put the seat back down after flushing.

Interview: Julie Andreacola

wpid-wp-1409886754092.jpeg

julie

Julie Andreacola is a Premier Field Engineer with Microsoft specializing in client operating systems. For the last 15 years, she has been working for medium to large enterprises in Retail, Finance, and K-12 Education. Her past areas of focus include Configuration Manager, Application Packaging, and PowerShell scripting, In her free time, she enjoys trying out great local restaurants, travel, and cheering on the Virginia Tech Hokies with her husband, Michael.

1. Describe what you do for a living – to someone who has no idea what it means.

I help the technical teams of large enterprises with their Microsoft software. My specialty is Windows 7, Windows 10 and System Center Configuration Manager. Configuration Manager is the way enterprises setup computers and take care of all the updates and software for the user.

2. How did you get into this type of work?

This is my second career. I grew up with computers (my Dad worked for IBM). He insisted that I know how to fix the computer when it broke or didn’t work correctly so I was always learning. He sent me off to college with an IBM portable computer. It weighed 30 lbs and I still have scars on my shin from hauling that thing around. After graduating with a degree in forestry, I worked out of a pulp mill helping land owners manage their forests. Computer skills were scarce in those days, so I was often called on to fix the office computers. I shifted into manufacturing management until I chose to leave the workforce and raise my young children.

Continuing to build my computer skills, I had a part time opportunity to do helpfile work and some light coding during this time. When I returned to the workforce after being home full time with my children, I got a job with the local school system fixing the school computers. Hardware, software, networking, there was always something new to learn and I loved working with the teachers to help them leverage technology in the classroom.

When the school system installed SCCM 2007, I packaged all of the applications for deployments and walked into the world of system management. I learned PowerShell and became active in the local PowerShell user group. My next job change brought me into the world of consulting with a focus on SCCM. I recently started with Microsoft and am loving all of the amazing learning opportunities.

3. What area or aspect of technology are you most excited about?

I’m excited about all the different devices available, phones, tablets, laptops. I love how you can do work across all these devices while keeping all the content in the cloud.

4. What gives you the most satisfaction today?

Teaching or helping others be successful. I love to mentor others and watch them grow in their career.

5. Name the 3 most inspiring people in your life or career?

In my current career, my father. He expected me to know and use technology. He didn’t fix it for me, but would teach me how to troubleshoot it.

Kent Agerlund, his teachings through books, blogs and MMSMOA has really helped me learn and grow. He spends a lot of time and energy to make self-learning for Configuration Manager available for everyone.

Ed Wilson (The Scripting Guy) – I met him after a presentation for a local IT Pro event. He convinced me to come to the Charlotte PowerShell User Group. I did and loved it. Once a month, we had free pizza, great discussions, and help with any scripting questions. Whatever the topic, Ed would participate and eventually say “I wrote a blog about that.”

6. If I hadn’t gone into this field, I’d probably be… ?

Some sort of planner? maybe events or travel planning?

7. Favorite place to travel?

Mountains feed my soul, but I love traveling everywhere. In the last year had great trips to Italy and Paris.

8. What 3 books, movies or other works have inspired or influenced you most in life?

The Bible, taught me about life and my place in the world.

The Five Love Languages by Gary Chapman showed me how to have better relationships with others.

Gone With The Wind showed me a strong woman finding a way to provide for her family in a man’s world.

9. There’s never enough ____.

Time to learn all this new technology.

10. There’s way too much ____.

Hate and division in the world.

11. What are your thoughts about the roles of women in technology today? And does the discussion topic bring up hope or dread when you hear it?

I think women can be extremely effective with technology. As a community, we need to continue to reach out and mentor students and women already in the profession, especially in technical roles. While there are large numbers of women in technology, there is a small percentage who are in a purely technical role. This results in many people with a unconscious bias that the woman in the room or in the meeting is not technical.

As a technical woman, it gets old really fast to always have to work to change people’s perceptions. The continuing discussion of women in technology brings me hope. I’m hopeful the discussions will encourage women, and everyone might consider how gender bias manifests in the workplace.

12. If you could go back in time and change the course of any one, specific, area of technology, so that it turns out different today, what would it be, and why?

It would be wonderful to change history so that there are no chemical weapons. Genocide, terrorism, and war are all awful and chemical weapons are an easy weapon of evil.

On a lighter note, ink jet printers. I so wish they had never been invented. Dried up ink cartridges, terrible USB drivers, and those evil all in one machines. So cheap, so slow, so problematic, the discarded carcasses just stack up at Goodwill.

13. How do you feel about the importance of college degrees, and certifications as it pertains to IT careers? Do those credentials mean as much, or more, than they use to?

This is always a hot topic in my household. I’m a firm believer in the importance of a college degree (but don’t pay a fortune for it, be savvy in your choice). It opens doors and the process of obtaining that degree teaches a person many things that are not academic.

I do pity those organizations that require a college degree, no exceptions. This eliminates some fantastic people who have found success through a different path. For example, military service teaches many of the same life lessons learned when getting a college degree.

An organization should recruit the best people for the role regardless of academic accomplishments. Certifications don’t mean much as the experience and actual accomplishments of an individual, but they are a resume checkbox recruiters love. Certifications can get you that first interview, but you better have a thorough understanding of the product and be able to articulate it. For those in consulting, they are often a requirement of the customer.

I don’t expect this to change as it an easy way for companies to create a vendor requirement. I am interested to see how certifications will keep up with rapid change in products. A great example is the Azure certifications. With the product changing and evolving monthly, the certification test has to change to keep up. At what point does the test become a different test?

14. Will most people still be using desktop computers in 2022? Why or why not?

I don’t think most people are using desktops now, especially in the consumer space. I think they will disappear just like floppy disks and CD drives. As tablets and phones continue to become more powerful, why have a desktop that stuck in one place? Technology is racing to provide the security needed with mobile devices to make this a reality.

15. If you could transport yourself back to ancient times, like say the 1100’s AD, somewhere in Europe, and you brought along a Surface Book (with a full battery charge), and you turned it on and used it in a room full of town locals, what do you think would happen?

Since most would be illiterate, I think they would find the device confusing and frightening. The glowing display and keyboard would seem very magical. With my red hair, I would probably be labeled as a witch and killed. I think I will stay firmly in the current century clutching my Surface Book tightly because it is just that awesome!

I don’t have any links to add.  Blog in the process of getting created.

(note: I will gladly update this when the link is ready)

Interview: Mike Terrill

Introduction

Each “interview” I’ve done has been a wildly different experience. This one was impacted by logistics and direction changes.  By that, I mean both Mike and I were in the midst of major scheduling demands, however, Mike is also navigating a job change, which is always a tough thing to try to insert extracurricular activities (like half-brained bloggers nagging people with questions).

Mike is one of my short-list, go-to people for information about things related to deploying and managing Windows devices via System Center technologies.  I’m not implying that’s all he’s good for, but that’s how I discovered his blog and Twitter feed. Mike is also one of several people I tried (and failed) to meet in person during Microsoft Ignite 2016 in Atlanta, due to crazy schedules, navigational challenges, and caffeine shortage.  I’m planning to be at Ignite 2017, so I’m going to be more diligent in meeting people (those that want to meet me in person, that is).

Name: Mike Terrill

Job Title: OS Engineer

1. Describe what you do for a living – to someone who has no idea what it means.

Architect systems to deploy and manage Windows devices electronically without touching them.

2. How did you get into this type of work?

I have always been interested in computers since the day when I got my first computer – C64. I was big into cars during high school and college, and I wanted to work on automotive computer systems when I graduated. I ended up getting into systems management instead.

3. What area or aspect of technology are you most excited about?

Self-driving cars and connected cars – people are idiot drivers and half of them should not be behind the wheel. It is exciting to see the progress in this area. The Systems and Industrial Engineering Department at the University of Arizona is one of the leaders in this space and have been ever since I was there in the 90s.

4. What gives you the most satisfaction today?

Family vacations, cart racing and Megadeth concerts.

5. Name the 3 most inspiring people in your life or career?

My parents and my wife. My parents taught me how to work hard to meet my goals. My wife has helped me make my career decisions, including my recent one (but she still thinks it is called SMS since I gave up talking to her about it a long time ago).

6. If I hadn’t gone into this field, I’d probably be…

an automotive engineer with an emphasis on engine (computer) management systems. I had a Mustang GT 5.0 growing up that I not only turbocharged, but I replaced the Ford computer system with a fully programmable after market computer system that provided engine management.

7. Favorite place to travel?

Hawaii (second to that would be Mars with skatterbrainz of course).

[edit: oh, this poor guy]

8. What 3 books, movies or other works have influenced you most in life?

Millionaire Next Door, Smart Couples Finish Rich, and of course one of Kent Agerlund’s books. The first two teach basic common financial sense and these concepts should be taught in high school. Kent’s books teach Configuration Manager common sense.

9. There’s never enough …

time in the day (or money).

10. There’s way too much …

hatred (and big government).

11. What’s your favorite sound?

Dave Mustaine’s guitar.

12. What would you say to those who insist that technology has only made life worse?

I would say “What about all of the technology that is now making things possible for disabled people that previously were only a dream?” Technology has made life better, not worse. Sometimes people use technology for bad things (like the recent WannaCry ransomware), but that is a people problem, not a technology problem.

13. How do you feel about the importance of college degrees, and certifications as it pertains to IT careers?

Do those credentials mean as much, or more, than they use to? Yes and no. There is still a need for advanced education. Formal college degrees and certifications show one’s desire and persistence to complete a goal (and hopefully learn in the process). However, there are plenty smart individuals out there that do not have formal education. As for IT, it is a mixed bag. Some people happened to fall in to IT as a career and probably should be in a completely different field (skatterbrainz knows what I am talking about – just read his tweets from his projects).

[edit: omg – I promise I did not bribe anyone for that mention]

14. You’ve crashed on a remote island along with 4 other engineers, and 5 sales people. There’s only enough food for five people total to last a week. What do you do?

The 5 sales people will end drowning since they believe they can walk on water. That will leave enough time for the engineers to come up with a plan and get off the island.

[edit: there is an alternate theory that the sales people will ultimately survive by way of verbally ‘synergistically envisioning’ the engineers to death]

15. If you could go back in time and change one piece of technology to end up better today, what would it be, and why?

Hmm…this is a tough one, but it would probably be with something that helps people or saves lives.

More about Mike

Mike’s twitter profile

Mike’s blog

Arizona Systems Management Users Group